Not crazy at all. Given a choice, most citizens would abolish property tax. The fact is that the entities that benefit from property taxes (schools, county, & local govt, etc) hold greater power over our legislators than the broad constituency. There is no such thing as home ownership so long as the government can impose property tax and can sell your home if you fail to pay.
Property taxes are only one part. School taxes are separate battle.