The way to control things such as that is to have zoning and covenants to not allow such a building that is so radically out of place with everything else.
However, having covenants and zoning stomps all over the property rights of the individual. I oppose covenants and zoning in almost all cases. Having lived in a neighborhood with a practicing "community association" board, I would never do so again.
Those restrictions usually come from the people in the McMansions.
Add a homeowners asso. and you have a real nightmare. Soon they will tell you to take your flag down.
So don't buy a house where there are covenants and zoning.
Instead, buy in a neighborhood where anything goes.
Just today we were discussing this very thing. The regional Boy Scout council has purchased a large parcel of land adjacent to its current camp. It's zoned agricultural, and they recently tried to get the zoning changed to accommodate Scout troops camping on it, development of hiking trails, and the building of a primitive cold-weather cabin.
Neighbors showed up at the zoning hearing and complained so vehemently about the Boy Scouts using the property (that THE BSA BOUGHT, PAID FOR, AND OWNED) that severe restrictions have now been placed on the use of the parcel. They cannot park a vehicle on it. They cannot have more than two (2) people on it simultaneously. They cannot build so much as a campfire on it, much less a cabin. My mouth was hanging open during the recitation of these restrictions, and I was sure it must be a joke. But no, the Scoutmaster assured me, it was no joke. Chalk it up to one more instance of absurdly restrictive "covenant" requirements enforcing ridiculous prohibitions on someone else's private property. (I mean, really, who could object to NATURE TRAILS being cultivated on the land, for crying out loud? And this is in a rural area, not some suburban neighborhood!)
You enter into covenants voluntarily when you buy a piece of property. Similarly, you know about zoning laws when you buy a property, so you are consenting to those rules, too.
It's nice to say that zoning is evil, but the alternative is allowing construction to occur without taking into account population densities, the ability of infrastructure to handle the new development, the effect such development will have on neighboring properties etc.