You have to have some regulation of public commerce. Otherwise someone may build a 400 foot high wall around his property except for access. Is that fair ? If I'm you neighbor can I build a mother of all walls on my land, blocking out all sunlight from your place. Am I allowed to use my house as a halfway house for the insane ? Am I allowed to grow pot on my land ? Can I as your neighbor, build a oil refinery on my land ?
The answer, in our current political environment, is yes if you have the political clout. If it were not yes, then progress would be slowed to a crawl and then a complete stop. The crime here is that everyone does not have these rights, regardless of political connections, except of course where they have been contractually restricted.
Go right ahead, our house is situated in the middle of our property, 26 acres.
Am I allowed to use my house as a halfway house for the insane ?
Only if you are a mental hospital.
Am I allowed to grow pot on my land ?
Pot is a controlled substance.
Can I as your neighbor, build a oil refinery on my land ?
Go for it, unfortunately it would just be a waste because all the mineral rights in this area were sold or given away 100 years ago.
I still believe and will always believe in 'personal property rights'.