So in 1830, Madison revisited his pre-constitutional question, the Great Desideratum. How to design a government strong enough to meet its objects, yet structured such that it didn't violate our natural rights?
Exactly. The Founders original design was brilliant. They limited the administrative powers to be exercised by the federal organ outside its physical area of jurisdiction via enumeration while simultaneously giving it full, local, municipal authority over the area inside it.
Unfortunately, the prevalent idea today is that everyone in the nation is subject to everything any branch federal government does.