Perhaps I can explain it better this way:
A close reading of the Constitution and the Founders' other writings (including the Federalist and Anti-Federalist papers) gives me the understanding that the Constitution in essence gives the government no more authority to act than we as individuals possess. We, as individuals, may properly set rules for ourselves and our families, even complete with punishments for those who break those rules. We may properly defend ourselves and our families and property with whatever amount of force we consider needful, even deadly force. We have also banded together to establish a COMMON defense, through the Constitution.
However, we cannot give any government one whit of authority which we, ourselves, may not legitimately exercise. We cannot, for example, give government the LEGITIMATE authority to ban or prohibit voluntary behaviors of others when done privately. We, as individuals, do have the proper and legitimate authority to act when something is done in public which could have a negative impact on ourselves or others and we can properly delegate this authority to government. We can even act when something is done by another IN PRIVATE when we have very good grounds to believe that it involves the INVOLUNTARY participation of another person, as for example a rape or attempted murder. This authority we can also delegate.
We can delegate no other authority to government, as it is NOT OURS TO GIVE. Sadly, FedGov and the Several States have taken on powers they were prohibited from exercising and we have not stopped them. Some folks, even here on FR, actually cheer on government for so vastly exceeding its proper and legitimate authority. I suspect you are not one of those, but you do seem to have some sort of majoritarian tendencies which need to be curbed.
I think we're in about 95% agreement on that with a few exceptions.
If I understand you correctly. You believe it is unconstitutional to put restrictions on someone making TNT for their own private use in the apartment above you. What if they desire to work with the small pox virus in the privacy of their home?
Could it be constitutionally acceptable to you to limit those private actions based on the risk factor of harm to others, despite the fact that no harm might occur?
Answering that will help me understand your position a little more exactly. I take it that you are not a strict adherent to the premise that actions not resulting in harm are uninfringable, but rather agree that the criminalization of going 120mph through a school zone can be constitutionally illegal, even if no child is struck.