Because the restrictions are not in the Commerce Clause. They are in Section 9 of Article I, and in several of the Bill of Rights and later amendments. Those, being amendments, override any power granted Congress in the main body of the Constitution, when their is a conflict.
Thus Congress can't regulate interstate commerce in such a manner so as to abridge "the freedom of speech, or of the press", or to infringe on "the right of the people to keep and bear arms."
Well, there you go. Congress cannot regulate all guns out of existence because that would infringe on "the right of the people to keep and bear arms".
Which answers tacticalogic's question.