I agree with you. But, speaking as a lawyer (i.e., telling you what the law is and not what I think it should be), the current interpretation is that Congress can regulate all commerce, even if it's not interstate (e.g., minimum wage laws), and can regulate anything interstate, even if it's not commerce (e.g., possession of a gun made in another state) . But Congress can generally not regulate something that's neither (Lopez and the Affordable Care Act cases say that, but Raich seems to contradict even that limit).
Seems to me that those interpretations walk rough-shod over the 10th Amendment. How do they get around the 10th Amendment? Where are those powers enumerated for the federal government in the Constitution?