The Commerce Clause does not apply to pot grown in Oregon by an Oregonian and sold in Oregon to Oregonians.
I'm sure you'll agree that if we let Oregonians grow and sell pot, we have to let everyone else, right? So assuming we have 1-2 million people growing pot in their backyards, do you think some of that pot might make it into interstate commerce?
Given that it's legal in the states themselves, the only time the federal government could intervene is the exact instant it crosses state lines, right? On either side of the state line it's legal, and no one knows if the pot came from out-of-state.
Now, you may be grinning and giggling at the fact that you think you found a way to scam the system and undermine Congress' efforts (which reflect the will of the people, by the way). But the Founding Fathers anticipated this.
The Necessary and Proper Clause in the U.S. Constitution gives Congress the power to regulate those in-state activities that have a substantial effect on their interstate regulatory efforts. Without that power, you can see how Congress can be subverted.
So, it is the Necessary and Proper Clause that controls in-state activities, not the Commerce Clause.
...which construction would permit Federal regulation of everything down to the length of our toenails, should the legislature see fit to pass such a law or to hand the regulatory authority over to an administrative bureau, for that matter.