Well, now we're into something beyond interstate commerce.
If an intrastate activity substantially affects Congress' interstate regulatory efforts, then Congress has the power, under the Necessary and Proper Clause, to legislate that intrastate activity. Without that power, states and individuals could undermine and subvert Congress. Surely you don't think that's what the Founding Fathers had in mind?
"I fail to see how patients smoking pot under California law in California, or killing themselves with a doctor's assistance in Oregon under the medical laws of that state, will "destroy or impede interstate trade" or represent a "threat to my safety and welfare" way over here in Florida."
Hmmmmm. That's a condition that seems to affect only those who are in favor of legalizing drugs.