True, but the principles involved are flawed. Our legal system operates under the premise that you are innocent until proven guilty. The Commerce Clause gives Congress the power to define what is or is not legal in matters involving interstate commerce. Prohibitions based on Commerce Clause authority imply that the intent to engage in interstate commerce is assumed, without having to prove it.
Possession of contraband can be a crime in itself. That goes back to the earliest days of our nation.