That is an interpretation "privileges and immunities" worse than the current thinking on the Commerce Clause. It is a very good thing that what you think is of no consequence.
What you apparently do not understand is the role of Natural Law competition among the States with which to determine the correct set of actions and statutes. Some will liberalize their drug laws. Others will tighten them. There will be smuggling across State lines. The States will then be forced to consider an Amendment granting the Federal government narrowly defined powers to regulate or prohibit drug trafficking. That's how it works. In the mean time, we'll all learn a lot about the damage liberalization does while the laboratory of Federalism will point out and develop the most cost effective means to deal with drugs altering perception or cognitive function.
Now I'm no drug libertarian by any means. Personally, I prefer executing people who sell drugs to children or commit a crime resulting in grievous bodily injury while intoxicated. Simultaneously I prefer legalizing drug resorts with the owners equally responsible for the consequences of intoxication outside their property. The point is that somebody other than government must be accountable for people who deliberately take actions to put themselves in a state of mind for which they cannot control what they do. That somebody had best be able to afford the scope of those consequences or pay the ultimate price. No more sugar-coating the reality of intoxication.
Now some might not prefer those rules for their State, to which I say 'go ahead and live in a State placing no value upon sobriety.' Let that State go to hell and see what happens. Federalism limits the scope of such damages and provides the object lessons in bad governance for all to witness while uplifting good ideas and their progenitors with them. Such is how we get better statesmen.
Hence, I am an unalterable opponent of your incontinent reliance upon the one-stop shop for one-size-fits-all, unionized, lawyerized, bureaucratic influence-peddling police state that has become the District of Criminals.