Actually, I think you have it reversed. A state could not make it illegal for residents of other states to possess marijuana while within that state's boundaries while making it legal for its own residents.
That is definitely true under the reading, but the reverse is also true.
"The Citizens of each State shall be entitled to all Privileges and Immunities of Citizens in the several States"
Maybe I am reading this differently. If I live in "each State" which would mean any State, then I would be entitled to the priveledges and immunities of citizens in any other State. A state can not make something a crime which is solely dependent upon the "offender" simply possessing and object or engaging in an activity while being located in a specific geographic area. There was much discussion of this concept in early America. To not respect this, would mean that all "travelers" would have to know the laws of anywhere they pass through, and could be incarcerated for simply doing something that was fine and dandy where they came from.