It’s from an old 1960s hippie song “Don’t Bogart that joint, my friend, pass it over to me.” The idea was that you were beibg selfish, like mean ‘ol Humphrey Bogart, if you kept toking on the doob instead of passing it around.
Don’t Bogart That Joint My Friend
The short-lived Fraternity of Man is undoubtedly best known for the pro-pot anthem "Don't Bogart Me," which showed up during an unforgettable scene in the genre-defining biker film Easy Rider (1969).
The song sound borrowed heavily from The Byrds.