The introduction of drugs to incapacitate the victim completely invalidates your viewpoint. Furthermore at no point is it justified to force anything on another person regardless of whatever transpired beforehand.
I agree that all people, men and women both, need to take responsibility to keep themselves out of situations that put them at risk, be it unwanted sexual activity or other bad situations. If every female that every consented to making out at the prom had to engage in intercourse afterward based on the discretion of her date very few would graduate high school without getting knocked up.
No, it doesn't.
If the drugs were taken willfully, you are culpable of disconnecting your will from any actions that happen, whether they happen to you, or whether you are the actor. If you are unwittingly drugged, what was the situation? Did you go into someone's apartment lair who you knew wanted more from you than a casual relationship? Did he offer you a drink there? Why would you accept from such a person?
There are a LOT of decisions a "victim" makes leading up to the final crime. Don't discount that.