"I don't want to spend years waiting and remaining pure, only to have my husband pass on a gift from one of his ex-lovers.
It's a concern but you shouldn't base a decision like marriage on this possibility. Seriously, what are you going to do? A sexual background check on this person you've fallen in love with?
"Or be secretly thinking I'm not as hot as he girl he knew freshman year of college."
This is a concern too but if he asks you to marry him, chances are he loves you. On the other hand, don't expect your man to give up fantsy after marriage. That's unrealistic.
Most of the girls who ARE labeled "slut" are not actually promiscuous. Most of the girls who get called "slut" in high school are either girls who are not popular or who have pissed someone off. Once you get the reputation you can't get rid of it, and you rarely get the reputation on account of anything slutty that you actually do. "Slut" is just a word people use to hurt girls they don't like because they know that once a girl is labeled "slut" men will regard her as "fair game" and other girls won't associate with her.
Women will never have a fair shake in this world until they learn to defend one another and not accept judgement on the grounds of their sexual choices... you can sleep with 100 guys, but if you carry yourself like a queen and expect no less, you will be treated like a queen-- but if you are called "slut" and you cringe because maybe once you let a guy go to third base, everyone will treat you like a whore and if you are in high school, you may very well be raped...