Funny thing, I have a severe allergic reaction to rose oil extracts perfumes, cremes, etc. However real roses do not affect me in any way. In fact one of my hobbies for years has been growing roses. Weird.
Anyway, the agent is what I am alergic to, not the scent.
Sounds like you - rose oil is expensive, so it is typically diluted with the carrier agent...