I really think that people should think about the issue in broad terms, as I think you have highlighted. Basically, I see the "sensitive people" thinking like this:
You did something at home.
Now you smell.
I don't like it.
I demand that you change.
Some people would nod if they think the topic is using Perfume as someone gets ready for work. But if the topic is foreign cuisine, then all of a sudden the "sensitive" person looks more like an Oppressor.
I think people ought to stop trying to control others. Perfume isn't illegal. You ought to be able to wear it in the workplace.
I use to work in a factory that had a few dozen Filipinas. They would break out bagon (spelling) which is fermented rotten shrimp. The break room was vomitous.