Smells brings memories. Its not such a bad thing. Ever been to a temple in India?
I really like the smell of incense, but the food offerings, sometimes get too ripe. Like you said, smells bring back memories and we associate events with them. I used to go to Glasgow a lot, and that place had a very distinctively unpleasant smell almost all over the place. Living most of my life near the water on the east coast of the US, I guess I’m spoiled by relatively clean air.