Few people are allergic to Goldenrod, even though many people say they are. In general, they are actually allergic to ragweed, which blooms at the same time - but the goldenrod is much brighter and more visible.
...So as you surmise, you could be allergic to something associated with the Cedar groves, rather than the Cedar itself.
Thankfully, being exposed 24/7 to whatever it is is building an immunity.
They say eating poison ivy will make you immune to it. Uh, no thanks.