There's Falcon AFS (I think they call it Schriever these days)...but that's east of the Springs. And then there's Buckley AFB...but that's all the way up in Aurora. Do you think you might be thinking about one of those places? (They both have their share of "nondescript" buildings...and they both host NRO organizations)
It has been a while. I think at the time it had NRO on it.
It was a metal frame building with reflective silver glass.
Seems like it was on the East side of the highway on the North end of Colorado Springs.
But remember, that was probably in 1986. My memory at that range is not that good.