I lived there for 3 1/2 years in the 80's. You have no idea how nice the place is. Two forts in great shape, victorian homes, tree lined streets. Amazing place.
As to toxic waste, it didn't stop Reagan from hosting the Statue of Liberty celebration from there.
It is a great place, I'm just remembering the reason that the US Gov't couldn't sell the place to developers who wanted that great location. I think the golf course is still there and open.