I'm too young to have seen Penn Station in its glory but I'm well aware of the atrocity committed there and seen the old photos - should have been made a landmark.
Like I said earlier, Grand Central is a real treasure. When I work out of my Lexington Ave office, I always take a midday walk around the city. But when it's raining, I go to Grand Central and walk around there. I've discovered pretty much all the passages that lead to 48th St, Madison Ave, the subways, etc., so that I can take a 3-4 mile walk without having to step outdoors and without having to recross my steps very much.
The great thing about NYC, is that there are quite a few underground passages, that one can use when it rains/snows, and not get wet.
As bad as NYC was about preservation, Chicago was much worse! So many wonderful architecture/buildings have been lost in both cities; sadly. :-(
Still and all, at least NYC has managed to keep quite a lot of its treasured old buildings.