I do not know when you were there. I was there in the mid 70s when it was being remodeled. It is in a poor part of town. It was not a dump with crumbling buildings when I was there, but I can not account for what the Carter years or Clinton years may have done to the facility.
I guess my point is that it's not like the buildings are sacred or anything. They're just brick and mortar.
More important is that it's in an extremely high cost area, and that a better facility elsewhere will cost far less. In many ways, those who are assigned to those locations are the ones who take it in the chin with the cost of living issues. The same for the New Jersey bases....especially Monmouth. The same for Atlanta's Ft MacPherson.
Just buildings.
If you are speaking of Walter Reed it is not in a poor part of town. Its between 16th street and Ga, Ave, actually a better part of DC. But when you think of moving the facilities out to the Naval Center at Bethesda its not that far , only 7 or 8 miles. Why have multiple facilities when both are so close? If you think Walter Redd is in a bad part of town I would gladly take you on a tour of the really bad parts of DC. They make Walter Reed look like Westchester.