Actually, the grounds at St. E's are very nice. Landscaped since the 1870's. The exterior court scenes for "A Few Good Men" were filmed there.
Never been on the Hospital's grounds but have been all over the Navy Yard, Naval Station and Bolling. Would not stray too far from the main roads though.
"...Actually, the grounds at St. E's are very nice....."
Well, not right at the moment. I was there last September doing some work, and there was massive construction going on. Mud, rebar, and cement as far as the eye could see. It was a mess.