Sure, with the Internet nowadays, people don't have to move around as much as they used to, but really, would that stop government officials from spending money on travel?
If you moved an agency to Philadelphia or Baltimore or Richmond or Wilmington that would just strengthen the liberal Democrat vote in the East.
That wouldn't change anything.
Move government agencies further away, and you'd have officials constantly shuttling back and forth to Washington DC.
Bitter bureaucrats exiled to Boise or Bismarck would likely circle the wagons and be even more hostile to outsiders.
Of course you would gave a problem with the best swamp draining idea - ever.