Believe me, it got very bad- very quickly. You are quite correct in your hypothesis. Saw it first hand.
You and some of your neighbors/buddies need to strike out for some podunk town and do your damnedest to assimilate. Get out now while the gettin’s good.
You can find houses out there for $20K in some little non-descript berg, and probably within 45 minutes of a decent airport or Amtrak (just in case you need to revisit civilization.)
There’s no point in going real hermit, like deep Montana.
Hell, if you guys could pool enough money...you could buy your own town...it’s happened before.