You will be fine.
I was a federal employee (before retirement) and I decided I wanted to move south. More precisely, my wife and I agreed on a go/no go strategy before buying a house.
I made the final list for seven different federal jobs in the south in a three month period (my qualifications are excellent).
I was chosen for none of them—they all were awarded to minority women.
So—we decided ok—we stay up north where we are and buy a house way out in the boonies. We are still there and everything is great.
It all works out..
A house out in the boonies. Man would I love to be able to live that dream!