Look into eminent domain rules. My understanding is that if you have had your fence up for at least 10 years, you’ve established an easement & they can’t force you to take it down.
My sweetie had a piece of property years ago, where a neighbor came over & was getting ready to put a fence post in the middle of what was his driveway. The driveway was there when he bought the property & had been for years. He told the neighbor ‘no’ & it stuck. I hope you won’t have to pay for a lawyer, but if the other side pushes you might have to find one.
Eminent domain is when some government entity takes your property and puts it to public use. Adverse possession is what happens when a status quo use goes unchallenged for a long period of time.