I've lived on a rural, 400 acre farm and I've lived in the suburbs. I love living in rural setting, but it's so much different than in the suburbs. If you live in the county, you don't need HOAs because you're so far from your neighbors that it's not a big deal. But if I'm in the suburbs, fifty feet from my neighbor, you better believe that I want an HOA. I have no desire for the value of my home to fall because my neighbor is too lazy to mow is lawn or because he's got his Camaro up on blocks in his drvieway.
I live next to a neighborhood that, while it has nice homes, doesn't appear to have a very effective HOA. It's a relatively new development and it doesn't seem to require homeowners to sod their lawns. As a result, there are a lot of lawns that have weeds and sparse, uneven growth. I don't want to live next door to my neighbor's crappy lawn because he was too cheap to spend a few grand on sod for his yard. Don't want any part of it.
Then you should move back to the country. Or build a privacy fence.
His property is HIS property.