Is your community surrounded by razor wire and machine gun towers?
I see your point. Am I free to leave? Yes.
In a constitutional republic, the states could hold different laws, with the fed taking a limited, constitutional role. If that were still the case, we could establish libertarian style communities in the US and I could go to one.
But that's not the case now, is it? Since there are no more 'communities' in the sense of the 9th and 10th Amendments, we would have to say that in practice, the entire US is one big community. And yes, it is surrounded by armed guards.
And yes, if it becomes too restrictive, and fails to observe even the most basic of rights, I will leave.