I think Fleming starts wrong with his question, "Freedom to do what?" That's the wrong question because it treats freedom as a positive quality, a presence, when in fact freedom is the absence of something undesireable. So the right question is, freedom from what, and the answer, at least in the political sense, is oppression, or unjust coercion, or harm to one's person or property except as punishment for a crime (to a libertarian all three of these say the same thing).
So your identification of freedom with family, community, and virtue is also mistaken. Freedom is an absence, and those are some (but by no means all) of the things that go into the good life. Freedom is not the be all and end all of life. It's obviously better to have both positive goods and the absence of evils, than just the absence of evils. It is, however, the be all and end all of politics. Fleming makes a mistake here, too. In other essays he called the good life the end of politics (although he mentioned the good life here, I don't think he went quite that far this time). It's a mistake with a venerable history, but still a mistake. The government is necessarily an institution built on violence; the best we can hope from from it is that the violence will be directed only at those who deserve it.
Its odd though. Fleming and his libertarian opponents at times act like two political candidates debating, who try as they can to disparage the other's position really can find surprisingly little to substantatively disagree on in a policy or ideological sense. The differences seem more cultural/preferential, in a way that makes it difficult to discuss. Like many things in politics - the campaign of Gore vs. Bradley, of Buchanan vs. Dole, of Bush vs. McCain.
There's a lot to that. Maybe it's why he fails to really refute libertarianism. If he did, he'd undercut himself, too. Instead he muses on Walter Block, and then starts calling the general theory "false and evil", without anything to actually back that up.
Funny how we cross-posted the same idea. See #30.