The Anti-Federalists were dead-on, I agree, but compromise with the Federalists was a necessary evil in order to create a nation, which was required for sheer size. It is simply inescapable that the country was created not only to secure internal rights, but also be able to effectively defend itself from enormously powerful foreign threats. It was an abiding pressure that never allowed the full acceptance of the Anti-Federalist freedoms, and that has plagued the country since it's inception.
But since the Anti-Federalist negative rights are truly unique in the world, I believe the Federalist restrictions are secondary to them in estimation. In other words, the success of the country is that negative rights have been able to exist at all, not that they've been under attack by Federalist expansion (which, I think, was always a given).
Nice to encounter freepers who understand the term. Have you heard Rush Limbaugh and others (some here) braying on about Obama's use of the phrase, completely misunderstanding its meaning? True, Obama was lamenting that we only have negative rights in our Constitution (not union thug style rights), but they thought he meant "negative" as in "bad." It was embarrassing to witness.