I think you're overestimating what you would have if you kept your SS tax yourself. Even if you made the maximum taxible amount from 1959 to 2006 and kept both the employees and employers tax you would only have $430,000 at the end of 2006 at 5% interest (before paying taxes on the earnings).
I have run the numbers on multiple occasions and can prove that my calculations are correct. However, for me it is an excercise in futility. When I reach the end, I just become fustrated with my current "benefit" compared with my "contribution".
Fortunately, I never planned to depend on SS as a means to secure my retirement and the "SS benefit" that I receive on a monthly basis is just a part of my monthly income. If it went away, I'd still be OK from a financial perspective {this is not an offer to give up my SS benefit}.