Because I don't think it would be gone, just renamed.
People afraid of starving the federal government of revenue are afraid of freedom.
The FedGov is not going to allow itself to be starved of revenue, and I am one of the people that they will go after to collect the shortfalls after the FairTax is enacted. They are doing so now, but at least I have an idea of how they are confiscating the money and I can plan for it. Under the FairTax, which won't work, they will come up with new and different rules.
Try as I might, I do not understand your objections.
You probably have a much larger hatred of the IRS than I do, I see them as an enforcement arm with the task of collecting the income tax and catching people who try to cheat, most FairTaxers think it is evil personified.
You probably also haven't really studied the FairTax and how much of what they are selling is pure smoke and mirrors. The 23% FairTax rate which is really 29.87%, which will really be much higher when you factor in government having to pay tax, and evasion, is only one aspect of the smoke and mirrors.
You have a point there.
I guess I would just like to see a major change, and don't really care what form it takes. The Fair Tax just seems to be one of the better alternatives.