Bingo. A guy like Warren Buffet is vey good at avoiding taxes. Guys like him set up incredibly complex tax arrangements. The IRS is reluctant to audit them because they know he can hire the legal big guns to beat them down in court (and that's assuming they can figure out how his tax arrangement works, which is not a given).
So, yeah, once you're wealthy and have a lot of assets, you don't really care about tax increases. But the situation is different for wage earners living in an expensive place like DC or NYC.
or CA.