True, but its notable that it hasn't been an issue until now. The problem isn't really the tax exempt status, it is the fact that we have elected totalitarians into high office. If they can use the tax code to come after you, they will. If not, they'll find another weapon.
Either way, they are coming for you if they perceive you as an enemy. They'll use whatever weapon is available or they'll invent one.
If they cannot use the tax code they will use things like zoning ordinances.