Keep in mind that these groups also don’t use a lot of public services that are tax funded...public schools, parks, some transportation...
Obviously, I can't speak about Israel, but in Brooklyn they sure do. Public schools, public transportation, parks, etc. Same is true here in Philadelphia. My opinion is not formed in a vacuum. I live in a very Jewish area. Not so many ultra-Orthodox (although there are some). I count among my friends a few regular Orthodox and many Conservative Jews (as well as the Reformed, although since they tend to be libs, I would call my relationship with them more friendly than actual friendships). Much of what I've learned of the ultra-Orthodox I've learned from them.
There is most definitely a Tradition for the men to study while their wives earn all the income while bearing 7,8,9 or more children.