It certainly is -- especially when the main reason fathers can't provide financially for their children is the plunder of the state. Adjusted for inflation, the current per child tax exemption is thousands of dollars below its original value. Fixing this alone would solve the problem for most families in America.
I have 25 cousins that are homeschooled (twelve in one family alone) using family funds exclusively (three other cousins are educated at private schools). Their parents are forced to pay for the state "public school" monopoly even though they receive no benefits from it. The same is true of childless couples and retirees that are forced to pay property taxes that are coerced "for the children".
It's not fair to criticize or punish one family for the injustices of the whole system. Government school is socialist by design: It takes money from some to pay for the state indoctrination of others. This is unavoidable, as no family can be precisely average -- having 2.54 kids and paying exactly average taxes -- in order to pay their "fair share" of education taxes.
Every family -- yours included -- is paying either more or less than its share of taxes for education. Injustice is unavoidable in this system. If you like me and Thomas Jefferson consider this arrangement to be evil, then work to get the state out of education completely. Start with your own family. And, in the meantime, stop attacking other families for being caught up in a system they didn't create and largely don't want.