I bet that he got property tax relief as a farm.
In Texas, they can and do exclude the house from your Ag Exemption. They did so to me for 17 years.
The house and 5 acres out of 200 were taxed at the normal rate.
Taxes did get to be a concern because of this.