Except the taxpayer is backstopping the loss in the case of default if it’s a government issued or backed loan program.
The mortgage holder loses nothing. They get to live in a house paying probably less than it would cost to rent a comparable house and if they default, they lose no personal money. They have no money invested in it.
For those that care,they lose self respect.