You are absolutely correct. I am more interested in how even I fell for what appears to be a sucker's bet. The bank making the home loan is not only getting paid for the loan (through interest, front loaded), but is also entitled to seize the property (at any value) if the borrower defaults, and can pursue a negative variance (in the event of decreased value) through seizure of other personal assets.
The borrower has a very easily defined upside and downside. The lender, on the other hand, will either break even (through asset seizure) or make a tidy profit in interest. A simplification, sure, but definitely in the lender's favor.
Not a sucker's bet, really. You got the loan because you thought the house was worth it when you asked for the loan. You probably expected the value to go up. When you signed up for the loan, you borrowed so much money for so much time, and agreed to pay it all back. You also agreed to let the bank collect the money using other means if you chose not to pay.
If the value of your house had doubled, you would have made lots of money and the bank would have only gotten paid the amount you agreed in the mortgage. If housing prices skyrocketed, I think you would have been offended if the loan officer knocked on your door after few years wanting extra money from you commensurate with the increased property value.
The bank is just expecting you to pay what you agreed to pay -- the loss in the value of your home is not their fault or concern.