Awww, you sound like a close family.
If I gave my kids that ultimatum, they might make sure to afford only a small house. “Sorry, Mom, the house is too small...”
Well my mother-in-law has lived with us since our kids were born, and since we both work, it was a god-send.
I hope to do the same with our grandchildren. Well, that is, when we finally get some (no pressure, kids!)