If we had waited until we could "afford" children we'd still be childless. I'm going to snuggle with my baby and read stories to all 3 of them at bedtime tonight. I pray we get to add to our family soon. My kids are my leisure time!
I think people are going to be unhappy, no matter how much money they have, if they give their love to what can't love them back. Your family can love you ... not as much as you need - that's one reason we need God ... but your house, investments, job, car, or education can't love you at all.