Joseph and Mary were probably not much poorer than anyone else in those days, because almost everyone was.
Good point. I started reading a book by Anne Rice (yes, that Anne Rice) that is a fictionalized story of the childhood of Jesus. It painted a pretty good picture of life back then. It didn't show them as dirt poor but because Joseph was a carpenter he had good skills to provide for his family.