Says St. J went to Bethlehelm
5 To be taxed with Mary his espoused wife, being great with child.
Says he took along Mary. It doesn't actually say if he went alone or with his entire kit and kin and others... You're making an assumption aren't you, that he went alone. Or are you relying on a traditional or pastor's interpretation?
Note that ALL went, but no other children went with Joseph and Mary because at this point THERE WERE NONE.
The verse doesn't actually say one way or the other that Joseph didn't take along his mom, pop, aunts and uncles and any brethern...
The passage says that all went to be taxed.
ALL.