It is believed Joseph was dead at this point which explains what happened in the passage.
Did Joseph die or something?He was around at least until Jesus was 12 years old
Luke 2:48-50
48 When his parents saw him, they were astonished. His mother said to him, “Son, why have you treated us like this? Your father and I have been anxiously searching for you.”
49 “Why were you searching for me?” he asked. “Didn’t you know I had to be in my Father’s house?”[a] 50 But they did not understand what he was saying to them.
Did He and Mary split up?
What would YOU do?
Joseph: Mary; are you asleep?
Mary: You bozo; you KNOW I have to do my prayers.
Joseph: I was just wondering if....
Mary: You hornytoad; you KNOW I have had this HEADACHE for nigh unto these 12 years or so!
Joseph: A fella can HOPE eternally; can't he?
Mary: Hope all you want; you ain't gettin' it: I HAVE to remain a VIRGIN for my FOLLOWERS!!
Joseph: What 'followers'?
Mary: The ones who will be bombarding me forEVER with their heartfelt requests!!!
Joseph: I may as well be dead...
Mary: What did you just mumble? I ain't DEAD; just FRIGID. Now roll OVER and go to sleep.
Mary: Son? are You listening?? I've got a bunch more good, Catholic prayers for you to deal with. Ready?