He didn't necessarily receive them at his birth. There's no indication that the wise men arrived at that time.
Matter of fact, since Herod had all the male children under two killed based on the information the wise men gave him, there is plenty of room for concluding that Jesus was older almost a toddler, when the wise men finally arrived.
And of course they went off to Egypt to escape. Leaving Joesph’s occupation behind. I imagine those gifts were used in the initial time in Egypt to survive until Joesph could find work.
Of course, if the wise men hadn’t come in the first place - they wouldn’t have had to flee!
But then Jesus wouldn’t have known what it was like to be lonely, far from home, in exile and in fear. Just one more aspect of the human condition that He fully felt.