It may be mind-boggling, but absent something to change the original orientation, why should't it remain the same?
I don't know. I just think it's remarkable that a mag field generates an electric field and vice versa at right angles so exactly that it continues to do this a billion times a second for 14 billion years and remains at the exact same angle. You can say cross product orthonormal basis and whatever incantation, but it's just remarkable.
Maybe more remarkable than that there is just enough water in the oceans to fill up the space between continents. Think about it.