The magnetic poles are moving and like you say, they may move back.
The problem comes if the movement causes a weakness in the Earth’s protective magnetic field.
exactly. To be clear: The magnetic field protects us from solar winds. This could heat the Thermosphere a great deal what that will do to the rest of the atmosphere is unknown.
It's fairly accepted that Mars atmosphere was stripped away because it doesn't have a magnetic field. While our field moves the field strength is weakened.