I haven’t taken that oath myself, but last I remember it’s to the Constitution, not the President.
You may not have taken it but you could have googled it.
“I, _____, do solemnly swear (or affirm) that I will support and defend the Constitution of the United States against all enemies, foreign and domestic; that I will bear true faith and allegiance to the same; and that I will obey the orders of the President of the United States and the orders of the officers appointed over me, according to regulations and the Uniform Code of Military Justice. So help me God.” (Title 10, US Code; Act of 5 May 1960 replacing the wording first adopted in 1789, with amendment effective 5 October 1962).