I figure he didn’t serve the Usurper. Rather, he served the country, and the country failed him by preventing a Usurpation.
Even in WWII, honorable men served under the Socialist Roosevelt.
Those that serve the country have sworn an oath to God to protect and defend the Constitution.