You might argue it, but you'd lose - decisively. From Ryder v. United States...
"The de facto officer doctrine confers validity upon acts performed by a person acting under the color of official title even though it is later discovered that the legality of that persons appointment or election to office is deficient"
Other decisions reaffirming or citing this principle include, but are not limited to Norton v. Shelby County (where it was initially established)The People of the State of Colorado v. Adolph Quinten Sherrod.
This is not a complicated question of law. Precedent is well-established and is reinforced under the principle of stare decisis.
Might. I would never venture to tell a court what it is going to decide.