In the case of voting, this type of restriction is as equally unconstitutional as involuntary servitude is unconstitutional. Likewise there is no constitutional requirement to have been a former federal employee to run for President, Vice President, or Representative. You would again have to amend the Constitution.
You think small. I’m hoping for an Article V deal where it can be re-written.