“There should be a special election for every elected office that is vacated.”
I would compromise with you. There should be special elections to fill vacancies in U.S. Congressional seats, but governors should appoint someone to fill the vacancy until after that special election is held.
Given that the appointee in most cases (and in this case) is only serving the remainder of a term, I can see laws allowing the exception for when that term remainder is less than 1/2 of the otherwise full term, in which the next election will take care of your concern.
By your lights the voters should reject this guy in 2024 merely because he is now being appointed to serve office for the next 22 and 1/2 months.