He’s not damaged at all.
He withdrew because he would have had to cut shitty deals to get senate approval. The example I heard was they said they would approve him if he promised not to investigate Fauci - that is just one example - so he said “F$@k that” and withdrew.
I like it that he was unwilling to cut a deal.
He could just as easily re-take his seat in FL-1 in the next Congress.