The judges had them come back after they were voted off? I don't recall any of that.
Like I said, yes and no. After four weeks of semi-finals, they had filled 8 of the 12 finalist spots. They invited back 12 people (from any stage of the competition, though most came from failed semi-finalists) for a wild-card round, and then eliminated 4 of them after rehearsals. The remaining 8 sang, the audience voted. The judges then each picked one of the wild cards to go to the finals, and the highest vote getter of the remaining 5 contestants got the final slot.
I did't watch Season 2, but my understanding is that Clay made it to the finals in a similar fashion.
The original set up was that they had 4 sets of 8 and America chose 2 out of each group to go to the finals and then they had a wildcard made up of some of the remaining 24 where 4 more were chosen (3 by the judges, 1 by America). No one gets a reprieve once the finals begin.