In short, passage of the 14th Amendment. Chief Justice Chase believed that the provisions of the 14th Amendment punished the southern leadership for their role in the rebellion, and trying and convicting Davis again would violate his 5th Amendment protections against double jeopardy.
He knew they couldn't win which would make them look, how should I say this, bad.