Actually, it looks like 3 Southern justices remained on the U.S. Supreme Court until their deaths (including Roger Taney, who died in 1864), one died in 1860 but was replaced by a Lincoln nominee rather than by a Buchanan nominee, and one resigned to go home to his state. There was also one justice from Ohio who died during this period.
For a while during the Civil War, Congress added a tenth Justice to the Supreme Court, to cancel the vote of a southerner who remained on the Court.