Why did a South in which only a relative minority owned slaves go to war to keep slavery as an expression of its supposed state sovereignty? For the South, freeing the slaves was recognized to be the end of one set of familiar problems and the beginning of other problems with no clear solutions -- problems that would affect every white Southerner, not just the slave holders.
First, once the slaves were freed, how would they house, clothe, and feed themselves, and how would the South's rural plantation economy function without their labor? How would slaveholders be compensated?
Second, if empowered with the vote and civil liberties equal to whites, impoverished and ignorant freed slaves could be expected to make their influence felt, resulting in corruption, the election of unsuitable officials, and high spending and taxes. There would be much detriment to whites in general and to the property owning class in particular.
Third, the presence of a large, poor, uneducated, restive and resentful mass of freed Black slaves would give rise to an enduring race problem. Notably, the Northern states were unwilling to accept freed slaves into their own states. That was too much trouble, and trouble of a kind that the South was better equipped to deal with and deserving the burdens of as punishment of a sort of rough justice.
As it was, after the failure of Reconstruction and much turmoil and hardship, the eventual resolution for the Southern agrarian economy was a combination of sharecropping , Black farmers on small free holdings, and a large pool of menial Black servants and laborers useful to Southern whites.
Copying laws from the North, virtually all the South adopted a rigid system of Jim Crow laws and a social code that marginalized Blacks for generations. Literacy and property requirements, poll taxes, and other manipulations that severely restricted the right to vote and ended the brief era of relative Black political power in the South.
Might events have taken a better course if Lincoln had lived? Maybe, maybe not. The profound discontinuity of the Civil War changed the country and changed Lincoln, moving both toward advocating greater equality between the races, while remaining uncomfortable with the many problems of applying the principle in practice.
Thus the Civil War led Lincoln far beyond his previous views as to race, and the failure of his African colonization efforts made clear that country would have to find a new path forward that accepted the permanence of a a massive population of former Black slaves.
Yet the contours of history are not as malleable as they may seem, and it is quite possible -- even likely -- that Reconstruction under Lincoln would have failed like it did under Johnson and Grant.
Rather it was socialist Woodrow Wilson who segregated the federal government. The Republican who was elected after Wilson. Warren G. Harding is acknowledged to be partly of African heritage.