Initially, the Civil War was about keeping the Union together. Not ony most Northerners but a significant minority of Southerners supported that cause. Then Lincoln made a major strategic mistake with the emancipation proclamation, which made the war seem to be "about" slavery. This made him wildly unpopular among many stalwart Unionists who (rightly) thought they were dealt a bait and switch, told that they were fighting to preserve the Union, a cause they believed to be worth fighting for, but were now told that they were fighting to end slavery.
Even anti-slavery northerners were generally not willing to fight and die, or see their sons and brothers fight and die to end slavery (and rightly so - "ending slavery" wasn't worth 600K + lives and a ruined nation). With the Emancipation Proclamation, Lincoln not only lost most sympathy in the South, but he lost a lot of political capital in the North as well. While only a small number of Northerners went "Copperhead," he wasn't exactly the most popular man among Union soldiers either.
Reminds me of Bill Clinton and "don't ask, don't tell."
Anyone with a lick of sense knew he was lying about his intentions from the get-go.