You are correct that many Northerners were not in favor of Negro emancipation. Most Unionists saw the Civil war as defending the Union, not as any crusade against slavery. It was the Confederates who made the war about slavery. Read their own words. The Confederate secession and war were an attempted counter-revolution against the US revolution. If the war had ended more quickly in a year or two, it is possible that slavery could have survived the war.
A good place to start is A Patriots History of the US by Freeper Larry Schweikart.