Lincoln was somewhere in this middle ground, along with the vast majority of northern whites of the time. By today's standards this made them racist. So what?
I happen to believe northerners with this attitude were wrong. However, they were a lot less wrong than those, the vast majority of southerners, who were willing to fight, kill and die to preserve the abominable institution of slavery. A more un-American attitude it is hard to imagine.