I think this question is key to understanding DiogenesLamp's arguments and accusations about alleged Northern racists.
In the minds of our pro-Confederates, whether 170 years ago or today, to own a slave is to love that slave, just as we love our children (or something expensive) and will do anything to protect them.
But a freed-slave is no longer owned and therefore no longer loved, so must be hated, especially by people who hate slavery, and so Northern freedmen were not just tolerated or accepted by Northern whites, but necessarily must be rejected, all by definition of "slavery = love".
It's almost as if, in the minds of pro-Confederates, in 1860 the ~150,000 freed blacks living peacefully, prosperously & lawfully in New York, Pennsylvania and Ohio were all "hated" because, "dontcha know", nobody "loved" them enough to make them slaves!
It's a somewhat equivalent feeling to disdain people used to express for an "old maid" that "nobody wanted" -- a similar "hatred" for Northern freed African Americans is being claimed.
Another analogy: often in 1860 and even sometimes today, people draw a distinction between the "love" of a slaveholder for his slaves compared to the "hatred" of a Northern factory owner for his "free-labor" industrial workers.
If a Northern worker gets paid more, it's still "hatred" because he gets laid off whenever he's not needed.
If a Northern worker is free to quit and find a better job, it's still "hatred" because nobody takes care of him when there's no work to be found.
So, slavery = "love" and freedom = "hatred" in the minds of our pro-Confederates.
That, I think, is where DiogenesLamp's criticisms are coming from.