They are effectively the same people. Mostly well-to-do urban Liberals that the rest of the country regarded as "kooks", as were abolitionists regarded in their day.
As Charles Dickens said on the subject:
"Every reasonable creature may know, if willing, that the North hates the Negro, and until it was convenient to make a pretense that sympathy with him was the cause of the War, it hated the Abolitionists and derided them up hill and down dale. For the rest, there's not a pins difference between the two parties. They will both rant and lie and fight until they come to a compromise; and the slave may be thrown into that compromise or thrown out, just as it happens."
And for what it's worth, Charles Dickens was very anti-slavery.
Douglass got to know the prominent abolitionists very well. He had no illusions and should be admired for doing what he had to do on behalf of his people. Decent people should take the same approach when dealing with liberals. You may have to deal with them, but you should know better than to ever trust them.