I wasn't going to get involved in this side issue, but for a seemingly intelligent man, you really don't seem to want to think too hard on the questions you ask.
Well apart from the fact that no man wants someone else telling him what to do, the Social Justice Warriors of the North were making a big deal over them. When they were coming in the 10s or 100s, they were providing help, and would integrate them into the societies where they settled.
When the numbers are millions, (after the Social Justice Warriors won) they no longer felt a compelling interest to see to their needs or provide the same degree of assistance. After all, they already established themselves as Morally superior to those evil philistines over there, so they no longer needed these people to fuel their righteous indignation.
Besides, the practicality of it was simply impossible. There were just too many people to give large quantities of help to all of them.
Once more, Liberals wanting "Change" found themselves being hammered by the law of unintended consequences.
Your position is clear: Slavery wasn’t really that bad and people who thought it was should have kept their mouths shut.