You minimize and mock the very real threat to their financial interests. It is not radically different from Obama's threats to the coal industry. It was an attack on those people's livelihood.
It is furthermore an explicit and implicit assertion that they are evil people because of how they make their living. While this can be regarded as true after a fashion, it doesn't persuade anyone to change their way of making money. If anything, it hardens their resolve.
They find themselves in positions of having to convince themselves that they are not bad people and therefore they embrace the "badness" all the harder, because if they run away from it, it implies they were bad after all.
I think Northern Abolitionists behaved like, and were viewed similar to "Carrie Nation" scold types preaching against the sins of Alcohol. Most people who enjoyed it probably felt inclined to spite them in every way they could. In other words, by embracing it and drinking more.
Another apt metaphor is the anti-fur activists throwing blood on people wearing fur coats. It simply is a bad way to convince people that they are doing something wrong.
So no one should ever have pointed out that slavery was a bad thing because people's financial interests might be hurt? Or because they might buy more slaves, just to spite the abolitionists? By that logic, we shouldn't say that heroin peddling is bad, because that's a threat to someone's financial interest and it might make them sell more heroin.
You make less and less sense as you go.
While this can be regarded as true after a fashion…
Your grudging admission that slavery could maybe kinda possibly sorta be considered morally wrong is noted.