I don't focus on the Northern Bankers, though I expect they made money too. I mention them along with Northern manufacturing, Northern Shipping, Warehousing, Insurance, and the "factors" who purchased the cotton contracts in the South. The Northern "Bankers" are just one of many groups who were profiting from slavery in the South.
As for the Southerners making a "killing off the slave trade", they weren't the ones claiming it was wrong. That would be the Northerners who were profiting from it. Kinda like Apple using slave labor in China to build it's products.
BroJoeK of all people, posted a statement by Teddy Roosevelt that looks suspiciously close to what i've been saying all along. I was amused at the dichotomy of him always saying none of that was true, and then posting a statement from Teddy Roosevelt which asserts that it is.
Here it is, in case you missed it.

I liked it so much, I put it at the top of my homepage on Free Republic.
It's never Jeff Davis sucks.
No. It's never that.
And no Southerner ever profited off the slave trade?
Is that your argument?