It wasn't like Southern slaveowners hoarded all their pounds and francs in their mattresses. They exchanged that money for dollars that they could use or invest, and then other people could exchange dollars for foreign currency to buy exports.
I'm aware that the North had a much higher GNP, but they still couldn't buy imports with it unless they somehow acquired the value from Southern (73%) or Northern (27%) export goods. (Roughly.)
It still comes down to the South producing the vast bulk of the products exchanged with Europe. The North used some of it's GNP goods to trade with the South to acquire the value for European import products, but they still had to glom on to what the South was producing, making it ultimately the South paying for the European products through the intermediary of Northern citizens.
And the prices for Northern goods were artificially inflated because of high tariffs that protected them. It still works out that the South could get better value for their production by going direct to Europe.