In fact, plantation owners used their money to buy goods from the North and elsewhere or they invested the money in New York banks. They got full value for the money they earned from exports. Northerners then could use the money they earned in those exchanges to buy foreign goods.
Let us say that everything you just said is absolutely true.
What happens to the economic conditions in the North when 72% of their import money gets cut off?
Then they start making stuff themselves, which is what the protectionists wanted.
Beyond that, I don't know. A lag in growth from not being able to buy new machinery from Europe could be offset by importing fewer finished goods.
Loss of export wealth would have cut into the luxury market, but Northerners did have the resources to produce what they needed at home and they weren't dependent on something like foreign oil to survive.