Southern Planters exported cash crops. They hired a ship and had to pay the cost of the ship and the crew for the transatlantic journey to deliver the goods to market. Once delivered, they needed to fill the holds of the ships with something to help defray the cost of the return voyage. So they purchased manufactured goods which they needed. They then had to pay a tariff on those goods.
But the map shows that tariffs were greatest in the north east, by far. European goods needed in the south would go to southern ports.