Not everything imported had a tariff placed on it. I don't believe raw cotton had a tariff and I don't know what food you're talking about that the South exported.
That will give you data on tariffs on cotton from China and India. Big Northern mills lost all of their supply in 1861.
If you go to the U.S. Customs data by category, you can find the info. On food.