Because Southern industries didn’t get the same preferential treatment as large Northern ones, hence it was to their deficit.
The only large Southern industry wasn't threatened by imports. It's customers, foreign and domestic, clammored for its goods and there were no foreign sources of cotton to compete with their supplying U.S. textile mills. So what would protective tariffs have accomplished for them?
Again, what difference did that make to the average Northern consumer who was paying the same inflated price as the Southern consumer was? Did it benefit him that some manufacturer was able to jack up his price due to tariffs? Did his community realize additional tax revenues because the business was located there? No. Did it create jobs? Some, but not for most people. So why did it matter where the industry was located? Tariffs hit everyone, North and South. They forced everyone to pay an inflated price. They did not harm one part of the country more than another.