This overlooks the fact that those protectionist tariffs were imposed because the Northern manufacturing did not compete very well with English and European manufacturing in a free trade market.
Congress could not magically wish those economic realities away.
Then, as now, US workers were generally the best paid workers in the world, making US produced goods relatively expensive.
But US tariffs went up & down every few years going all the way back to the beginning, around 1792.
At their highest, tariffs averaged around 35% in the 1830s, under president Andrew Jackson and Vice President Calhoun -- the "tariff of abominations".
At their lowest since then, tariffs reached 13% in 1840 and again 15% in 1860.
Confederate tariffs averaged around 15%.
Point is: for political as well as economic reasons, US tariffs went up & down, without causing major disruptions to either the economy or political alignments.