I think people during this era where much more knowledgeable about economic matters than we are today. I read an article a few weeks back (sorry can't provide url) about how ordinary people used to debate the gold standard (when we had one) and I assume they knew about tariffs and their effects. Classical liberal political and economic theory was much much wipespread in those days. IMHO. I think we both suffer and benefit from the effects of a intellectual division of labor; i.e., why should I bother to think for myself, when I can just line up a few experts who confirm my prejudices. Course, one could argue that that has allways been the case. IMHO, real economic understanding among real folks is saddly lacking. Most folks have a dim idea that the markets should being rising and if not the Feds need to do something, etc.
You are on to something here. I would add that this is evidenced on this very thread among those who discount the tariff's role. Many like to throw out stats that more physical import tariff revenues were collected in northern ports than southern ones, then conclude the issue settled. They ignore basic economics in doing so by neglecting the primary costs incurred by a protectionist tariff. The cost is not revenue, but the act of protection itself felt in higher prices, not to mention export shifs and foreign retaliation. The same people who cite import stats almost always neglect export stats, which show the South exported the overwhelming majority.
When a tariff is installed, domestic prices go up thanks to that tariff and the writing of foreign competitors out of the domestic economy. Imports pay for exports making the two inescapably separable. It's as Bastiat said - when goods do not cross borders freely, armies will.