I'm not sure where your 1863 figure comes from. Source please. My own figures for that and other years year come from F. W. Taussig's "The Tariff History of the United States" published in 1910 [Link], page 345.
Also, you are perhaps forgetting that the North underwent inflation during the war, a natural consequence of the war and of not having cotton exports to offset the imports. The tariff income (duties collected) for 1863 was worth $45.8 million in 1860 dollars. That was down from what was collected in 1860.
See my linked table that includes the effect of inflation: [Inflation effect on tariff income. Post 174].
The link in that post to Inflation Rates no longer works... I'm sure you can find comparable rates listed elsewhere. From my table you can also see the effective tariff rate for each year. The North kept raising tariff rates during the war.
I should have pinged you to post 303.