“Most northerners, OTOH, just wanted to get on with their lives and have the whole issue go away”
The North had large population of 48ers, European radicals from the failed revolutions of 1848. 48ers were one of the founding groups of the Republican Party.
They were far from peaceful, many having fought in revolutionary armies. They were as enthusiastic for war as the abolitionists. A large number served in the Union Army and several became General officers.
You are correct.
However, it is fair to point out that a great many, possibly a majority, of the “radical” 48ers that came to America were no more radical than the American revolutionaries of 1776.
An unfortunate meme has been floating around that the men of 1848 were all pre-commies. They were not. A significant majority were classical liberals opposed to the absolutism of their rulers, a position fully compatible with American values.