Why were they a particular problem? An affront, yes, but a problem? I don't see it.
Remember too, the threat of Indian attack was very real.
My family and others had to form the NE Iowa Protection Association that was no less a militia to also protect from marauding thieves and bandits.
Loss of Federal protection would not have been sensible.
The guy who writes at "Curmudgeonly and Skeptical" blog is a freeper, (though I don't know his freeper name) but years ago he offered this pearl of wisdom.
"Why don't all the sane states get together and pull a reverse secession by kicking all the insane states out of the Union? "
Seems like a pretty good idea to me. :)
BTW, I have a friend from Waterloo Iowa.
Fugitive Slave Act of 1850:
Any person aiding a runaway slave by providing food or shelter was subject to six months’ imprisonment and a $1,000 fine.
Slave catchers often acted without restraint much the same as today’s bounty hunters. There was violence.