I think John Brown's raid scared the bejezus out of them. I think it contributed greatly to the belief that the two sides needed to separate.
Brown’s raid did scare people. But it didn’t begin the worry about slave revolts. Haiti did that, as well as Nat Turner.
That history is well described in Thomas Fleming’s “A Disease In The Public Mind”.