Brown’s raid did scare people. But it didn’t begin the worry about slave revolts. Haiti did that, as well as Nat Turner.
That history is well described in Thomas Fleming’s “A Disease In The Public Mind”.
I have heard of that book many times. I shall have to read it.