Abolitionists of the 19th century supported terrorism and many were themselves in fact terrorists. Abolitionists were hated in the North too. That’s why it was such an insult to be called one. The only worse thing was to be called a “Tory”.
So, you & I would consider "terrorist" to be the bigger insult, and yet, interestingly, that's not what the Confederate cavalryman threw at W.T. Sherman's skirmishers.
The word in July 1861 he considered more insulting, or just more truthful, was "abolitionists".
Perhaps "terrorist" would be added later?