Depends on how you want to look at the problem. I don't want to get into a discussion about the difference between classical liberalism and modern liberalism. It has become commonplace to regard liberal and leftist as interchangeable, and in our current times, they pretty much are.
In Sumner’s time, congressmen threw insults at each other. Sometimes, Sumner was the target, yet he never physically attacked anyone in retaliation.
Did anyone suggest he was diddling his slaves? (Yes, I know he didn't have any.) Suppose someone had insinuated he was diddling his children? Are those fighting words?
Sumner should have not crossed the line. He chose to cross the line, and he found out.
What Brooks did was not self-defense.
No, he was acting as a champion for a man who could not do it himself.
With that said, Sumner did insult Butler's speaking ability.
If the Democrats wanted to "hit back" at Sumner, they could have insulted and ridiculed him in return (which they often did, according to what I've read).
There was no need to beat him nearly to death.