A lot a liberals hate Carter too. Remember that “The Swimmer” hated his guts.
Because he lost to him, and then he was a terrible president. There’s a lot of reason to hate Carter from both sides, he was a serious loser, still is.
John Anderson had 5.7 million votes in 1980. Those were probably mostly cast by Democrats who couldn't stand Carter, along with some liberal Republicans.
I had the bad luck once to be at a banquet where Douglas Brinkley was the after-dinner speaker. He was then planning to write something about Carter so his talk was mostly about that and gushing about the occasion or occasions when he got to have some one-on-one interaction with Carter. I was sorely tempted to get up and walk out.