You are quite correct, historically speaking.
However, blacks, not unreasonably, view that as ancient history. And since the 60s they believe, probably correctly, that the GOP made a conscious decision to go after the white southern vote, which blacks, rightly or wrongly, view as pandering to their enemies.
The friend of my enemy is my enemy.