Though you are correct regarding "athleticism", baseball requires much more athletic "skill" and coordination. Of course, the real attraction of baseball is that there is a lot more to the game than just making contact with a 90+ mph fastball. The gamesmanship and strategy is extremely intricate. So much so that baseball is a much more cerebral game. That's probably the main reason why soccer fans can't relate to baseball.
Slow doesn’t necessarily mean cerebral. A quarterback, or point guard, or soccer player has to see so many things before they develop and react so quickly. A point guard or soccer player has to see a couple passes ahead. And i’m not so sure baseball requires any more coordination than golf or any better reflexes than tennis.