I've been kicked by a soccer ball going near same speed probably far more than you've been hit by a baseball. Granted a baseball is tougher than a soccer ball but a soccer ball is bigger and will hit more surface area. I have friends with dozens more injuries than that playing soccer. There are lot more ways to get hurt in soccer than the ball. I have friends in the same boat as you from baseball but with nary an injury to their name. I grew up playing every sport you can imagine. I knew more people who got hurt badly from soccer than any other sport, including football.
Okay, you win. Soccer is the most dangerous sport ever played.