Africa contains a great deal of arid land, and most bodies of water contain something that will EAT YOU.
I think the avoidance of water goes way back before the slave trade.
Facepalm.jpg
I think it has more to do with those with higher bone and muscle mass not floating as well than anything attitudinal.