Per Wikipedia
“In August 1980, he also suffered a head injury while leaping onto a New York City Subway train.[106] This head injury permanently upset his equilibrium, resulting in his avoidance of driving.”
sounds like he was already avoiding driving