It may be something that basic. Self imposed White guilt. Being white, he would be on the outside looking in for most of his career.
i grew up in Detroit...saw a lot of that...