Look at who he married and whom he surrounded himself with.
I am not saying he was a horrible guy. Just part of the “elite”. He probably wasn’t a jerk to the “help.” He just didn’t want to hang out with them.
He took time out of his day to talk to the enlisted & junior officers on White House duty, which I very much doubt Mr. Obama does. I will say, though, that President Carter wasn’t too aloof, either, compared to some.