Your observation is probably very relevant to Einstein’s life.
Like idiot savants, autistics, down syndrome children, geniuses are differently abled people in many senses.
All of those special people have much to contribute but we have to adjust to them and help them to interact with a world they perceive differently. Geniuses can pass more easily because of their mental abilities but sometimes like a 12 year old that looks 17 they get in over their heads because of diminished social ability.
Not to throw a monkey wrench, but you should also consider that they often perceive motivations much more clearly than “average” folks do. What society perceives as normal is most often an illusion.