I get the impression that Robert E. believed he had to make up for the rest of them. The Revolutionary generation of Lees, even Richard Henry, coin-toss for greatest orator of his time, were way out on the Far Side.
As I remember from my old history books, there was a lot of “Southern Gothic” in Robert E. Lee’s family.