Someone who ended up agreeing with that position was a Union combat officer named Charles Francis Adams Jr.
Who in addition to being a soldier and a scholar was also the grandson and g-grandson of the two Presidents from Massachusetts. You can find his reasoning in his essay/speech "Shall Cromwell Have A Statue?"
Not the first guy to be disillusioned with the ideals of his youth and turn his back on them. Adams led a Black unit in the war and didn't think much of his troops. And like many a Northern member of the old elite, he hated the new industrial rich and came to look back on the older America and its southern leadership with nostalgia.
The house CFA III (the nephew, rather than the son) built burned down last month. The blaze was truly spectacular: