Many southerners had mixed feelings about leaving the Union, especially if a father or grandfather had fought in the Revolutionary War. Among these was Robert E. Lee, the son of the Patriot hero, Henry “Light Horse Harry” Lee. Robert E. Lee was strongly pro Union, but his first loyalty was to his native Virginia. His ambivalent feelings were expressed in a letter he wrote to one of his sons in January 1861: “Secession is nothing but revolution... the framers would not have devoted so much care to the formation of the Constitution if it was intended to be broken by any member of the Confederacy at will... In 1808 when New England states resisted Mr. Jefferson’s embargo law, and when the Hartford Convention assembled, secession was termed treason by Virginian statesmen; what can it be now? (emphasis added) Still, a Union that can only be maintained by swords and bayonets, and in which strife and civil war are to take the place of brotherly love and kindness, has no charm for me. If the Union is dissolved and the government disrupted, I shall return to my native state and share the miseries of my people. Save in her defense, I will draw my sword no more.”
http://www.clis.com/friends/UnionOrSeces.htm
That was pretty much what sectional discourse had become. Don't forget that the Wide Awakes and various abolitionist societies were marching under banners of black-and-white stripes with slogans in the canton like "NO UNION WITH SLAVERS", that kind of thing. Never mind that not one person in a thousand, even in New Orleans, could be even remotely called a "slaver", and only a fraction of the population had ever owned a slave, much less a gang of 10 or 12 or more.
Slavery was a rich man's arrangement -- like H-1B's and L-1's and offshoring and plant relocation to Mexico and illegal immigration. Even the Canadians are fighting it now: the Athabaska Tar Sands are a fabulous energy resource and need to be ramped up with new mines. But rather than hire Canadians, who are unionized, employers are making contracts with labor "contractor" firms, who then do the necessary dirty work by saying, "well, we tried to find workers at the below-market wage we want to pay, so <phony sigh> there's nothing to do but petition the provincial government for relief, in the form of permission to import 1,500 foreign workers whom we can pay those yummy low wages we like."
Sierra Oscar Sierra, in other words.
People in the South could no more do anything about what rich merchants and agriculturists wanted, than fly to the moon -- nor more than, as it turned out, the Knights of Labor could do about working conditions in the industrial North.
So two Virginians might have held it to be treason. Any 1st person accounts?