I agree. Everything I've ever read about the man says the same thing. He was against secession, but his loyalty to Virginia forced him to fight for her. It must have tormented him to have to make the decision that he made, but he felt that he had no choice. It probably seems strange to people that a man whose father was a Revolutionary War hero could fight against the Union that his father helped to establish, but I don't think we can understand the loyalty that men had for their home states at that time.
Even the most conservative of us have grown accustomed to a strong federal government, but almost no state government. Most state "pride" is generally reserved for sports team (except of course the Texans). And all of this is unfortunate, because we don't realize how much state sovereignty we have handed over to Washington. Abortion for instance should certainly be a state issue (as murder is), but everyone seems to believe it is a federal issue; on the other hand gun rights, which IS a federal issue (we have the Constitutional right to have them), has somehow been turned over to the states (my right to carry a gun in Virginia does me know good in another state, which makes no sense).