A lot of them DID leave it.
But many more of them stayed, but still felt they were right. My ancestors were among the latter.
And a lot of those who stayed were like Randolph; have nothing but pure, bloody hatred for the country but stick around anyway. A lot of your cohorts are still like that.