Switching sides used to be called being a traitor.
Some didn’t want to be on the side they were on in the first place, having been pressed in because neighbors knew each other and would quickly notice anyone who didn’t rally to the flag, so to speak. Such men had to choose on the spot knowing their families would bear the brunt if they chose wrongly, and had to bide their time to get out of Dodge. In the small towns people probably already knew your family’s leanings from chatter before hostilities formally broke out [like our chatting on social media now] and once the war started they were going to keep a close eye on you or run you off your land and would demand some example of loyalty or force you to be complicit and join.
Historically it’s not really quite so simple when speaking of the Civil War. It’s difficult to understand today, but prior to the war, very few people thought of themselves as Americans. Most people owed allegiance to their state more so than to the US. If you asked someone where they were from, more than likely they’d call themselves a Virginian, a Pennsylvanian, or a New Yorker, rather than an American.
For those living in the states that seceded this posed a quandry — either you had to support your state of the USA. Either way, you’re a traitor! For most, the easiest and most natural decision was to support your state. That’s the reason, for example, Robert E. Lee turned down a commission in the Union Army. He could not bring himself to fight against his home state of Virginia.
Someone like this soldier hardly could be blamed for fighting for his home state. It’s really to his credit though that he realized eventually that he was fighting for the wrong side and for a futile cause.
Mose may have been a conscript.
A great many of the men serving in the Confederate Army were drafted men who lived in the South but had no slaves and were opposed to slavery.
I have read of many such conscripts that switched sides when the opportunity presented itself.
Many poor men in the South considered the Civil War a rich man’s war that the poor died fighting.
I believe that constitutionally the South had the right to secede, but I also believe that the Civil War was a rich man’s war from the vantage point of the Southern common man. You had to be at least an upper middle-class man to own even a single slave. If you were drafted, you could pay another man to serve for you. There were plenty of reasons for a poor man to want to switch sides.