And forcibly conscripting men solely to preserve involuntary servitude makes more sense? The Confederacy enacted conscription in the spring of 1862. At the same time they forcibly extended the enlistment of all the current soldiers for the duration of the war. After that point literally every Confederate soldier who fought, fought because they had to. Union soldiers could leave at the end of their enlistment, an option denied rebel soldiers.
The cognitive dissonance of believing that recent immigrants forcibly conscripted went to war to end slavery remains. Its a peculiar belief no matter what sort of deflection or smokescreen you throw up.