See I'm Irish and I know that many recent immigrants were paid by conscripted Yankees to take their place in the Civil War. So, while I don't know all the details about who got drafted when, it is quite apparent that many Union soldiers and would-be soldiers were drafted and didn't want to fight to save the Union or free the Negroes. They fought because they had to or they paid someone else to fight for them.
Even if a majority of Union soldiers volunteered, as you say, that probably would not have been enough to win the war. The difference was involuntary conscription. Or do you disagree?