Attend me closely, old son, and take notes. I'll try to spell it out for you.
Opinion, as you very well know from voluminous posts to other threads, was all over the lot at the commencement of the secession movement, and it did not acquire monumentality until the end of the War. The black Southerners who fought for the Federalists did so out of a personal calculation and motives of various combinations of personal and racial revenge and racial vindication. The white Southerners who resisted the Confederacy were Jacksonian Democrats who believed in Manifest Destiny strongly enough that they did not countenance secessionism, and to the point that they failed to recognize, with Robert E. Lee (who was their fellow in opinion as late as January 1861), a) the Southern States' right to secede in Convention, as Sovereigns of their fate, and b) their own personal duty to support their neighbors and go with their States.
I would also call to your wandering and slothfully inductive attention the principled opposition to Lincoln's war of people like Rep. Clement Vallandigham of Ohio and the entire Copperhead movement, which correctly thought that it wasn't worth a war (to them -- it was worth it to businessmen in New York City, to Them Who Must Be Obeyed!) to chain the Southern States to the Union against their will. The reason that Copperheads didn't appear in more numbers in the Southern armies, or resist Mr. Lincoln's conscription measures more effectively like the Unionists did Mr. Davis's levies, is that Mr. Lincoln had more troops, and more howitzers and canister with which to "reason" with his political opposition over various unpopular measures!