Your view of History is rather simplistic.
U.S. Grant's wife, Julia, brought along one of her slaves on all of her visits to Grant's headquarters during the Civil War. When Julia was with Grant, their youngest son, Jesse, was in the charge of "black Julia," the slave that Julia had used since her girlhood.
By contrast, in 1858, Robert E. Lee wrote, "There are few, I believe, in this enlightened age, who will not acknowledge that slavery as an institution is a moral and political evil."
The Civil War was not a simply matter of "slavery" vs "emancipation" although it was certainly that simple to some Southern planters and some Northern abolitionists. U.S. Grant, fought to save the Union and tolerated slavery in his own family and had one of the four family slaves in his own Union Headquarters. Robert E. Lee, fought to defend his native State from attack and personally detested slavery.
What would Jesus do? Maybe, as seen in Luke 7:1-10 and Matthew 8:5-13, he would see 19th Century Americans from both North and South as men of their time just as he saw the slave-owning Roman Centurian as a man of his time.
If you demand pure simplicity in your History and demand that a nation fighting a war be judged solely on the basis of it's recognition of slavery and what one of the belligerents considers "illegality" and "treason", then the next flag you would have to ban would be this one:
As for Lee, he said he disliked slavery but thought it necessary, unavoidable and beneficial for the time being and for years to come. If all opponents of slavery were like him, there might still be slaves and slaveholders in the US.
True, early in the war. But the Dent family slaves were all freed in January or February of 1863 and the trips Julia Grant made to army headquarters at Petersburgh were accompanied by a hired girl.
By contrast, in 1858, Robert E. Lee wrote, "There are few, I believe, in this enlightened age, who will not acknowledge that slavery as an institution is a moral and political evil."
Lee's letter was written in 1856, not 1858, and was in praise of message by President Pierce against those in the North who would interfere with 'domestic institutions of the South', i.e. slavery. Taken as a whole it is not any sort of ringing denunciation of slavery, on the contrary Lee saw it as a benefit. " The blacks are immeasurably better off here than in Africa, morally, physically, and socially," Lee wrote, "The painful discipline they are undergoing is necessary for their further instruction as a race, and will prepare them, I hope, for better things.
Nine years later, Lee's views on slavery hadn't changed much at all:
"Considering the relation of master and slave, controlled by humane laws and influenced by Christianity and an enlightened public sentiment, as the best that can exist between the white and black races while intermingled as at present in this country, I would deprecate any sudden disturbance of that relation unless it be necessary to avert a greater calamity to both." -- Robert Lee, January 1865.
U.S. Grant, fought to save the Union and tolerated slavery in his own family and had one of the four family slaves in his own Union Headquarters. Robert E. Lee, fought to defend his native State from attack and personally detested slavery.
Both, as you say, tolerated slavery in their family - Lee freed the slaves of his father-in-law's estate in December 1862, only weeks before the Dent family did. Yet Lee fought for four years for a government that was founded on the belief that slavery was worth a war, and Grant fought for four years for a government that eventually dedicated its effort in part to the end of slavery. Go figure.
Your view of History is rather simplistic.
Let's flesh it out.
Lawrence Keitt, speaking in the South Carolina secession convention, said, "Our people have come to this on the question of slavery. I am willing, in that address to rest it upon that question. I think it is the great central point from which we are now proceeding, and I am not willing to divert the public attention from it."
--From the Confederate Constitution:
Article I, Section 9, Paragraph 4: "No bill of attainder, ex post facto law, or law denying or impairing the right of property in negro slaves shall be passed."
Article IV, Section 3, Paragraph 3: "The Confederate States may acquire new territory . . . In all such territory, the institution of negro slavery, as it now exists in the Confederate States, shall be recognized and protected by Congress and the territorial government."
--From the Georgia Constitution of 1861:"The General Assembly shall have no power to pass laws for the emancipation of slaves." (This is the entire text of Article 2, Sec. VII, Paragraph 3.)
--From the Alabama Constitution of 1861: "No slave in this State shall be emancipated by any act done to take effect in this State, or any other country." (This is the entire text of Article IV, Section 1 (on slavery).)
Alexander Stephens, Vice-President of the Confederacy, referring to the Confederate government: "Its foundations are laid, its cornerstone rests, upon the great truth that the negro is not equal to the white man; that slavery . . . is his natural and normal condition." [Augusta, Georgia, Daily Constitutionalist, March 30, 1861.]
On the formation of black regiments in the Confederate army, by promising the troops their freedom: Howell Cobb, former general in Lee's army, and prominent pre-war Georgia politician: "If slaves will make good soldiers, then our whole theory of slavery is wrong." [Battle Cry of Freedom, p. 835.]
A North Carolina newspaper editorial: "it is abolition doctrine . . . the very doctrine which the war was commenced to put down." [North Carolina Standard, Jan. 17, 1865; cited in Battle Cry of Freedom, p. 835.]
Robert M.T. Hunter, Senator from Virginia, "What did we go to war for, if not to protect our property?"
Alfred P. Aldrich, South Carolina legislator from Barnwell: "If the Republican party with its platform of principles, the main feature of which is the abolition of slavery and, therefore, the destruction of the South, carries the country at the next Presidential election, shall we remain in the Union, or form a separate Confederacy? This is the great, grave issue. It is not who shall be President, it is not which party shall rule -- it is a question of political and social existence." [Steven Channing, Crisis of Fear, pp. 141-142.]
"The nullifiers it appears, endeavor to shelter themselves under a distinction between a delegation and a surrender of powers. But if the powers be attributes of sovereignty & nationality & the grant of them be perpetual, as is necessarily implied, where not otherwise expressed, sovereignty & nationality are effectually transferred by it, and the dispute about the name, is but a battle of words. The practical result is not indeed left to argument or inference. The words of the Constitution are explicit that the Constitution & laws of the U. S. shall be supreme over the Constitution and laws of the several States; supreme in their exposition and execution as well as in their authority. Without a supremacy in those respects it would be like a scabbard in the hands of a soldier without a sword in it.
The imagination itself is startled at the idea of twenty four independent expounders of a rule that cannot exist, but in a meaning and operation, the same for all."
- James Madison
"South Carolina...cannot get out of this Union until she conquers this government. The revenues must and will be collected at her ports, and any resistance on her part will lead to war. At the close of that war we can tell with certainty whether she is in or out of the Union. While this government endures there can be no disunion...If the overt act on the part of South Carolina takes place on or after the 4th of March, 1861, then the duty of executing the laws will devolve upon Mr. Lincoln. The laws of the United States must be executed-- the President has no discretionary power on the subject -- his duty is emphatically pronounced in the Constitution. Mr. Lincoln will perform that duty. Disunion by armed force is treason, and treason must and will be put down at all hazards. The Union is not, and cannot be dissolved until this government is overthrown by the traitors who have raised the disunion flag. Can they overthrow it? We think not."
Illinois State Journal, November 14, 1860
The cause of the war WAS slavery and secession IS outside the law.
Walt