Like you I have a great grandfather who served the Union Army during the Civil War, fought all through the western theater, wounded in Mobile Bay in April 1865.
His experience was quite different from many -- for one thing he was fresh off the boat from Europe, spoke little or no English.
My Dad fought in WWII and served in Germany, but he bore neither Germans nor Japanese animosity and had no problem with President Reagan's visit to the Nazi cemetery near Bitburg in 1985.
I myself was soldier enough to realize that any real soldier who served honorably and suffered or died for his country, even on the losing side, deserves honor & respect, certainly from his own family and country, but also from the wider world which depends on such brave men (& now women) for our safety.
Of course we give no respect to their political leaders -- none -- but soldiers themselves were doing their duty and that's all we can ask.
Today Southerners are the backbone of our military and I can promise you, when President Trump goes looking for the next General McArthur and General Patton, the ones he finds (assuming we still make such men?) will almost certainly have some Southern twang in their voice.
So I have no problem with Confederate flags generally, or specifically in cemeteries.
I don't equate them to the Nazi swastika, which was first and foremost a political symbol of warped ideology.
By contrast, Confederate Battle Flags were just that -- used to rally troops not to twist ideology.
Yes, I have huge problems with insane lost-causer propagandists hoping to white-wash the Old South's political class, but I don't blame troops for their leaders, and I personally have no problem with their flag.
As mentioned, it flies on homes and pickup trucks in many small towns in central PA, usually beside Old Glory.
My friend, a civil war is not the same as two opposing nations fighting each other. What if the South had won? What kind of a nation would America be? What kind of a world would there be today?