These are questions that can probably be resolved in a calm and moderate way, even though some groups on both sides take extreme positions on such questions for propagandistic reasons, like fund-raising and membership drives. If the pro-CBF groups seem to be less controlling now, it's because they were in control for over a century, and they determined what was built and what wasn't. So now the more radical of them don't act so much to take things down as to try to prevent new monuments -- like the Lincoln statue in Richmond or the projected National Historical Park commemorating Reconstruction -- from being established. But some of them are as radical and as determined to impose their vision as anyone on the other side.
I can understand why some Southerners are so upset about efforts to take down flags and monuments. But it is a two-way street. Plenty of what the most militant Confederates say is offensive to others, even to those who don't otherwise have a strong opinion about the Civil War. To say that Reconstruction and Black Suffrage weren't closely related or that enmities between Black and White can be put at the door of Reconstruction, and slavery be more or less ignored, as the SC SCV man does, is to take a wrong-headed view of things, and it calls one's opinion on other issues into question.
We're seeing forms of this type of activity all over the place. The ten commandments are suddenly being removed from courtrooms after being there for generations. Nativity scenes that have been placed in certain areas for 200+ years are now suddenly not welcome. Crosses are being taken down in certain buildings and town sqaures. It's the same thing. These neo-leftist-idealists are practicing a form of censorship and they have their "thought police" out in full force to ban or protest anything that does not conform to their ideals. As for the Confederate flag being used by hate groups, let me remind you that the KKK has marched under the American Flag for well over 100 years and slavery was conducted under the American flag. We do not equate the American flag with racism - and rightfully so. Those who truly know the history of the Civil War know that what is being advertised by the likes of Jesse Jackson, Al Sharpton, Kweisi Mfume, etc is simply not accurate. Unfortunately, too many Americans are afraid to challenge these folks on anything for fear of being called the "R" word.