According to this account, two local groups were protesting to save the monument (Robert E. Lee) and Soros bused in KKK and Skin heads from surrounding areas.
https://thebottomlineat.blogspot.com/2017/08/the-truth-about-charlottesville-riots.html
The two local groups posted ‘Press Releases’:
“It has come to our attention that several out-of-town groups associated with white supremacy and identarian beliefs conducted events and protests in both Lee and Jackson Parks today. Neither Save the Robert E. Lee Statue nor The Monument Fund were in any way involved in these violent events.
We remain committed to preserving the Robert E. Lee Monument in its park through the legal process in the courts because of its historic and artistic value," said a group called Save the Robert E. Lee Statue. "We soundly and completely reject racism, white supremacy, and any other identity-based groups that preach division and hate no matter which side of the issue they happen to support."
~~~~~
This article matched what I heard about the protests at the time but it's almost impossible to find this information now - the Internetz have almost been scrubbed clean, it seems. Truth has been buried and now when the left rages about Charlottesville and accuses the president of lying, search results don't reveal the truth. Google posts biased articles as if they are objective and invites the reader, "You Decide".
I don't believe it was a mix...Numerically there was a very small number of white power people but that is who the media focused on...The protest permit was given to a multi-cultural patriot group who support nationalism...