Well, short of spending months or years researching, it would be disingenuous of me to claim I knew particularly one way or another.
This article seems to give a fairly objective view of the controversy:
http://www.tennessean.com/sii/99/10/22/blackconf22.shtml Historians fight a civil war over black Confederates
By Associated Press
CHATTANOOGA -- Past the rack of pistols at the Tennessee Civil War Museum and the video on firing a cannon is a grainy 1861 photo of Andrew and Silas Chandler.
Both wear Confederate gray. Both hold swords in their right hands and guns in their left. Both are about to go into battle.
But this is no ordinary picture of Southern loyalists. Silas is black and Andrew is his white master.
The photo is part of a display -- maybe the only one of its kind in a museum nationwide -- stating at least 35,000 blacks fought in the 1.2 million man Confederate army.
It's a politically loaded claim many historians say is wrong.
"The numbers are vastly overinflated," said William Blair, director of the Civil War Era Center at Pennsylvania State University.
"There are people who want to distance slavery as the cause of the war. This feeds nicely into that whole view."
Craig Hadley, who designed the privately owned Chattanooga museum, which opened last year, believes critics are trying to bury a sensitive topic.
"Nobody wants to acknowledge these people because they 'fought on the wrong side,' " said Hadley, a Southern Adventist University professor.
Historians agree that some blacks enlisted as Confederates, even though the South banned them from the army until the desperate few months before the war ended. No one knows for sure why they joined up.
Some may have thought of themselves as Southerners first or believed they would be given money, land or even their freedom in exchange for fighting, said historian Ervin Jordan, author of Black Confederates and Afro-Yankees in Civil War Virginia.
Some may have felt loyal to their owners or pretended to be loyal to join the troops and plot their escape, he said. Others may have been influenced by talk of undisciplined Union soldiers mistreating blacks on their march.
And free blacks who owned land may have wanted to protect their property.
The Louisiana Native Guards, a group of prosperous free blacks in New Orleans, volunteered in 1861 to fight for the Confederacy.
But after the North took control of the city a year later, the regiments reversed course, volunteering for the Union.
"The bottom line is most white Southerners did not trust black Southerners but they were willing to consider the use of blacks in the military to save the Confederacy from defeat," Jordan said.
The research gets murkier when historians try to count the number of black Confederates. Estimates range from a few hundred to more than 50,000.
Thousands of free men and slaves served the Southern army as laborers, cooks and musicians and may have been armed. Many were so-called body servants -- slaves like Silas Chandler who traveled with their owners as personal attendants and often carried guns for protection.
John McGlone, president of Southern Heritage Press and an editor of the journal Black Southerners in Gray, is among those who believe such laborers should be counted as soldiers, even if their masters forced them.
"When you do get a battle commencing it all becomes a big blur," said McGlone, a history lecturer at Motlow State Community College in Tullahoma. "Often, they got involved in battles even though their normal role was support."
But many historians find this approach illegitimate, saying armed forces always make a distinction between soldiers and support crew.
"I would say that while the distinction was blurred around the edges, it was still a distinction," said Civil War historian James McPherson, author of Battle Cry of Freedom.
Adding to the confusion are sparse war records in which some soldiers are identified by nothing more than their initials.
There are eyewitness accounts of black combatants. But newspaper reports often were biased, written by journalists who never saw battles, McPherson said.
Documents from burial details also were unreliable, as crews often reported finding "negro corpses" when the bodies simply had turned black after hours in the sun, McPherson said.
Jordan traced the origin of one account of Southern troops at Gettysburg with a "colored flag bearer" and discovered the eyewitness actually had written of a "flag bearer bearing the colors."
Records of pensions awarded to hundreds of black Confederate veterans raise more questions. Blacks applied as laborers, but Jordan said he came across documents where blacks had crossed out "soldier" since they officially had been banned from combat and written "body servant."
Major historical sites including the National Museum of the Civil War Soldier in Petersburg, Va., the Museum of the Confederacy in Richmond, Va., and Gettysburg National Military Park have no exhibits on black Confederate soldiers and no plans to add displays.
"It would be something that we would probably address if there was evidence there were substantial numbers," Gettysburg historian Scott Hartwig said.
"There have been a lot of people who have written about it recently and the evidence has been very flimsy."
Jordan, who is black, believes Civil War museums should be sensitive to the feelings of the black community.
"My attitude about blacks who were loyal to the Confederacy is I don't condemn them nor do I praise them," he said. "My goal is to explain them."
Hadley, who is white, said reaction to his Chattanooga exhibit ranges from praise to virulent condemnation. He expected as much when he developed the display and hopes it will generate more discussion.
"It's not something we need to be politically correct about," Hadley said. "We love to talk about the Civil War in general terms like the whole war was about ending slavery. The war was a whole lot more complex than that."
~~~~~
And this articles seems to corroborate the "little bit of both" angle:
http://www.defenselink.mil/news/Feb1996/n02051996_9602053.html Black Confederates
By Jim Garamone
American Forces Press Service
WASHINGTON, Feb. 1 -- Confederate Maj. Gen. Patrick
Cleburne was a born fighter. A division commander in the
Army of Tennessee, Cleburne hated to lose.
In 1864, Union forces, with their virtually unlimited
resources of men and materiel, were grinding the Confederacy
toward defeat. Cleburne saw an untapped Southern resource he
wanted to use before it was too late.
Cleburne made a revolutionary proposal to Army
Commander Gen. Braxton Bragg: Arm Southern slaves and have
them fight for their freedom with the Confederate army.
What mattered to Cleburne was not the institution of
slavery, but the establishment of the Confederate States of
America. He believed logical men would see the only way to
overcome the tremendous Union advantages in men and materiel
was to arm the slaves.
But there was nothing logical about slavery. Bragg, his
corps commanders and selected division commanders in the
Army of Tennessee listened to Cleburne’s proposal in shocked
silence. The whole idea was repugnant to them. Still, Bragg
forwarded Cleburne’s proposal to Confederate President
Jefferson Davis.
Davis killed the idea and in fact was so worried about
the effect of such a proposal on morale that he suppressed
any mention of it. Cleburne’s novel idea did not see the
light of day until 40 years after the war.
But African Americans did serve with Confederate
armies. And eventually they even bore arms for the
Confederacy.
Early in the war, "Free Negroes" tried to enlist in the
Confederate army. Black militia units, most notably in
Louisiana, rushed to join in the war. The Confederate
government did not accept the black militia units for army
duty. None of the units appear to have been in combat, but
many may have performed what is called combat service
support today.
Thousands of African Americans marched off to war for
the Confederacy. Many accompanied their masters, and there
were isolated instances throughout the war of these "body
servants" -- as these slaves were called -- taking up arms
when their masters went into combat.
Many other slaves served as laborers for the
Confederate army. During the Atlanta campaign of 1864, for
instance, Confederate Gen. Joseph Johnston used thousands of
slaves to prepare fortifications as his army sparred with
that of Union Maj. Gen. William T. Sherman.
Thousands more slaves served the Confederate army
driving horse-drawn supply wagons. The Confederate fighting
force was white, but much of its support was black.
But sheer Union numbers facing the Confederacy meant
arming the slaves and giving them freedom was almost
inevitable. The Northern population was 20 million. Of the
South’s 9 million people, one-third were African American.
By late 1864, it was becoming apparent to even the most
optimistic Southerner that the North was winning. The fall
of Atlanta and Sherman's subsequent March to the Sea, Union
victories in Virginia’s Shenandoah Valley and Lt. Gen.
Ulysses S. Grant’s death grip on Richmond and Petersburg,
Va., meant time was running out for the Confederacy. The
last hope expired when Northern voters re-elected Abraham
Lincoln president.
Now desperate, Jefferson Davis embraced an idea he
thought revolting a year earlier. The Confederate Congress
began looking at bills allowing the enlistment of African
Americans into the army in early 1865. Confederate Secretary
of State Judah P. Benjamin spoke at rallies around Richmond.
He said 680,000 African-American males were ready to fight
for the Confederacy: "Let us say to every Negro who wants to
go into the ranks, 'Go and fight, and you are free ... Fight
for your masters, and you shall have your freedom.'"
Representatives from the Deep South were especially
keen on getting blacks to enlist -- theirs was the land
Sherman was laying to waste. Some in the Confederate
government saw the measure as an admission the Confederacy
was wrong about slavery from the beginning.
"If we are right in passing this measure we were wrong
in denying to the old government [the United States] the
right to interfere with the institution of slavery and to
emancipate slaves,” Virginia Sen. Robert M.T. Hunter said.
“Besides, if we offer slaves their freedom ... we confess
that we were insincere, were hypocritical, in asserting that
slavery was the best state for the Negroes themselves."
In February 1865, the Confederate Congress, after
months of stalling, passed an act allowing black
enlistments. Immediately, Virginia started enlisting slaves
to fight for the Confederacy.
White officers commanded these battalions. They drilled
and marched in downtown Richmond. Recruiters hit the areas
around Richmond and Petersburg, but they moved too slowly
for Rebel Gen. Robert E. Lee. He took officers from the Army
of Northern Virginia and started recruiting blacks
immediately.
But time ran out. On March 31, Union forces broke the
Confederate lines at Petersburg. Lee was compelled to
evacuate Richmond and Petersburg. His only hope of carrying
on the fight was to escape to North Carolina and link up
with Confederate forces there.
Records from the time are incomplete, but several
thousand African Americans may have served as soldiers for
the Confederacy. Anecdotal evidence implies at least some
went into combat against Union forces.
On April 4, a Confederate courier observed black
Confederates defending a wagon train near Amelia Court
House, Va. When Union cavalry approached, the black soldiers
formed up, fired and drove them off. The cavalry re-formed,
charged and took the wagon train.
Later, near Farmville, Va., white refugees saw black
Confederates building and preparing to man fortifications.
Lee surrendered to Grant at Appomattox, Va., on April
9. The enlistment of black Confederate soldiers was the
dying gasp of the South.