I'm surprised this fact is not mentioned more often. From Sherman's Memoirs:
"On the 18th day of March, 1864, at Nashville, Tennessee, I relieved Lieutenant-General Grant in command of the Military Division of the Mississippi. About this time, viz., the early part of April, I was much disturbed by a bold raid made by the rebel General Forrest up between the Mississippi and Tennessee Rivers. He reached the Ohio River at Paducah, but was handsomely repulsed by Colonel Hicks. He then swung down toward Memphis, assaulted and carried Fort Pillow, massacring a part of its garrison, composed wholly of negro troops.
At first I discredited the story of the massacre, because, in preparing for the Meridian campaign, I had ordered Fort Pillow to be evacuated, but it transpired afterward that General Hurlbut had retained a small garrison at Fort Pillow to encourage the enlistment of the blacks as soldiers, which was a favorite political policy at that day. The massacre at Fort Pillow occurred April 12, 1864, and has been the subject of congressional inquiry. No doubt Forrest's men acted like a set of barbarians, shooting down the helpless negro garrison after the fort was in their possession;
But I am told that Forrest personally disclaims any active participation in the assault, and that he stopped the firing as soon as he could. I also take it for granted that Forrest did not lead the assault in person, and consequently that he was to the rear, out of sight if not of hearing at the time, and I was told by hundreds of our men, who were at various times prisoners in Forrest's possession, that he was usually very kind to them.
He had a desperate set of fellows under him, and at that very time there is no doubt the feeling of the Southern people was fearfully savage on this very point of our making soldiers out of their late slaves, and Forrest may have shared the feeling."