Somehow, I don't believe racism particularly bothered a man who made his fortune (over $1,000,000) as a slave trader.
Forrest's speech to the Independent Order of Pole-Bearers Association July 5, 1875.
"Ladies and Gentlemen I accept the flowers as a memento of reconciliation between the white and colored races of the southern states. I accept it more particularly as it comes from a colored lady, for if there is any one on God's earth who loves the ladies I believe it is myself. ( Immense applause and laughter.) I came here with the jeers of some white people, who think that I am doing wrong. I believe I can exert some influence, and do much to assist the people in strengthening fraternal relations, and shall do all in my power to elevate every man to depress none. (Applause.) I want to elevate you to take positions in law offices, in stores, on farms, and wherever you are capable of going. I have not said anything about politics today. I don't propose to say anything about politics. You have a right to elect whom you please; vote for the man you think best, and I think, when that is done, you and I are freemen. Do as you consider right and honest in electing men for office. I did not come here to make you a long speech, although invited to do so by you. I am not much of a speaker, and my business prevented me from preparing myself. I came to meet you as friends, and welcome you to the white people. I want you to come nearer to us. When I can serve you I will do so. We have but one flag, one country; let us stand together. We may differ in color, but not in sentiment Many things have been said about me which are wrong, and which white and black persons here, who stood by me through the war, can contradict. Go to work, be industrious, live honestly and act truly, and when you are oppressed I'll come to your relief. I thank you, ladies and gentlemen, for this opportunity you have afforded me to be with you, and to assure you that I am with you in heart and in hand. (Prolonged applause.) "
I'm not sure that slave owners were "racists," in the manner that that term is used today. Not that that makes it any better, of course.