Robert E. Lee for one.
This Robert Lee?
"Considering the relation of master and slave, controlled by humane laws and influenced by Christianity and an enlightened public sentiment, as the best that can exist between the white and black races while intermingled as at present in this country, I would deprecate any sudden disturbance of that relation unless it be necessary to avert a greater calamity to both. I should therefore prefer to rely upon our white population to preserve the ratio between our forces and those of the enemy, which experience has shown to be safe. But in view of the preparations of our enemies, it is our duty to provide for continued war and not for a battle or a campaign, and I fear that we cannot accomplish this without overtaxing the capacity of our white population."
From a letter to Andrew Hunter, January 1865.