Lee didn’t want slaves in the south at all. He wanted them freed when he accepted leadership as the head of the army.
"Considering the relation of master and slave, controlled by humane laws and influenced by Christianity and an enlightened public sentiment, as the best that can exist between the white and black races while intermingled as at present in this country, I would deprecate any sudden disturbance of that relation unless it be necessary to avert a greater calamity to both. I should therefore prefer to rely upon our white population to preserve the ratio between our forces and those of the enemy, which experience has shown to be safe. But in view of the preparations of our enemies, it is our duty to provide for continued war and not for a battle or a campaign, and I fear that we cannot accomplish this without overtaxing the capacity of our white population." - Robert E. Lee letter to Andrew Hunter, January 1865
That doesn't sound like someone who didn't want slaves at all to me. And that was four months before he surrendered to Grant.