Washington and Jefferson were slave owners too. Does that mean they didn’t oppose slavery?
General Lee spoke out against the “peculiar institution.”
I believe Lincoln, too, was critical of the abolitionists. At lest the more aggressive ones.
Washington and Jefferson inherited slaves, it was illegal at the time to free slaves. There were laws in place that prevented people from freeing slaves. There are many facets to the story.
Lee didn't "speak out" against slavery.
A private letter to your wife doesn't count as "speaking out."
Lee was in a bind of course -- soldiers and officers weren't supposed to "speak out" on public affairs -- but Lee never showed any inclination to "speak out," and it would have been better for the country if he had.