All this kind of problems would easily be solved with segregation
Lincoln was putting a fleet of ships together to expatriate the freed slaves.