From the Abraham Lincoln Association via University of Illinois --
From another issue of the Journal of the Abraham Lincoln Association, also published by the University of Illinois --
This anecdote, which appears in both Stephens and Campbell's accounts and which Lincoln later repeated to his portraitist, reveals a harsh side to Lincoln, perhaps caused by his desire to reassure the Confederates that he did not seek a social revolution in the postwar South. The story also belies Lincoln's earlier expressions of sympathy for black refugees from slavery and his approval, one month after the Hampton Roads Conference, of the Freedman's Bureau bill providing temporary aid for the former slaves (and white refugees) in their adjustment to freedom. Still, Lincoln, like most Americans at the time, optimistically expected emancipation itself to be "the king's cure" for blacks in the South. Lincoln believed that a free person, now including blacks, should be able to make his way in America through his own ability and effort without the assistance of the state. Though he had admitted the difficulties of the white and black races living together in freedom (his earlier support for black colonization reflected this concern), the president envisioned a limited role for the federal government in protecting and aiding blacks after the war. Had he lived to witness the postwar threat to black freedom, Lincoln might have changed his mind regarding federal responsibility for black liberty.
It's a good article on the Hampton Roads meeting. I recommend it.
And yet the account I gave was the only one where Lincoln explained his actions. Who better than he would know what he meant? His remarks show that his lack of sympathy was for the southern white population that would have to work for a change.