To the best of my knowledge there were no minutes taken during the Hampton Roads conference, so we’re going by individual recollections. Stephens implies concern only for the freed blacks. Lincoln’s story make it clear that the interest was for the white population. The Richmond story doesn’t say that the question of the fate of blacks or whites was even mentioned.
Of course, Davis wasn't at the conference, but I'm sure he would have gotten first hand reports of the proceedings immediately after the conference.