It isn't that I doubt your word Rustbucket, but something as important as this phantom quote has to remain suspect until a primary source can be found.
The link I posted to you above reported a meeting between Virginia Convention Delegate John B. Baldwin and President Lincoln on April 4th, 1861. Baldwin, I think, was one of the more Union favoring delegates to that convention, at least in early April. As the link indicates Baldwin said something similar to Congress in 1866. He is the guy who met with Lincoln in person and later testified before a congressional committee about what Lincoln said. Believe whatever you want, rockrr, of course.
“(From testimony before the Reconstruction Committee of the US Congress by Mr. Baldwin in 1866,
http://valley.vcdh.virginia.edu/personalpapers/documents/
augusta/p3baldwininterview.html#baldwin)”