Phrase root hog or die "work or fail" first attested 1834, Amer.Eng. (in works of Davey Crockett, who noted it as an "old saying").
According to Campbell's book, Lincoln used the expression to defend emancipation against objections. It does seem strange that there'd be argument about emancipation in the last year of the war.
Given that both Campbell and Stephens allude to the phrase, Lincoln probably said it, but consider that in the final months of the war, Confederates, who we're told had been considering freeing the slaves themselves, were trying to get around the effects of the Emancipation Proclamation. Lincoln's impatience with such talk was natural.
I would tend to agree, it was inevitable, and by this time, Lincoln was adamant - so many had died, he was determined that slavery would end. Given Justice Campbell's accounts (considered an abolutionist by many a Southerner), I'd say he would have had little occasion to lie about the conversation. Regardless of whether Lincoln were addressing whites or blacks, considering it was the union armies that had so mercilessly devasted the Southern farms, it was still a very callous viewpoint to hold.