Throughout the war, Lee frequently used the phrase *those people* to refer to his Union adversaries, rather than the word *enemy,* as he found the situation of making war against his fellow Americans distasteful at the very least.
It's interesting to see that phrase used in this context, as if referring to Grant and those who followed Lincoln and very possibly helped to murder him. You may well be right that Lee bore Lincoln no personal animosity, but that may not have been the case regarding the rest of that nest of vipers.
As for whether the Stockdale quote is accurate or not, I certainly don't know this far after the fact, and can only go by the governor's generally good reputation for veracity and accuracy in repeating the statements of others, at least as far as can be expected of anyone in politicial life. But voters were more concerned with such reputations then than now, and I believe my bet would ride on his quotation of Lee was more accurate than not.
In any event, it's clear that Lee was speaking privately, and not speechifying or expecting his remarks to be come known. Had he said them to an assembled crowd of southron Texians, they'd have likely had a far different meaning, and might well have produced results far outweighing their actual intent.
But I wasn't there, so can't be certain.
-archy-/-