That comes from the Garland case mentioned upthread. Garland was granted a pardon for crimes that may have occurred during the Civil War.
‘Ex Parte Garland’ and a couple other cases are the reason the Ford pardon of Nixon was never seriously challenged.
The power of pardon is rooted in common law and original intent. Were it being newly considered by the current lawless courts, it might not stand, but now there are long-standing precedents and significant historical actions connected to it, so it will remain nearly absolute.
Flagrant abuse of presidential pardon could always result in impeachment.