A recusal would be an admission he could not fairly judge the case. What judge is going to admit that? But that's why there is an appellate system.
It is an admission that he has conflicts of interest that would give the appearance of impropriety.