Ultimately, Joe’s Nazi sympathies, antipathy toward the British and antisemitism was too much for FDR, who fired him. Course, it didn’t help his cause that he was interested in the 1940 Dem nomination before it became clear FDR was going to go for a third term.
Yes, that’s all true, but also see my #70 above re why firing Kennedy still ranks as small beer, imho.