Interesting. That might well be it. Because he was in the tradition of the Encyclopedists and Deists, who hated and despised the Church, he rooted for the Muslims who conquered Constantinople, without ever having been one himself.
Voltaire said, "Ecrase l'infame," which is pretty much the same attitude.
Then if the business about Francis Scott Keye has any basis, perhaps he managed to persuade him that he was wrong to root for the Muslims, because they were a far worse alternative.
At the time, I doubt anybody in the West viewed Islam as any sort of an alternative, better or worse.
Islam was very near its low point in its contest with western civ, and the whole idea of western/white guilt hadn't even been developed yet.