Why would a non-Christian have any interest in Baby Jesus to begin with? I think the pope did okay in this regard. And you aren’t giving people in general much credit. I think most people know that “Baby Jesus” is the infant form of Jesus, no?
If you are a Muslim, one of the chief people groups involved, they definitely have a different idea about Jesus, so different that it’s like to create a caricature and slap the name Jesus on it. This “Jesus” defers to Mohammed, a very fundamentally wrong notion. Lose the divine creds, lose any chance to apprehend Who He is.
A chance to edify, missed — but still could be supplemented. Why all the defensive argumentation? Why not pray for the right thing?