I can’t imagine anyone making a problem if an Orthodox Jew entered a Catholic Church in full regalia. This is more of a secular and political attitude than a religious one, IMHO.
As I noted on another thread, an observant Jew wouldn't enter with a tallit or tefillin because they're worn when praying, and he wouldn't go to a Church to pray. Many observant Jews wear a kippah or other head covering all the time, that wouldn't be allowed when visiting, for example, St. Peters.
Since this is one Rabbi's opinion, and people visit with cross' all the time, it's really not much of an issue.
Still this is a contrived controversy as the pope will pray as he wishes.