America’s founding political policy depended on a heavy moral consensus that was headed up by Christians and followed on closely by Jews. Surely if a Jewish person was sworn in, he might choose to use an Old Testament, not (yet) recognizing the New. And there would not be a lot of fuss about it by Christians who understand.
But a system like this is vulnerable to faith that becomes slack.
I love religion. But there isn’t a religious reason for swearing in to an elected position. It could well be the Constitution they are going to uphold.