While this may anger many, I don't think they have a case.
"The senators and representatives before-mentioned, and the members of the several state legislatures, and all executive and judicial officers, both of the United States and of the several states, shall be bound by oath or affirmation, to support this constitution; but no religious test shall ever be required as a qualification to any office or public trust under the United States."
So while it might be tradition to swear on the bible, the Consititution does not require it.
In fact, the Constitution forbids requiring it.
What did Senator Shumer put his hand on?
More importantly...the oath is a joke. Every Congressman and the President take an oath to "support and defend the Constitution of the United States against all enemies, foreign or domestic; that I will bear true faith and allegiance to the same..."
What a laugh...how many congressmen have ever actually read the Constitution? How many are faithful to it? Answer: maybe one...Ron Paul...if there are any others, I'm not aware of it.
99.9% of today's federal government operates without any Constitutional authority.