Of course they have the right to their beliefs. They do not however have the right to suppress the expressions of the beliefs of others because they claim that they are somehow injured by hearing people around them utter the name of God.
That is not the argument. The question is, does the state have the right to promulgate religious beliefs in a state institution? The answer is clearly NO.