Yes. When Moore is acting as a judge he is acting on behalf of the government. He should not impose his 'will du jour' on religion.
I'm glad that he's a religious man and I think that we as individuals should acknowledge and properly worhsip God, each according to his own conscience. The government should not pick and choose among the religious beliefs of its citizens, favoring some and discriminating against others.
I am sure you agree, whatever you think of him, Alan Keyes is not unsophisticated in his understanding or presentation of Constitutional issues. He asks "BY WHAT LAW is the order to remove the monument given?"
Your answer, I believe, is the first amendment establishment clause. That is the mainstream answer.
Still ignoring states' rights for the moment, there is SCOTUS scholarship (a Rehnquist dissent) that very forcefully argues that 1. it is not incumbent on USA government to be neutral on religion versus irreligion; historically it favors religion, and 2. it is senseless to level Christianity with other religions given the history and makeup of the country, 3. At the time A1 was ratified, "establishment" meant something like The Official Church of the United States is (fill in the blank: e.g. Lutheranism).
That said, of course there is required to be staunch protection of citizen's freedom of conscience/religion. But neither you nor anyone else is claiming those rights are being infringed upon. You argue establishment. Even the judges in the majority (in the Rehnquist case) agreed that at one time the meaning of establishment had been different.
If you are an originalist, that "at one time" should give you pause.
Precedent can be changed when it is proved to be wrong.