I am sure you agree, whatever you think of him, Alan Keyes is not unsophisticated in his understanding or presentation of Constitutional issues. He asks "BY WHAT LAW is the order to remove the monument given?"
Your answer, I believe, is the first amendment establishment clause. That is the mainstream answer.
Still ignoring states' rights for the moment, there is SCOTUS scholarship (a Rehnquist dissent) that very forcefully argues that 1. it is not incumbent on USA government to be neutral on religion versus irreligion; historically it favors religion, and 2. it is senseless to level Christianity with other religions given the history and makeup of the country, 3. At the time A1 was ratified, "establishment" meant something like The Official Church of the United States is (fill in the blank: e.g. Lutheranism).
That said, of course there is required to be staunch protection of citizen's freedom of conscience/religion. But neither you nor anyone else is claiming those rights are being infringed upon. You argue establishment. Even the judges in the majority (in the Rehnquist case) agreed that at one time the meaning of establishment had been different.
If you are an originalist, that "at one time" should give you pause.
Precedent can be changed when it is proved to be wrong.
That is certainly the answer that Judge Thompson gave.
It is DECLARED that defendant Moore's placement of his Ten Commandments monument in the Alabama State Judicial Building violated the Establishment Clause of the First Amendment to the United States Constitution, as incorporated into the Fourteenth Amendment and enforced by 42 U.S.C.A. § 1983.Rehnquist is a fine scholar and jurist, but dissenting opinions are of scholarly interest only. More than 200 years after the righting of the BOR we look back on those powdered and be-wigged statesmen with justifiable awe. But they did not write the 1st Amendment based on some idle wish for religious harmony. They had more than 150 years of religious strife within the colonies to draw upon as an example to avoid. Certainly there is a difference between levying a tax to support a religion and using a taxpayer purchased building to display a religious monument. But I'd argue that is is a difference of degree rather than kind.
Glassroh v Moore