Reynolds vs. US 1878 - the restriction over religion practices was born:
‘Laws are made for the government of actions, and while they cannot interfere with mere religious belief and opinions, they may with practices. Suppose one believed that human sacrifices were a necessary part of religious worship, would it be seriously contended that the civil government under which he lived could not interfere to prevent a sacrifice? Or if a wife religiously believed it was her duty to burn herself upon the funeral pile of her dead husband, would it be beyond the power of the civil government to prevent her carrying her belief into practice?’
Muslims are free to believe, but the religious practices are not unlimited free.
Buildings are not religious - people are.