All free persons have the right to keep and bear arms; statutes, regulations, ordinances, and other rules whose purpose or effect is to prevent or impede the free exercise of that right are illegitimate.
That having been said, not all persons are free. Children are subject to the will of their parents, convicted criminals serving their sentences are subject to the will of the state, and persons adjudicated incompetent are (to a significant degree) subject to the will of their guardians. The carriage of arms by such people may be legitimately restricted by those with authority over them.
If a person has authority over a certain building or geographic area, and if that person may legitimately expel visitors whom he doesn't like for any reason he sees fit, such a person could if so inclined expelling people found to be in possession of firearms. The person could also expel people who refuse to be searched for firearms or anything else he doesn't like. A person without the authority to expel arbitrary visitors, however, would not have any special authority to expel visitors carrying firearms.
What you are saying then, I think, is that the right to bear arms is not an absolute right. But you seem to prefer the arbitrary whims of some unidentified ‘guardian’ to a set policy. If any restrictions are to be made then I prefer that they be made at the state level and not the federal level. As the 10th Amendment provides.