Yes, if company policy bars all religious stuff in the office.
But by your logic, driving a car to the office, and leaving a Bible in the trunk (which is where the guns were kept) would be banned as well.