I suspect ideologically neutral muslims DO NOT attend mosque.
Perhaps not, but when those who do walk around with the sword, they are all consenting, perhaps silently, but they certainly do not interfere with voice or action. Have you ever been in a Muslim society? Culturally they do not think like westerners.