I think the key is whether or not the "house" is actually a house or (at least temporarily) a business establishment.
You have complete control over who may come into your house But that's less true with a business establishment.
Having said that, this is an obvious case where a religious exemption should apply.
Funny that. I can think of at least a dozen high end stores, galleries in mid-town Manhattan that have buzzers and armed guards in blazers or fine suits. True, some of those might have discrete tiny plaques that read "By Appoinment Only" but what the hey.