I doubt that they would because they don’t want the rest of us in their sacred building.
So, mormons don't want coffee-drinkers in THEIR sacred ward houses?
If, as you say, " It is sort of like inviting people to dinner. If you don’t want them to bring their illegal drugs (coffee!), that should not be a problem. If it is, they should not accept the invitation"........ Voters are FORCED to vote in their specific precinct, not INVITED. Declining this invitation disenfranchises the voter.
If coffee-drinkers are not welcome in your "home" you should never issue the disenfranchising "invitation"! MOVE THE POLLING PLACE!