Veteran of Desert Storm/Shield. My best friends mom (recently deceased) used to be the bartender at the VFW post in San Diego.
Those who say small, dark, smoky (in violation of the CA law, you GO old guys!), and old guys drinking cheap booze have the right of it.
The problem with exclusivity is that of declining membership.
VFW tended to break the link between the WWII generation and the Vietnam era warriors - and now they have almost no link to the new generation of warriors.
My advice is to try to get younger men who are veterans of foreign wars in there. The problem being that younger men tend to want to drink and carouse where there are young women in evidence.
I'm a member of the American Legion (AF retired, 1978-1999. You can still join the Legion or VFW - you don't have to "hang out" or become involved with the local Post if you don't want to. Your membership will count for the National Organizations, both of which keep an eye on Washington to protect Veteran's interests. There are numerous other benefits from belonging as well, again which don't require any activity at the "local" level.
I've chosen to be an active member of our local Post because I want to be involved. We do more than just sit around and drink & smoke. Most satisfying is providing well deserved Funeral Honors and our fund raising activities which allow us to provide scholarships and donate to other community functions.