If you let anything go because you don't want to 'judge', you will eventually end up where the ECUSA, PCUSA, UCC, and other mainline groups have found themselves.
No argument there. My complaint, rather, is the readiness with which some folks call others non-Christian -- which is not their job.
I suppose I'm taking it a little personally -- I've had a bellyful of that sort of crap in recent months, dished out by folks who are not particularly pure in their own lives and actions.