I can certainly understand your sentiments, we have to deal w/ those “gotcha” moments here on a regular basis. However, the saints in the NT had to deal w/ those types of issues on a regular basis as well. Can you imagine them shirking to tell others what church they belonged to b/c they were afaid of those “gotcha” moments. I think Paul in particular would have come down on them like a load of bricks.
If you’re proud of where you belong you should sing it from the rooftops not be afraid of what man might say. We do so on a regular basis. Do we get hammered for doing so? Obviously. As w/ the early saints, are we proud to do so, absolutely! We stand tall in our proclamation. We fear not what man has to say but rather respect what the Lord would have us testify to.