When people are agitated, no amount of blending will help. I'm sorry you got targeted like that, again. But you're right to never give in. Keep shooting and you'll get the shots you need. Basically, I don't say anything to anybody, I just walk up, fiddle with my camera along the way, raise it and shoot when I see what I want. Smile, and walk to the next photo op. You have every right to stand whereever you are and to take pictures of whomever you like.
I hope to get back to town when it's late summer or early spring, but I'm not sure yet of my schedule.