There are two proper ways to handle this sort of provocation. Unfortunately method #2 requires a special skill.
Method #1. Quietly ignore the demand. Just calmly go about your business. But have a plan if things suddenly get physical.
Assuming it’s not a lethal threat, I’d suggest police-grade pepper spray. Sabre makes a gel-like version. So there’s more change of hitting the bad guy, and less chance of spraying some innocent.
Method #2. If you know an exotic foreign language, speak that language if and when you’re singled out. And make wild hand gestures.
That’s a two-for-one. You will confuse the bad guys and entertain your companions.
Side story: A friend of mine and I were verbally accosted by a pushy street vendor many years ago. My friend suddenly starting speaking rapidly Croatian. It worked. The vendor walked away.
#2 Is good. I wear a Palestinian Keffiyeh and recently a Moroccan Shesh. I also have acquired some Arabic (mainly curses). It does confuse and destabilize the aggressor.
Be ready to go physical if need be as willful violation of the 6-foot rule may interpreted as an effort to spread a “deadly” disease or so I am told.
I am fluent in a niche, well-corrupted dialect of a well known language. I have used it to confuse a homeless man accosting me, as well as in other situations. It’s a cheap thrill, but also effective.