I make it a point to look directly into their eyes, those whose mien I perceive as menacing. That way they know this white woman is not an oblivious or willing pigeon.
I find speaking to them or confronting them in any way is not productive. I guess we all have our own way of dealing with this.