Feeling threatened or taunted by an American flag should not be one of them. Yes, racist and bigotted morons may use the American flag as a symbol of exclusion for immigrants but letting them get away with that and treating the American flag as a taunt puts it in the same category as the Confederate flag and no American should ever be excluded from showing or waving an American flag. In fact, I argue that if blacks would simply embrace their status as American (the ancestors of many blacks were here before the ancestors of many whites, including my own family) and immigrants would take up and embrace the American flags, the bigots and xenophobes wouldn't be able to use it as a symbol and would be a lot more welcome by those wary of the loyalties of those groups.
I've lived as a foreigner in another country -- Japan. I never complained about needed a visa, being fingerprinted, or having to carry an ID. I would never think to ask to vote, especially as a non-citizen. It's their country. And when I went to visit the Imperial Palace on New Year's day, they handed me a Japanese flag and I was willing to hold it up and wave it because I was in Japan. Show some gratitude and respect to the country you are standing in, people. If you can't stand the country, go somewhere else.