Yes, those street performers in Times Square are nasty business. In fact, I would recommend staying away from Times Square period. Even for tourists. So many other places in the city are worth seeing and Times Square is a poor representation of NYC.
+1. Just stay away from New York completely.