That's fine. But if you care to interfere with someone else's right to take a picture in a public place then keep in mind that we have a second right that protects the first one.
I’m fully supportive of photographer’s rights, and am aware that most people don’t know those rights, including many police. However, if you’re using a telephoto lens to take pictures inside of people’s houses, I think you’re going to run into problems.