If you are telling the truth, and have honest motives, as these guys did, it will work. At any rate following someone isn't a crime. They might question you but so what? You haven't broken the law and neither did the neighborhood watch guys. They certainly didn't deserve to be shot by a moron.
Try following some random young lady around and let me know how that works out for you.