"The only way this concept holds any water is if you are not visible"
Invisibility is technically possible with nanoscale "LED" like devices, an entire suit of them, reflecting the light from directly behind at every angle. There is some amount of research going on in this direction. So, invisibility alone would be insufficient to prevent some sort of minor change. I think I agree with SoothingDave, that the mere fact that you are, you exist, says that anything that happened in the past, happened. Which opens the door to a fixed future, something that infuriates some, for whatever reason.
Of course the concept is flawed. Time travel is inherently paradoxical. "Bill and Ted's Excellent Adventure" showed that. We need not even bother going back into time to make ourselves rich, we just have to resolve to do it in the future and our present instantly changes. Hence, when Bill and Ted needed Ted's dad's keys, they just appeared. One commented "We have to make sure we remember to go back and steal them" and the other responds "They're here, so we must have."
It's all a pile of illogic and paradox.
SD
Unless your affect on the past sets off a series of different events that eventualy converges to a "present" universe that is exactly the same as if you weren't there. Multiple paths to the same goal, is that possible?