Actually no.
Think of it this way. Let's say we COULD affect the past (and thus the buffer that the physicists are theorizing does not exist), and the following scenario takes place. We have a guy called Joe, and for some reason he wants to take out his great-great-grandfather (who we shall call Ancestor J).
Ok, so Joe hops into his thingimajig TimeMachine and zips to 1855 and shoots Ancestor J with a blunderbuss, and thus no more Ancestor J.
Well, the moment he does that the time-continuum (I CANNOT believe I just said that) will shunt, and thus there will be no great grandfather, no grandfather, no father, and thus no Joe. Hence Joe will immediately cease to exist the moment Ancestor J breathes his last.
Now, the moment Joe disappears into the ether the time continuum will shift again! Since Joe ceases to exist that means he never got into his time machine thingimajig, and thus it means he never went back to 1855, and consequently it means he never killed Ancestor J. And because Ancestor J was not killed he does not die, and because Ancestor J is alive then that means that greatgranddaddy is, as is g.dad, as is dad ....as is Joe.
Hence Joe now exists, and because he exists it means that he is angry with his great great grand dad Ancestor J, and thus he hops into a time machine with a vintage blunderbuss and heads off to kill him.
Now, the moment he terminates Ancestor J then Joe ceases to also exist, and thus he never gets into his time machine, and thus ........
Well, you get the picture.
Just do not ask me why he decides to use a blunderbuss!
Two things. First, this is known as the "grandfather paradox" and there is no reason that the timeline won't shift back and forth between the two and still appear seamless to folks in those timelines. Second, the blunderbuss was about 200 years out of date in 1855.