Quite simply, each time one of the travelers decided to take trip to the futre or the past, they found themselves in deep outer space, with no way to get back to their ride, the Earth. How fast does the earth move, anyway? :-)
How fast? See below link:
http://woodside.blogs.com/cosmologycuriosity/2006/02/earths_speed_ar.html