Right. The coordinates ARE different.
An interesting (not scientifically rigorous, but interesting) take on this concept is Stephen King's novella, The Langoliers. In this story he explores the idea of "the past" being an actual PLACE.
A curious notion in that story, that the past is being consumed and that the past is not inhabited. The folks on the plane did not see humans until they made it back to the present.