There is obviously a well-understood relationship between time and distance: it takes a certain amount of time to go a certain distance, and since we understand that, the notion that these two concepts go together is a comfortable one. A character travels and time passes.
We're also familiar with the often circular nature of travel. A character sets off for a particular destination and then returns home. The end of the trip is also the end of the story. It's a satisfying relationship -- everything neatly tied up.