Perhaps he didn't walk a straight line? If you are seeking a ship, what good does it do to walk directly to the coast if there is no port there? If you are an escaped slave, you might have to take an evasive and circuitous route. In my opinion, there seems to be some reasonable explanations for what appears to be an inaccuracy. But even if this is inaccurate, it doesn't necessarily invalidate the entire story.
A circuitous route would make sense for a man on the run. Everybody is going to be looking for you on the beeline.