To visualize this, one can think of Stargate SG-1. When the characters step through the Stargate, they might end up on the other side of the universe, but they haven't travelled anywhere. The just took one step.
So, a better way to put it is: When stepping through a wormhole, one arrives at one's destination faster than if one had travelled there at the speed of light.
But, if you step through a wormhole to the other side of the universe, you haven't travelled across the universe to get there. It's as if the universe is folded up like a sheet of paper with two distant points touching one another. All you did was take a step from one end of the folded paper to the other end it touches.
But time dilation still applies. Consider the twins problem (one twin leaves on January 1st, travels close to the speed of light, and returns on December 31st) where the travelling twin carries the far end of the wormhole along for the ride. A year elapses for the one end of the wormhole, but only a week or so has elapsed for the "far" end (which has been brought conveniently back home). The well-travelled end doesn't just connect to the end that stayed on Earth; it also connects to the previous January 8th. That "little step" now takes you 51 weeks backwards in time, instead of across the galaxy.