I don't see any reason to be so pessimistic about travel speeds. We can already, now, imagine ways to travel some significant fraction of the speed of light, v>.1c. If we felt like spending the money, we could use these technologies today, or very soon. There are the nuclear propulsion systems of the Daedalus project, solar sails, etc.
It's not extravagant to speculate that in several centuries we would be capable of traveling even faster and at a practical level of expense.
It's a question of something called "specific impulse". Essentially, it's a measure of how good a fuel is at carrying itself. If the specific impulse is too low, then in order to achieve any reasonable velocity, the starting fuel fraction (that is, the fraction of the starting mass of the ship that is composed of fuel) comes unacceptably close to 1. For any interstellar journey, even nuclear propulsion systems end up with a starting fuel fraction that is exquisitely close to 1, meaning that very little stuff can come along for the ride. The more stuff you want to take, the slower you must go.
(For the record, nuclear rockets have an excellent specific impulse for zipping around the solar system. This WILL happen.)
The only way to do it is to have an externally powered ship. A solar sail with a big driver laser in the home system can do it, although the economics are staggering and the range is limited to a few light years, max. A Bussard ramjet may be the only way to effect long-distance interstellar travel.