Well, yeah, but, more to the point, isn't Columbia derived from "Columbus," just as with the country of the same name?
Yes, the District of Columbia was named after ol' Chris, as were many other places -- cities, provinces, countries and more. For a list, click.
Note that the South American country you referred to is Colombia, not Columbia. The various spellings no doubt reflect Italian, Latin, Spanish, and English -- just like the name of my state, North Carolina, named after King Charles I of England, but derived from the Latin thereof.