FWIW, there is no text about slaves or colored/coloured in the US Constitution.
take a gander: http://www.house.gov/house/Constitution/Constitution.html
They don't use those words, but it's in there.
Representatives and direct Taxes shall be apportioned among the several States which may be included within this Union, according to their respective Numbers, which shall be determined by adding to the whole Number of free Persons, including those bound to Service for a Term of Years, and excluding Indians not taxed, three fifths of all other Persons.Article I, Section II
The phrase, "other Persons," of course, refers to slaves. And these slaves, were, by strange coincidence, all of African descent (i.e., "colored".)
Neither slavery nor involuntary servitude, except as a punishment for crime whereof the party shall have been duly convicted, shall exist within the United States, or any place subject to their jurisdiction.Amendment XIII