It was not defined by law.
It existed to be sure but it was not written as law of the land.
There I believe lies the difference, Lincoln offered the south a Chance to make it the law of the land, they refused and legalized it in the Confederacy.
You are mistaken. Slaves were chattel property, same status as cattle or livestock and thusly the institution was considered "covered" by Constitutional guarantees of private property ownership. Today, we view the slaves as though they were an equal human, but most Caucasians of that time did not. Simple color made them "not equal". Refer to the treatment of all peoples of color in that era for proof.
Your position is validated only from today's perspective interjected upon Nineteenth century life. It took a war to revise that old philosophy.