The 4th Amendment does not cover Z. Moussoui, French waste-product, unextraordinaire.
Slaves were not citizens, but they were "of the United States". They were Persons held to a term of labor or service - deprived of liberty under due process of law. Negro slavery was lawful, but slaves were not considered citizens. This use of the term "person(s)" in the Constitution, instead of "citizens", was primarily the Founders' attempt to be inclusive of all people of the United States. They hoped to see a peaceful end to slavery, and give it as little endorsement in the Constitution as possible without jeopardizing ratification. Slaves had been refered to implicitly as "persons" in the proposed Constitution, and when the Bill of Rights later was penned, that convention was carried through.
You forget there was a time (i.e. prior to the Squalid 14th Amendment) when the population had large quantities of people in it who, though OF the United States, were not citizens. The Founders' use of the term "persons" was a commendable and visionary, if naively optimistic, attempt to inspire States to recognize the inherent rights of Negroes (persons), but it certainly was not to protect foreign enemies like ZM, despite the misguided, erroneous, smug, contemporary, politically correct interpretations that appeal to you.