You're right! Self servitude was only a part of it.
My GGG Grandpa arrived in New Orleans with parents that soon died. That made hie and his siblings wards of the court, and the court named an "Honorable" local citizen as their guardian. A few years later, all those kids were split up, my GGG Grandpa was twelve, and census records have him living in a house with one 35 year old French woman that owned the place, that woman's 18 year old daughter, ten other females aged 14 to 20, and one 25 year old male "driver."
Let's face it, it was a whore house.
Would anyone care to guess that "guardian's" religion?