I thought so, too! On some documentary, I heard of a man sleeping in the pig/horse/oxen sties.
I remember reading that the animals were on the ground floor and the people lived above them.