I always heard that it was "up stairs, down stairs, we all fall down."
A buddy of mine transferred to England and he told me that the up stairs, down stairs was a class thing. The very poorest lived on the ground level. That stuff did not recognise class and the up stairs and down stairs were affected equally.