“hard” not “rough”. The joke originated when it was rumored that Hugh Beaumont was gay (he wasn’t).
Semantics, it’s funny no matter what the word was...
It was the Brady dad that was.