The joke is, like most Christian symbols, the fish was co-opted from an earlier pagan fish symbol, which denoted the vagina of the mother goddess. It's theirs only by theft. And it's a little rude. :-)