My cousins used to sing that song at family reunions. I have no idea how they remembered all the words.
I used to hear the old Lonzo and Oscar version on Dr. Demento. I never memorized the lyrics either, but the refrain stuck with me.