Either your memory is faulty or it is you who is constipated. Google it.
From the Wikipedia article on Barry Goldwater:
"Goldwater boldly (and famously) declared in his acceptance speech at the 1964 Republican Convention: "Extremism in the defense of liberty is no vice. And let me remind you also that moderation in the pursuit of justice is no virtue." This paraphrase of Cicero was included at the suggestion of Harry V. Jaffa, though the speech was primarily written by Karl Hess."
If you doubt that this was made in reference to the John Birch Society you will have to do a little Googling yourself. It's true. You can believe it if you like. Or not.