It was Oscar Levant who said it, and if you remember him, you'll realize he didn't "know" her "personally."
He wore the same brand of loafers as Gore Vidal.