“I won’t speak to you and I don’t want to be quoted as saying I won’t speak to you.”
I think he was saying he wouldn’t talk to the press, not Joan.
He wanted to let her know he didn’t talk to them and what was printed was a fabrication.
Even if you don’t like someone, I think it’s a good idea to let them know you didn’t say what was printed.