That's right, most secular Jews enjoy the Christmas holiday. Many religious Jews do as well, and aren't offened in the least by anything connected with Christmas. The rabbi you quoted is an exception.
But if pointing out the exception and presenting it as if it's the rule gets you off, so be it.