Why explain? Just tell people you don't decorate.
That's always met with "what do you mean you don't decorate for Christmas, why would anybody not decorate, everybody decorates for Christmas, decorations are great". Some of these people are just really annoying.