Chomsky is right in a sense, but in another sense he completely misses the point. Yes, it is true that sports really don't matter. They are entertainment. Are movies a distraction too? Does Chomsky consider people stupid for getting emotionally involved in a fictional movie? I hope not.
Sports are a unique form of entertainment in that nobody knows in advance who will actually win. In a movie, the plot is whatever the writer and director decided it would be, but nobody arbitrarily decides the outcome of a football game.
Having said that, I do think that some people get too wrapped up in sports and lose sight of more important things.
This is the essence of Chomskyism - overanalysis of the mundane.