In defense of Fred, I think “silly” may have modified “rule”, not “hat.”
Written, yes....there's no hypen. However, we can go only by his spoken word.
Ah, English, what a crazy language. (Not that others are better.)
Who are you - Mike Geragos?
Yeah, I understood it too. He wasn't calling the fireman's hat "silly."