The men in my family always wore hats, though not Stetsons.
My daddy favored a fedora. In the summer, he wore a straw one; in the winter, as above, it was felt.
My daddy's paternal uncle, "Uncle Lut," the sharpest dresser in a family of sharp dressers. Here he's wearing a zoot suit and the requisite broad-brimmed hat.