Rude, ignorant, raised in a barn. Part of his "look" indeed.
Perfectly reasonable, and I wasn't raised in a barn. Hats help with glare and shading, and some men who are balding prefer to wear them indoors sometimes.
After all, in Japan it's customary to remove one's shoes when entering a house. In America, not so. Why should hats be any different?
Nothing shouts "I have premature male pattern baldness" as loudly as a 19 year old college student whining about having to take his hat off.