My husband started wearing a nice fedora to work, and has gotten a lot of compliments and queries as to where he bought it.
I would welcome the opportunity to wear a fedora. Unfortunately, I work in a manufacturing area, and most of the dress is business casual.
One thing that many are missing on this thread is that a little formality can breed better manners. When you wear a hat, there are some standards of respect. You take the hat off when entering a building or room; you take the hat off in a hospital, or during a church service, and certainly during the playing of the National Anthem. Of course, many people have lost this sense of manners, as you see them all the time at sporting events and parades, wearing their ball caps when the Star-Spangled Banner is played.
Formality in dress requires you to care about your clothing, and how you look. Not in a metrosexual way, but in a thoughtful way. A way that shows you respect whom you're doing business with enough to present yourself in a professional way.
And looking professional doesn't have to mean being uncomfortable. A tailored shirt and pants, a properly-knotted tie, and a jacket that is hemmed properly doesn't restrict your motions or your thoughts. It can bring about a certain level of respect however, in that people dealing with you--especially people whom you've never met before--may take you a lot more serious in more formal dress than they would if you were in torn jeans and a t-shirt.