I have lived in a lot of places and I can tell you that most NFL fans are a$$holes. Not so much with the Saints. You can go New Orleans and walk down the street with your opposing team jersey on and you won't be hassled. You might get a bit of ribbing but not threatened.
Try doing that in Pittsburgh, Chicago, Seattle, D.C etc.