This is exactly the thing. There have been several ugly incidents were someone carrying accidentally showed and someone flipped out. Those need to stop. Education is the key and open carry is one way to do it.
I've been carrying concealed daily for about 8 years now. It's now like my wallet and keys. My holsters and weapons are part of my attire whether I'm dressed casually or formerly. I think this is the way it is for most. But to your point....
I never gave much thought to "who else" was carrying until about a year ago. My wife and I were meeting with another couple in a Pinerra Bread Restaurant. I happened to notice an elderly couple sitting a little ways from us enjoying their meal. The Gentlemen was sitting in a way that I noticed his concealed piece "printing". I had to grin a bit. From that point on, I started paying more attention and looking for tell tale signs on occasion when in public places. When you know what to look for, I think many people would be surprised how many folks are carrying concealed weapons.
I live in Indiana. I think Hoosiers, in general, are pretty comfortable with full knowledge that many unsuspecting folks are carrying weapons. At a Casino in Shelbyville, IN, I have had to stand in line to check my weapon in. They have a window for that.