You have no idea of how many people in this country carry guns. Some have "permits" from the state (anywhere from 2-4 million). Others live in states where none is required (Vermont, Alaska, etc...). And others simply choose to ignore unconstitional edicts which "ban" them from carrying weapons for self-defense.
BTW, what other Rights do you want to take away from these "strangers"? The Right to vote? The Right to a jury trial? The Right to own property? The Right to Free speech?