Except when you have a wife and two kids to look out for.
Though you bring up a good point. When the SHTF, it will be the young guys with no families dragging them behind who will make short work of the rest of us.
Don't think invisibility: think camouflage.
Bullpup rifles, that Keltec mentioned up thread, high firepower to size pistols, etc....