Some may have a better gun than you and a better shot.
Of course not all. If a criminal is intent on going down that day, he may look at you as a target. Fact of the matter is, that rarely, if ever, happens. Most of the time, the criminal is out for easy prey, commit his crime, and be done with it.
Home invasions aren't really a discussion of "open carry" because it is impossible to do so if the criminal doesn't confront the homeowner. The closest thing to it would be a criminal robbing a home with NRA stickers all over it. And most criminals would pass right by that house.