Charlie Miller is not your average hacker. He is an ex-NSA computer security expert who has several other ex-NSA coworkers helping him. Just because HE will find a vulnerability does not mean that OS X, or Safari is weak, it means that an expert can find a way in retail but the wholesale exploitation of OS X is still extremely difficult.
It would not bother me much if he finds a vulnerability that he can exploit. That means that one more possible exploit vulnerability will be soon be closed. That is the way that open source software advancement works.
Good points, thanks for the clarification.