Good comments. You're right. If we take the situation too far, I wouldn't be able to prove yesterday happened to me- and truthfully, sometimes I am very skeptical that it did (my reasoning- people talk in the movie theatre while the movie is playing. I'll explain if someone wants to know). You go too far down that road and next thing you know, you're in a Wachowski Brothers' film.
Quite so. At a bare minimum, we all make certain assumptions about the world around us - that our senses are conveying accurate information, that the evidence we have in hand is therefore reliable, and so forth. Convincing ourselves about what happened yesterday is easy enough, most of the time - we assume that our memories of the past are accurate. Convincing me about what you did yesterday is likely to be a tough nut to crack, if I decide to be really skeptical about it.
So you say you went to a ball game yesterday? Okay, prove it. So you bring forth your friends who testify that they went with you, and you have a ticket stub. Possibly you caught a foul ball, or maybe you were lucky enough that one of your friends snapped a picture of you at the game, sitting in the bleachers. To which I point out that your friends could very well be delusional, just like you might be delusional, and the ticket stub doesn't really prove that you actually went to the game, only that you could have gone to the game if you had wanted - assuming it's genuine at all, of course. And that's really just an ordinary baseball like millions of others, or that if you give me an hour with a computer and Photoshop, I can produce a "photograph" of me on the surface of the moon, so that's hardly proof either. And no matter what evidence you provide, it's never really "proof", because I can always present some alternate explanation for it.
But the rational approach is to ask which explanation is the most likely, which one is more plausible. Most people, when considering the evidence, will likely conclude that your explanation of it is far more likely than mine, and that you did actually go to a ball game yesterday. And they do that because the evidence pretty clearly supports your explanation better than mine. Combine that evidence with a little elementary reasoning - people will usually ask themselves "cui bono?"-type questions, and probably conclude that you have little reason to lie about going to the ball game yesterday - and it's obvious that your explanation is far more likely to be correct than mine.
Is it proof? Of course not. But we rarely get actual proof - all we get is evidence, whether the question is how you spent your day yesterday, or how life on earth developed. No explanation of either of those things will ever be conclusive proof, but we might be able to construct a strong argument for how it happened. And so we need to be careful not to set the bar for the theory of evolution higher than we would for other sorts of knowledge, lest we find that we don't know anything at all about yesterday ;)