“I usually don’t look at any weather forecasts as i can’t do anything about it anyway.”
You could know to take a jacket or swimming trunks with you.
I wear shorts & a t-shirt or polo shirt most days of the year (outside of tax season). I carry 3-4 sweatshirts in my car & a couple of pairs of sweat pants, just in case it is really cold (for me) standing out at a game. Usually, if anything, I will only need the sweatshirt, but can slip on the sweatpants over my sport shorts if necessary.