You are very right. And you have to be looking in the correct direction and azimuth.
About 10 or 15 years ago, I was lucky enough to see a bolide over the San Francisco Peninsula. I was driving home westbound on CA-237 and it dropped out of the sky right over the hills to the west of me. It flashed with spectacular green/blue colors, looked like it was breaking up, and it was gone in less than two seconds. If it had been elsewhere around me or higher up, I would have missed it.
You only have a few seconds to look up at just the right time and no replays.
The sound comes later when the light is gone. You have a better chance to spot one at night when it lights up a dark sky.