A fine book on this very subject is "Fire On Earth" by John & Mary Gribbin. They maintain that asteroid and comet collisions have played major roles in shaping human history. I highly recommend it if you're interested in the subject.
These collisions are rather more frequent than most people believe but luckily most are small or at sea. Our planet is mostly water and the majority of objects striking the surface wouldn't be noticed at sea. It is only when a big one hits solid ground or in shallow waters that our attention is drawn.