It's probably yellow jackets which build underground bunkers. The vibration from the lawn mower has enraged them.
Yellow jackets can bite as well as sting and the bites can cause septic blood poisoning because wasp feed on carrion.
You will need to locate the hole and mark it someway and wait till dark, so they will all be comfortably settled down for the night. Then fry them in their beds with wasp killing spray (Black Flag).
Hmmm, you've given me an idea, use a MOAB bunker buster!