It’s time to return as much control to the children as is possible. Send the ‘grown-ups’ home or have them watch from behind a barrier. Most kids are quite capable of running their own ballgames.
I was fortunate growing up to have my elementary school and a golf course and park within walking distance. We played pick ball games or street bail and just adjusted the rules for the number of players. Pitcher’s mound out, batting team catches, call your outfield, etc.