He must have been a rookie- probably thought they were betting horses!!
The song refers to “old men playing checkers by the trees.” A few years ago, cops in the Rampart area, where MacArthur Park is located, arrested some men in a doughnut shop for illegal gambling—they were playing chess, which the cops, unfamiliar with the game, mistook for gambling.