If the surface weren’t soft dirt, perhaps. If these were projectiles, that could be mildly disruptive, and would wind up on the ground in the middle of the charge as it continued into the rain of stuff. But rocks are free, and there aren’t enough of these to make this make sense.
Broken pottery would also work, and the Romans left a lot of that in piles wherever they lived a while. But they are known to have used javelins, arrows, even sling stones, plus their artillery (which is where the word comes from I think), which was effective against troops as well as fortifications.
Well, maybe these things were a bad idea in the first place, but if you’ve ever ridden a horse, then you know their footing on lose rock is not so hot. So the more unstable the ground, the harder it is for horses to make speed - particularly a full on calvary charge. I see this as the best option for a military use by the Romans.
Broken pottery does not roll and is fairly stable, compared to a multi-surfaced hollow object with knobs.