I suppose an agricultural people wouldn’t just be sitting on their hands throughout an entire year of no work.
You have a point.
That is what was done during the Second Temple Period. It is why the multitudes were available for the Messiah's teaching, as His first year of ministry was a Sabbatical year. The reason the original plan was never followed during that time was that their Persian, Greek, and Roman suzerains would have killed any rabbi teaching on the topic. That is why the founding verse for the Sabbath for the Land is not cited AT ALL in the entire two volumes of the Jerusalem Talmud dedicated to the Sabbath year (Tractate Shviit).