They ate what was available, not what was on sale at the supermarket. When they had a feast, relatively speaking, it wasn’t necessarily because of abundance. Those are the habits of hundreds of generations that got all of us born.
Why would they speculate at all that "threshing waste" might indicate a special occasion? Maybe it does, or maybe it means they weren't overly concerned about a little chaff with their grain.