Did the DOJ ever answer why they stopped keeping that information?
Not that I know of.
However, this was the era of “Don’t ask, don’t tell.” https://en.wikipedia.org/wiki/Don%27t_ask,_don%27t_tell
While that is the DoD, I believe it permeated all of government.
All these government agencies do their masters bidding. The senior leadership is a bunch of politicians and the decisions made are in synch with the political, ideological and even economic agenda of the day.
The most recent example is the reaction of the DOJ to the so-called 6 January insurrection. They can’t even define ANTIFA as a threat or organization. They immediately published a vast paper about Domestic Terrorism which of course is right on the edge of defining anyone opposed to the current agenda as a national security threat, but no word about Islamic terrorism:
They don’t react for months while federal court houses are smashed, streets are lit on fire, cars (example Portland)... Now they arrest >400 folks, interview everyone (even those that never entered the Capital building and did nothing wrong) for essentially “trespassing” and throw the books at them for protests that where largely peaceful, why do you think?