The DoJ needs to be funded to some extent because federal laws need to be enforced at the courthouse.
The FBI is dispensable. Indian tribes have their own police forces, or could have. DC has its own police force. States can go after bank robbers.
The DOJ doesn't even need its own building in the District of Columbia.
It didn't even exist until 1870. Before that, the Attorney General and the U.S. Attorneys for the various districts operated under the Treasury Department.
When the AG position and the District Attorneys were established around 1790, the only Federal crimes they needed to enforce were treason and tax evasion. The legal offices of the U.S. government functioned mainly as real estate attorneys to process transfers of property titles for the purchase and sale of Federal lands.
The first order of business to restore America as a free nation would be to eliminate at least 98% of the U.S. Criminal Code.