I'd go with axing the agencies that aren't explicitly-named or implicitly-commissioned in the Constitution; that would be like the Treasury, the Post Office, the Navy, the Secret Service**, and [grudgingly] the IRS*. The Army is an intriguing special-case; by all rights it should be dissolved [into the States's militia] because its fundings aren't supposed to last more than 2-years... congress gets around this by continually budgeting the Standing Regular Army for two years every two years.
* Sadly the 16th Amendment is part of the Constitution, is it not?
** The SS is supposed to be anti-counterfeiting (one of the few Constitutionally explicit federal-level crimes).
Yes it is and so was probation but it was over turned (now we can drink :-)) and the “Fair Tax” can abolish the 16th Amendment (without the Prebate!!)
Your point about “I'd go with axing the agencies that aren't explicitly-named or implicitly-commissioned in the Constitution” is 100% correct and would help make our Government a lot smaller.