What defines the “Administrative State” is the control by long-term bureaucrats over the executive branch. The Deep State is the collection of bureaucrats who use their power to benefit themselves and their friends. Part of it is the “revolving door”, where officials retire from a department (leaving behind hand picked cronies) and go on to well paying positions in the companies they used to regulate.
People talk about “term limits” for elected officials. What we really need is the ability to clean out the bureaucracy. We need to empower new administrations to fire and replace Deep State people. We need to terminate civil service protections for administrators.
I think you have summed up the problem. The executive essentially becomes the implementation and enforcement arm of the Deep State bureaucracy.
Look at how the embedded Deep State “health bureaucracy,” unchanged for decades and even having been wrong for decades, took over the Presidency. And this happened under Trump, even though he had railed against the Swamp and the Deep State for year.
But he wasn’t strong enough to stand up to them, and no executive can be because there’s simply no way of getting rid of them as things are now.