The EPA was established by an executive order. It can be eliminated- lock, stock and barrel- by exactly the same action.
All we need is the right Chief Executive.
Correct, but incomplete.
The laws that the EPA was created to enforce were passed by Congress and signed by various Presidents between 1955-1970. Those laws are broad in scope, grant enormous powers to the Executive Branch, and do not give the President authority to ignore them.
The EPA was a cost-saving measure, so that multiple Executive departments weren't wasting resources to enforce the same laws.
It's the laws that are the problem.