[lepton]

which dept/agency could Congress use to evict him?

The new President.

[tpaine]

which dept/agency could Congress use to evict him?

lepton wrote:

The new President.

Debatable point..
Only if the new, appointed president was accepted by the executive branch dept heads as the valid Chief Executive.
-- They too are obligated to defend the Constitution as written.

It is not written to allow impeachments for specious political matters.