I think it varies on the denomination or synod.
In the LCMS, I know of a few pastors who were defrocked because of affairs. Not assault, affairs.
But that is easier to do the way we are set up than say the SBC where there is no central body that says “This guy can no longer be a pastor”.
I know for a fact that local SBCs can and do kick out wrong doers.