The thinking isn't flawed at all. The degree of action and reaction is meant to be measured and "timed" (easier to unelect a president, harder -- but not unreasonable -- to impeach a judge for political reasons).
Eternal debate is always settled, eventually, by force or threat of force.
Of course, the present situation, with judicial activists interpreting the Constitution whatever way they please has also resulted in a Constitution with no meaning. What we need to do is go back to the situation we had until approximately the 50's, where the Supreme Court was the final authority on what the Constitution meant, but the Justices on the Supreme Court viewed themselves as bound by the language of the Constitution, and did not view themselves as exercising power to change things.
We can argue forever about what the law should be and endlessly revise and amend it. But we do need some surety about what the law is at any given time. Who owns the property? Who has the right to vote? If you can't give clear answers to such questions at a particular moment, then force rules (or else political and legal pettifogging and chicanery that eventually gives way to force).
In political debates people often assume that there are only two big alternatives: liberty, republicanism, or democracy on the one hand, and tyranny on the other. But the third alternative is the "failed state" which is too weak to impose law on anarchy. Such countries find it impossible to establish law and order, and usually collapse into tyranny.
For the Jeffersonian or libertarian political debate is a struggle between liberty and tyranny. For Washington or Hamilton, the prospect of a "failed state" torn apart by irreconcilable conflicts was a very real possibility and one that needed to be avoided. The idea of some arbiter who could compel the legislature to abide by the Constitution looked like a good idea, given the alternative of each faction taking up arms to prove that it had the right view of things. Whether or not the court has gone too far since the days of John Marshall is another matter.