Because he wasn't a member of some useless party.
He was a de facto Federalist. The opposition to him was, you guessed it, the ancestors of the current Democrat Party.
Post of the thread.