It seems to me that I looked into that a few years ago, and it seemed that two major parties was pretty natural, and became the situation pretty quickly.
The Whigs were pre civil war America, the party only lasted 22 years itself (1833 to 1856). The Republicans held their first convention on July 6, 1854 and four months later held 19% of the Congress and 25% of the Senate, six years later they held the majority in the Congress, the majority in the Senate, and they had the Presidency. The Republicans were never really a third party at all.
Before that, from the founding on, here is an article, covering the earliest years, “Two Parties Emerge”.
http://www.ushistory.org/us/19c.asp
Factions developed almost immediately in the Cabinet and Congress behind Hamilton and Jefferson, but they did not become organized political parties until George Washington’s body had been cold for at least a few days. Washington opposed the creation of British-style political parties with every fiber of his being, so the actors waited until he was gone before making their moves.