Madison’s the one who fled the White House while DC burned. He gets the credit for that. As for the Federalists, didn’t they basically just morph into the Whig Party, and then eventually the Republican party?
The Whigs were no more the heirs of the Federalists than ObaMao is the heir of Andrew Jackson.
The Whigs fielded their first presidential candidate in 1836, more than 20 years after the Federalists died. You may have heard of him, the war hero William Henry Harrison who managed all of 38% of the vote against the worthless Martin Van Buren, who went on to make most historians ten worst presidents lists.
The last pure* Whig presidential candidate was another war hero, General Winfield Scott, who managed 44% against Franklin Pierce in 1852 who (unfairly, I think) also generally makes that ten worst list.
The first Republican presidential candidate was explorer John C. Fremont four years later who managed 33% against 45% for James Buchanan, who (deservingly, at least until BO) makes #1 worst president on everyone's list.
Incidently, there was a thrid guy in that contest, Millard Fillmore, the last Whig President who garnered 22%. The big issue dividing the Whigs and Republicans at the time was slavery and the expansion thereof into the new territories. The Republicans took a hard line. The Whigs generally hated the institution but wanted to see it gradually phased out in an even handed manner so as not to provoke a civil war, the very postion which Lincoln tried to sell in the 1860 election with less than stellar results. People often forget there was a middle of the road party in that election as well which was called the Constitutional Union Party and carried three border states: Virginia, Kentucky and Tennessee.
So while it might be fair to say the remaining rump of the Whigs sort of morphed into the Republican Party, it is not fair to draw the same analogy between the Federalists and the Whigs because there was a clean break of 20 years and a total realignment of parties.
* Note: I say Winfield Scott was the last pure Whig candidate, because Millard Fillmore, four years later, understood the Whigs were pretty much dead and used the party label only in areas where it still had signs of life. His party was officially the American Party, dubbed by opponents as "Know Nothings". Fillmore was a far better president than history treats him, but that's another story for another day.