They were a bunch of Hitler appeasing isolationists, whom most Americans despised and shunned. America Firsters Charles Lindbergh and Joe Kennedy ended up disgraced, their reputations in tatters. Their "America First" rantings just covered up their bigotry.
Yelling America First has nothing to do with putting the USA above all, which most Americans do automatically, and that loyalty is not in question because they feel a connection on some level to their family roots in another country (as JFK did Ireland.)
It's just more Pat Buchanan-style bigotry, which is why Pat is basically a pariah.