Perfect fourth, is it? Tell that to even such wonderful singers as Nat King Cole, who, marvelous singer that he was, butchered the tune in front of God and the nation before a World Series game in 1959.
I think the reason so many people dislike the tune is that they've heard it butchered too often by pop singers who feel like they have to add notes that aren't there, slide around, mess with the tempo, indulge themselves with LOOONNNNNNG high notes, and so forth. Best rendition I ever heard was a small high school choir (about 10 people) who did it in parts for an Atlanta Braves game back in the 80s at the old stadium. Strict tempo, straight tone, no foolishness. Sounded great.