Gladstone was no different that a lot of people back then, or today for that matter. They dislike people as a group, in this case Americans, but had individuals as friends. In Gladstone's case it was people like Charles Sumner.
Well, Sumner was also a personal friend of the Lincolns. He liked to come to Lincoln's office to shoot the breeze, and he and Mary Lincoln used to converse in French.