Memory?
History!
Grant was probably higher, but then he did a little more like defeating the slaveholders and during his presidency sending in the army to hunt down the KKK. But there wasn't as much exit polling as now.
And speaking of exit polling, who expects it to be any more accurate than phone polling? If anyone rushes me and demands to know how I voted, my answer will be something both obscene an anatomically unlikely.