In 1869, the vast majority of Harvard students were legacies, who didn’t have to take the exam.
Harvard intentionally made the entrance exam so rigorous in order that they could keep “riff-raff” out.
It wasn’t till the mid 1920’s that middle class students started being a large percentage of the Harvard student body.
And it wasn’t until the GI Bill, that the Ivy League became more socially integrated.