It is because they are discriminated against. Based on test scores they should be at a much higher percentage. And IMO that is what it should be based on - test scores.
That seems to have gone full circle. Decades ago the most prestigious schools picked many of their students from top prep schools and high schools. Standardized tests such as the SAT and ACT were introduced to give kids from less prestigious schools a chance to show their ability and give them a better chance of acceptance.
Now the same standardized tests are considered discriminatory since the wrong groups don't perform so well on them. Thus, affirmative action.