Not politically correct but you got it.
Sure there are other factors, but I think that's the biggest, and the one with the most explanatory power. Fred Reed wrote a great column on this topic a couple of months ago.
In that article, he references a longer paper which has a lot of comparative detail. Social consequences of group differences in cognitive ability
These articles are about as un-PC as it gets.