A restatement of what I said to you. All you're saying is that "better" traits -- whether alone or in combination, lead to improved probability of reproductive success. Thus, "fitness" is still in the picture -- and your argument merely demonstrates your agreement with that claim.
Does evidence exist to support the "greater intelligence" claim? If not, then it is purely conjecture? Also, what is meant by "greater intelligence"? Is this a measure of speed of acquistion of knowledge, ability to solve problems or is some other metric used?
Empirically, one might point to the fact that sub-Saharan African countries tend to be (and have pretty much always been) savage sh*tholes, whereas non-African countries tend to be less so. One might also point to the positive correlation between IQ and income -- and note which races appear at the upper and lower points of each scale.
This still does not demonstrate the superiority of one race over another.
One could use the aforementioned observations to state that "racial superiority" has been demonstrated by empirical observation. (I don't hold to this conclusion --I'm merely showing that one can adduce the conclusion based on evidence.)
Getting back to the source of this, though, please note that we got onto this topic by your asking "How" the theory of evolution has been used to justify racist claims. The foregoing is one such example.
Another interesting example can be found in a couple of the Declarations of Secession, prior to the Civil War.
Mississippi's stated that:
These products are peculiar to the climate verging on the tropical regions, and by an imperious law of nature, none but the black race can bear exposure to the tropical sun. These products have become necessities of the world, and a blow at slavery is a blow at commerce and civilization. That blow has been long aimed at the institution, and was at the point of reaching its consummation. There was no choice left us but submission to the mandates of abolition, or a dissolution of the Union, whose principles had been subverted to work out our ruin.
Texas stated that:
We hold as undeniable truths that the governments of the various States, and of the confederacy itself, were established exclusively by the white race, for themselves and their posterity; that the African race had no agency in their establishment; that they were rightfully held and regarded as an inferior and dependent race, and in that condition only could their existence in this country be rendered beneficial or tolerable.
There is, of course, a lot of additional context to these declarations; the thing I'm pointing out to you, however, is the very explicit statements of racial superiority, and the obvious "evolutionary basis" from which those claims are drawn.