Have you also been reminded many times that two plus two make four, and have you failed to learn that fact as well?
Two words, Steve: Social Darwinism.
The idea was inspired by the theory of evolution, as you well know. As were a whole host of other eugenic (and economic) theories.
I suppose you can ignore the fact that ideas -- even the theory of evolution -- have moral consequences. But of course, anything that has a practical application carries with it a set of moral implications. I rather doubt that you'd claim that evolutionary theory has no practical applications, which leaves you with the problem of dealing with its moral implications.
I'd wager that you are not in favor of Social Darwinism. But in order to be against it (or for it, for that matter), you must make the argument why evolutionary theory cannot (or can) be applied to humankind. The moral implications just sit there, unable to be ignored.