The thing any reasonable thinking person has to keep in mind when reading those words is that they were entirely mainstream for the times. Judging history - or historical characters - through the lens of contemporary times is the height of foolishness.
Lincoln was no more a racist than any of us, and certainly less than any of the southern insurrectionists.
It should also be noted that Abraham Lincoln's views on race changed throughout his life. The views of the 1840's Lincoln weren't the same as his views later in life. He was heavily influenced by Frederick Douglass.
Agreed that Lincoln’s views can’t be judged by today’s light. In that same speech (that I linked to) Lincoln made some really funny comments on how he would not change the law which forbid inter-racial marriage, (as without that law, it sounded like Douglass and his friends would be tempted to marry blacks). Lincoln also said that these laws were best done by the states, and suggested that Douglass run for state office instead so he could fight the changes on that level.
Lincoln:
“I will add to this that I have never seen, to my knowledge, a man, woman or child who was in favor of producing a perfect equality, social and political, between negroes and white men.
I recollect of but one distinguished instance that I ever heard of so frequently as to be entirely satisfied of its correctness—and that is the case of Judge Douglas’s old friend Col. Richard M. Johnson. [Laughter.] I will also add to the remarks I have made (for I am not going to enter at large upon this subject), that I have never had the least apprehension that I or my friends would marry negroes if there was no law to keep them from it, [laughter] but as Judge Douglas and his friends seem to be in great apprehension that they might, if there were no law to keep them from it, [roars of laughter] I give him the most solemn pledge that I will to the very last stand by the law of this State, which forbids the marrying of white people with negroes. [Continued laughter and applause.]
I will add one further word, which is this: that I do not understand that there is any place where an alteration of the social and political relations of the negro and the white man can be made except in the State Legislature—not in the Congress of the United States—and as I do not really apprehend the approach of any such thing myself, and as Judge Douglas seems to be in constant horror that some such danger is rapidly approaching, I propose as the best means to prevent it that the Judge be kept at home and placed in the State Legislature to fight the measure. [Uproarious laughter and applause.]”