He was worried about being called a "racist" if he was mean to the poor little affirmative action idiot boy. I'm not joking. Obama's "blackness" was his most effective shield, because it kept his opponents from treating him as they would a white man.
Had Obama been white, (with the same record) Romney would have smashed him into a little greasy spot, but Romney (and McCain) held back because they were so worried about what all the beautiful people would think of them if they spanked poor little idiot Barry.
None of the R candidates running today would have done anything different ....except one.