I dislike Clinton as much anyone who's ever lived, but I don't contrue these comments as racists, but a valid observation about his experience.
What makes thiss comment deliciously ironic, IMO, is that it was made to Ted Kennedy, who, when he was first elected served alongside Ed Brooke, senior Republican senator from Massachusetts, who was not only good enough to get elected , back in the day Clinton refers to, but apparently good enough to sleep with one of the top reporters of the time, if Barbara walters is to be believed.
Once again its established exactly which side of the aisle is chock full of racists and it is not the right side. BTTT! ;-)