The book version of “The Wanderers” was more explicitly racist by the standards of today, but keep in mind that Price was writing about the mileau he observed growing up in the Bronx at that time, as one of the few Jewish kids in a housing project with a large Italian and black population.
By that standard Black Like Me is a racist book.