The white guy that didn't say anything about race or skin color has to now genuflect and supplicate before an organization specifically based on skin color, and an organization to which he would not be welcome as a member because he has the wrong skin color.
From the black nationalist perspective, African-Americans and Hispanics “own” poverty and the ghetto. Because of past history, they believe that the supposed white-run federal government is responsible for providing them with reasonable employment or a fair income in lieu of jobs, as well as housing when they can’t afford it on their own.
Their rationale is that the modern federal government (i.e. taxpayers) still owes them compensation for white colonial slave owners and the permanent damage they inflicted on black culture. It’s written in their own literature and spread through racist indoctrination centers such as Trinity Church in Chicago, the Abyssinian Church in NYC, and in every major city throughout the country.
Ryan insulted this belief, which is represented as a reasonable expectation by the CBC when they visit their home districts, by implying that all individuals are responsible for their own work and income, regardless of color. That flies in the face of the race-based excuses and justifications that have been sold to black voters in depressed urban areas, in order to explain their plight. It can’t be their responsibility, therefore Ryan has insulted them.
That racial mythology and biased perspective is upheld by their community leaders and elected representatives, when they meet with their constituents. But that’s not the public face we see when they come to Washington.