That’s the same sort of twisted logic that BLM and other groups used when they claimed that “Silence is violence”, etc.
Still, the perception among many whites is that a large percentage of blacks are hostile, and that there doesn’t seem much enthusiasm among the rest for pushing back against the hostility.
Compare that to the number of white people who marched for black civil rights and who speak out against racism.
It thus becomes natural for many whites to assume hostility until otherwise demonstrated.
And it then logically follows for white people to ask themselves “why should I be in favor of people who wish me harm (or at least will stand aside and not offer opposition to those who do) attaining any sort of power over me and my loved ones?”