That is true. If Ye, for example, were actually the anti-semitic character his recent speech implies then all of the people who associated with him, and did billions of dollars of business with him, should also be judged for their interactions with him. Obama endorsed Ye, and posed in photographs with Ye, and Adidas did hundreds of millions of dollars in business with him.
I’m reminded of the Christian leader that was flagged as a racist for saying Nelson Mandela did not speak for the majority of blacks in South Africa. Yet his statement was absolutely accurate.
https://www.nytimes.com/1990/08/24/world/zulus-vs-mandela-s-group-strife-sets-back-both.html