Imagine getting a job just based on your race/gender and nothing else. How would you feel about yourself? How do you handle how others will perceive you as unworthy of the job, having not truly earned it? To me, this only increases racism and division.
That is a real problem; like the Blade Runner replicants who become self-aware, the tokens realize they are just window-dressing - and it is destructive. They know they live a lie, even when non-token co-workers play along with the charade to pretend assimilation is working.