Other workers are also victims if they didn't give in to his sexual demands in order to receive promotions, doing skits, etc.
Also, the women who did have sex with him can also complain because if the favors they received were ones they should have gotten in the first place without having sex, then they are also victims.
What you are saying is that if someone had sex with the boss in order not to get fired they wouldn't have a leg to stand on, and this is not true. Neither is it true that if you have talent or should be promoted and they only way you can do it is sleep with the boss.
Either case is a sexual harassment case and if Letterman wasn't such a liberal, women's groups would already be coming down on him.
I agree, but the scope of his liability and exposure is fairly wide.