Don’t think that there couldn’t be two feminists in the room with the guy? This is not a remote possibility, and it only takes one time to take someone down. The question is can a male boss have absolute trust in their female chaperone?
Who’s to say that they wouldn’t collude against their boss? Two statements against one? So now we need two males and two females in the room for a “private” meeting such as a performance review or salary discussion.
Seriously, it could come down to this. A special private meeting room with video, and glass separated window; similar to meetings of prison inmates and relatives. The feminist can be “protected” from harassment, and the male boss can be protected against defamation.
Not even considering the 'male' feminists, you would have to even pick the witnesses carefully or possibly end up with pyjama boy.