If you take a picture with black+white film, a blaze orange hat will come out as blinding white!
Years ago I had my picture in the newspaper with a deer I shot. I looked like a moron wearing bright white hat during deer season.
Leaving all the science aside, as one who hunts mostly from bow stands during gun season, they see me in orange, grey to them, in the same positions they’re clueless of my presence in favorable winds. I’m no scientist, but I’m not sure wearing any shade of grey in the woods fools deer. They see you and don’t think giant squirrel. The safety issue is another thing, legitimate, probably not to be confused with deer eyesight.