I can’t sight-read Solesmes notation, but when my schola’s director starts-off a particular chant, I can follow along afterwards. I’m the same way when singing from regular musical notation. I can only sight-read when playing the piano.
I took piano from the age of 5, about the time I started singing in an (Episcopal) cathedral choir school. So I can sight read both. But I think they go through different parts of my brain. The piano music leaps straight from my eyes to my fingers, the vocal music drops down through my head . . . that doesn't make much sense, but that's what it feels like. Two different paths.