It does not agree with Scripture because it teaches things which are contradicted BY Scripture.
It doesn’t mean the Protoevangelium is wholly inspired or accurate.
If there's no certainty of inspiration or accuracy, then one is left picking and choosing which parts they WANT to believe and don't know if they're believing the parts which are fables or not. In that case, anyone who believes it is a fool.
So point out the contradiction (I responded to some alleged contraditions on the Protoevangelium thread already).