I think the rapid onset of G is the biggest thing. A co-worker who is a former F-16 pilot told me that the highest number of G-LOC incidents happened in the T-37. T-37? It’s because it can rack up the G forces very fast even though it doesn’t pull high Gs.
Every pilot that flies high performance aircraft experiences "greying" in their vision under G from time to time. The hazard in front line fighters is that they can normally pull well in excess of 7.5 G's so quickly that the grey out warning never happens.