I rode Amtrak or Conrail from Princeton Junction to Penn Station everyday and night, while working at Rock Ctr (70s & 80s) and on Madison Ave, for 17yrs, and we never had a problems. Trains are safe; it’s the damned engineers texting, dozing, calling etc, and not paying attention, which are today’s troubles.
I think it’s the local trains that are the problem. As a kid, I rode NJ transit from Manhattan to New Brunswick every damned day. We had endless problems in the 70s. We would always have to stop to let you Amtrak guys through, lol! That still goes on today!