The problem many times has to do with delays from freight, as Amtrak relies on running over some of those same tracks. Amtrak needs its own tracks!
That may be part of the problem but the real problem is that Amtrak needs to just go away and let the market dictate need.
Amtrak is a pleasant, quaint but expensive thing that is heavily subsidized today. It is inefficient and a waste of money. On routes where ridership is high enough someone will provide the service.
A couple of years ago I looked into taking the train from Houston to Raleigh... waste of time. You could walk from Charleston to Raleigh in the time of the layover to catch the next train.
Amtrak should have been gone a long time ago and this comes from a minor train buff.