Certain large cities were blessed with a multitude of passenger terminals (Chicago once had seven). Of course with most of the stations scattered, this was simply a curse for the passenger connecting through the city on lines with different terminals.
Chicago retains four terminals scattered around the periphery of the Loop, plus Els and subways, and you have to walk (or cab or bus) among them if the Loop is not your destination. There has been (mostly idle) talk in recent decades about a rapid-transit ring connecting the stations, but it's simply going to be too expensive, at least in this cycle of the Universe.
There have long been campaigns for a "North-South Rail Link" but nothing ever comes of it -- or is likely to after the major highway project "The Big Dig" wasted all that money.
New York uses Grand Central for commuter rail and (what once was) Penn Station for Amtrak. Years ago, though, some lines stopped at Hoboken across the river in New Jersey.
Back in Civil War times, though, cities were smaller and more of them may have had only one station than they had in the postbellum golden age of railroading. Now a lot of cities may be lucky if one working station survives.