Sounds a little like Railroad Ave, in Ashland VA. Ashland has some restaurants and a hotel along Railroad Ave which has a railroad track running down the center of the road. Part of the avenue has some historic houses on each side.
I know the Amtrak Auto Train uses the track, but I’m not sure how much freight passes over it.
The spot for Dave’s Depot & Diner is a railfan’s dream. To date there is no safe place to watch two Class I railroads in operation.
If this idea comes to fruition it will be on the map not only for train lovers of all kinds, but people who want a great meal without a lot of noise.