I have told everyone I ever met that, if there is no rust covering the top of the rails, there will be a train coming through.
Expect a train on any track at any time in any direction. The tracks can look rusty after just a few hours in the right conditions.
I managed to survive 2 years of working on AMTRAK's high speed rail electrification project with that logic. A 150 mph electric train can really sneak up on you, especially with drill rigs, compressors & generators running all around a guy who is wearing hearing protection.