Sounds to me like these folks butted up against land they didnt own and thought nothing would ever become of it. They expanded their yards/lot into this area and never thought about the usage changing.
Well usage did change and the owners (city) now say get your sheds and fence and all else out’s here!
Oh well
People do this all the time, and frequently get away with it like this couple did for 40 years. Folks that buy property that abuts National Forest land sometimes think they have the right to deny people access to that land, like it's their own private forest.