I don’t disagree, but why a grocery store in a place where there certainly were no grocery stores in Victorian times?
It makes no sense.
I don’t know why but I think it has to do with a relationship with that particular location no matter what is eventually built there. They are from another time in the past and remain in that one location. There may have been Victorian homes there, or a cemetery, or maybe even just a family plot which was legal then, you could just go bury loved ones in the backyard per say. From what I understand the Civil war battle grounds are hot with ghost sightings and there are no buildings at all. And if you built something there later they would still be there from before.