I think I found the cave, in Yonkers, just north of the Bronx:
Former employees of nearby St Johns Hospital can still recall nights when chanting and torch flames were seen and heard in the depths of the woods, especially from the area of the now-demolished Devils Cave. There are those who maintain that harmless teenagers were the only ones frequenting the backwoods at night during the Seventies, but that belief flies in the face of some disturbing facts . . .
I missed the seguey - what does a creepy park have to do with masonry?