I've lived in New York City all of my 60 years. As a onetime geology major in the 1980s and avid camper, I've been Upstate many times (Catskills and Finger Lakes mainly). Lived in Central NY for ~two years in '89, in an old farmhouse in Beaver Meadow.
Yes, the Catskills are basically a large set of sedimentary beds atop layers of fossiliferous limestone. Where rivers and/or roadcuts dig deep enough the limestone sequences are exposed. This occurs mostly along the perimeter of the Catskills.
Yeah I go to the town of Barton near Waverly around 3 miles north of the PA border. I have found some interesting rocks there.