As far as i know, City permits are not state permits, and thus not valid outside of the 5 boroughs.
You can absolutely transport your firearm onto long island. Gun must be in the trunk, locked container, with ammo in separate locked container while traveling through the city. As soon as you hit Nassau county, right back on your hip.
I live in western NY and do not have a NYC CCW permit so I never knew the parameters of a NYC CCW permit. I know that mine says not valid in the five boroughs, but assumed that if you have a NYC CCW permit that it was also valid outside as well.
Thanks for the update. A serious note to self.