South Jersey is relatively rural compared to the north. It's politics and people are different. It's views are more aligned with PA and the states south of the Bay. During the Civil War, there was a vocal group of South Jersey residents that sided with the Confederates (aka Copperheads) and there was active debate of secession during that period.
As for Long Island's idea of secession, Islanders (with the exception of the Hamptons) have always been working class conservative that have been enraged by New Yorkers taking, blocking, and hogging everything. NYC basically blocks access to the island with excessive tolls while the capital, Albany, taxes her to death for social programs given to other parts of the state. Like South Jersey, Long Island's views are more aligned with places other than the city. Long Island is still rural compared to the City and it takes a number of hours to drive the length (days if the expressway known as the L.I.E. or more lovingly termed as The LIE, is snarled with holiday traffic).
Interestingly, many parts of both South Jersey and Long Island are pine barrens. And both rely on the sea for economic livelihood.
That is a very short history and probably not 100% current because I have been away for over 20 years.