Not a military threat, but definitely an economic and political threat. They would have eventually taken away New York's wealth and power, and the Empire City of the Empire state was never going to allow that to happen.
That is why the Empire City of the Empire state is still running Washington DC to this very day.
I believe the city of New York was ready to secede along with the Southern States:
"With our aggrieved brethren of the Slave States, we have friendly relations and a common sympathy. We have not participated in the warfare upon their constitutional rights or their domestic institutions. While other portions of our State have unfortunately been imbued with the fanatical spirit which actuates a portion of the people of New England, the city of New York has unfalteringly preserved the integrity of its principles of adherence to the compromises of the Constitution and the equal rights of the people of all the States. We have respected the local interests of every section, at no time oppressing, but all the while aiding in the development of the resources of the whole country. Our ships have penetrated to every clime, and so have New York capital, energy and enterprise found their way to every State, and, indeed, to almost every county and town of the American Union. If we have derived sustenance from the Union, so have we in return disseminated blessings for the common benefit of all. Therefore, New York has a right to expect, and should endeavor to preserve a continuance of uninterrupted intercourse with every section."
"It is, however, folly to disguise the fact that, judging from the past, New York may have more cause of apprehension from the aggressive legislation of our own State than from external dangers. We have already largely suffered from this cause. For the past five years, our interests and corporate rights have been repeatedly trampled upon. Being an integral portion of the State, it has been assumed, and in effect tacitly admitted on our part by nonresistance, that all political and governmental power over us rested in the State Legislature. Even the common right of taxing ourselves for our own government, has been yielded, and we are not permitted to do so without this authority."
[Fernando Wood, "Mayor Wood's Recommendation of the Secession of New York City." Teaching American History, Jan 6, 1861]
Apparently, the good Mayor promoted the doctrine of "working together as a team." Too bad the Northern merchants didn't agree. Our nation would have been both prosperous, and free.
Mr. Kalamata