Probably only a small number of people ever knew about it at the time, and all of those were veteran law enforcement. I'm sure that helped kept the secrecy pretty tight.
http://www.time.com/time/covers/1101020311/story.html
For a few harrowing weeks last fall, a group of U.S. officials believed that the worst nightmare of their lives—something even more horrific than 9/11—was about to come true. In October an intelligence alert went out to a small number of government agencies, including the Energy Department's top-secret Nuclear Emergency Search Team, based in Nevada. The report said that terrorists were thought to have obtained a 10-kiloton nuclear weapon from the Russian arsenal and planned to smuggle it into New York City. The source of the report was a mercurial agent code-named dragonfire, who intelligence officials believed was of "undetermined" reliability. But dragonfire's claim tracked with a report from a Russian general who believed his forces were missing a 10-kiloton device. Since the mid-'90s, proliferation experts have suspected that several portable nuclear devices might be missing from the Russian stockpile. That made the dragonfire report alarming. So did this: detonated in lower Manhattan, a 10-kiloton bomb would kill some 100,000 civilians and irradiate 700,000 more, flattening everything in a half-mile diameter. And so counterterrorist investigators went on their highest state of alert.
"It was brutal," a U.S. official told Time. It was also highly classified and closely guarded. Under the aegis of the White House's Counterterrorism Security Group, part of the National Security Council, the suspected nuke was kept secret so as not to panic the people of New York. Senior FBI officials were not in the loop. Former mayor Rudolph Giuliani says he was never told about the threat. In the end, the investigators found nothing and concluded that dragonfire's information was false. But few of them slept better. They had made a chilling realization: if terrorists did manage to smuggle a nuclear weapon into the city, there was almost nothing anyone could do about it.