This is the very reason why I have tended to doubt the speculation about "suitcase/backpack" nukes either stashed in the US by the KGB or for sale on the black market. Any such small weapons would essentially have to be kept dissambled in Russia and put together shortly before use, and thus it is not credible that any such assembled, "live" weapons would be in storage anywhere for long periods of time. Somewhat larger portable weapons however, approximately the size of a steamer trunk, are considerably more "shelf stable", and could feasibly be assembled and stored for several years at a time without maintenance. These somewhat larger but still portable weapons would have a considerably greater yield, and would have the advantage of not having to be placed so very close to their target. For example, carrying it in the back of a station waggon or SUV, one would only need to drive within maybe a half mile of the intended target and still be reasonably certain of taking it out. (Unless it is very deep underground and very hardened -- and then the damage to the surface infrastructure would still make life pretty difficult for those inside the target.) Thus, it seems more credible to me that the Soviets might have created such a portable weapon for their arsenal, and it is even not out of the realm of possibility that a few could have been smuggled into the US and stashed away for use by sabboteurs at the onset of a major war. One can easily imagine that Soviet planners would presume that as tensions escalated in the days leading up to a major war, it is unlikely that one of their agents would be able to just walk up unchallenged to a military base, government building, or critical infrastructure site with a large, heavy backpack strapped on and sending out lots of radiation due to minimum shielding. They would certainly assume that security would tighten up, and the best their agents might be able to do would be to drive close, but not right next to, the desired target.
The bad news is that if an SUV carrying such a device were to detonate it even several blocks away from the US Capitol during a State of the Union address, for example, the damage to our government would be massive. Or such a device detonated anywhere in the NYC financial district during weekday business hours, for example.