Who does Kim Jong Il think is the greatest long term threat? I don't think it is the US - the US is a tool to an end, i.e. the transition of North Korea. I think he is worried about China, which eventhough has seriously toned down its own rhetoric (since joining the WTO), will for the foreseeable future pose the most significant threat to the Korean peninsular, whether it be military or economic. Japan doesn't count - they are hated because of history, but they are a minor strategic entity. My prediction (!) is that Kim Jong Il wants a united nuclear Korea as protection from a growing China (I won't mention flashpoints such as the Spratley's). He neither seems irratic, nor mad to me. I always feel that when journalists start using such words, it is because they have failed to come up with an analytical theory/ don't have a clue. Kim Jong Il's methods may be scary, but he knows what he wants...
VRN
N. Korea does not bargain with good faith. That has been clear from the start. I do not favor the collpase of N. Korea as a nation but the current regime has shown no sign of pulling successful economic reform. So it may have to be replaced.
The problem with N. Korea is that it refuses to change its system in any meaningful way. At best, it will do a marginal change in glacial pace. In the meantime, N. Korea wanted to survive as northern steppe nomads used to do. Extorting tributes from well-off countries.
It all stems from the regime's fear of internal collapse once the change is under way. It wants an economic reform while still keeping its tight control on population, including the control of infomration flow and human contact from outside. That is what the reigme is afraid most, not American bombs. Even Chinese system is too dangerous for them, in their views. The regime caters to its oversized military, too. Unless their control is looasened and the military is reduced in its size, there will be no effective economic reform.
United Korea does not need N. Korean help for nukes. S. Korea has been trying to have one since '70's. Its nuke program was clandestinely alive until Kim Dae-Jung killed it. S. Korea, if she chooses to, can develop her missile and nuke programs to deter potential regional threats like China.
All S. Korea wants is for N. Korea to bait America in order to get tributes and negotiate away one bargaining chips(current missiles and nukes) while making better ones for future bargain as I stated. N. Koreans have already enough to destroy S. Korea, and other surrounding region with missiles, chemical and biological weapons. It also has huge conventional force.
The whole crisis comes from the fear that disastorous economic situation and gradual N. Korean general population's exposure to outside world will doom the regime from within, when you get down to the root cause.
Because of N. Korea backed themselves so hopelessly into a corner, there is no easy way out of it.
Finally, I am not one of those who see N. Korea as insane. Their action is all calculated, as you said. They have practiced it since 60's. But they are going for increasingly slimmer odds now. This act also entails a lot of risk to themselves and their neighoring countries. I always doubted that this N. Korean problem will end in some orderly fashion with minor bumps along the way. Whichever way N. Korean reigme plays it, it will give us a great scare even if there would be no shooting war. It is their style.