It's simply NOT something that everyone is going to be able to build in their backyard. I'm not worried about everybody. I'm concerned about all the pissed off middle eastern grad. students I used to study physics with.
I'm concerned about the tens of thousands of unemployed Russian Biopreparat scientists.
You need 10-20 bright, resourcefull people and perhaps 50-100 million bucks.
Not all that rare these days.
And please don't call me gloomy. I'm just offering my view of the future of warfare. Its all about bang for the buck.
"You need 10-20 bright, resourcefull people and perhaps 50-100 million bucks. ... Its all about bang for the buck." 
Oh please. IF you were correct and it was ALL about bang for the buck, then terrorists would simply be using conventional explosives. Consider: The destructive power of the Daisy Cutter is equal to about 12,600 pounds of TNT. 2,400 Daisy Cutters would get you one Hiroshima-sized blast (14.5KT).
A Daisy Cutter costs $27,000 each. Thus, one could get the same explosion as a nuke for $64.8 Million worth of Daisy Cutters, with ZERO risk that your investment in "research" would fail to fission (which is obviously at risk if one pursues the nuke angle).
And Daisy Cutters can be built by almost every Third World country.