The main problem with a dirty bomb is that the news media will blow it (pun intended) all out of proportion.
Even Japan and Germany in WW2 had the ability to make dirty bombs, it's just that they were more educated and less sensationalistic than the journalists of today (i.e. they knew better than to deploy a militarily useless weapon).
If your radiation source isn't engaging in a chain reaction, then its lethality is limited to people either breathing it in as dust or hanging on to pieces of it like a souvenier trophy necklace.
Brief exposures to non-chain-reaction radiation sources are trivial to your health.
Sadly, our modern news media will scream "PANIC" and "NUCLEAR ATTACK" at the first detonation of a dirty bomb, when the reality is that a dirty bomb is just a bomb...whose dust you don't want to breath and whose schrapnel you don't want to wear as jewelry.