The biggest problem would be breathing the radioactive dust from the explosion.
Again, how much of it would there really be? Uranium is only MILDLY radioactive.
Your body does NOT care that it is radioactive, it sees anything you breathe in as dust and will try to remove it just like non-radioactive dust.
A dirty bomb is all about FEAR, not actual damage.