Here’s an article that somewhat explains it.
The article explains that depleted uranium is not particularly dangerous as a radioactive source. It is dangerous for its chemical properties, similar to lead.
It basically has similar toxicity to lead. But the worldwide public at large sees the word ‘uranium’ and freaks out.