I must admit I have never had a problem with any USB item whether I have safely removed it or not under XP. I am not the most careful in this regard.
I think the safe hardware removal tool is a obsolete artifact, but since MS continues to support some really old stuff, like from 1995, they leave themselves open to some really old problems. This is the weakness that Apple nuts exploit, and frankly speaking, it;s a gift from MS and I appreciate it cuz I have some really old stuff that I still cling to.....LOL