Hmmm, I've always thought push the clip in straight and pop the caddy out..
You might be right. I haven’t done it in a while