My perfect world would include having an "all" that is distinct and different from the self search, so that the two could be handled differently. If I want to see the things specific to my comments, I search on me (by the self search link). If I want to see the "all" comments, I choose to see them (previously, by searching articles+replies addressed to:"All").
Right now, for example, I get paged to this thread repeatedly, even though I asked the moderators to remove my post to this thread (which one of them did). I hoped that removing me from the thread would remove me from the all pings to the thread. It did not. That has to change at a minimum.
For a person who posts a lot (and I do, or at least did until recently) this "All" feature amounts to nuking our use of the self-search feature.