Catholics rely on a lot more that that verse. This is covered in the short form in "Rome Sweet Home" by Scott Hahn and more extensively in "Crossing the Tiber" by Stephen K. Ray. The entire question is EXHAUSTIVELY examined in "Upon this Rock"--also by Stephen K. Ray. Both authors are Protestants ministers who became Roman Catholic.
To put it in the "really" short form--the Protestants constantly tout the "Petros/petra" (Greek)language, because it can be twisted to make the verse sound like it's NOT referring to Peter. The problem is that Christ was speaking Aramaic at the time--where that "interpretation" is not possible.
By the same token, we're dealing with a translation of his words written decades later and so we don't really have the words of Jesus to examine and analyze.
This is one of those arguments I hate because it can't be definitively resolved. People tend to believe what they want to believe even when the evidence is indisputable.
It's my personal belief that no single church has an exclusive monopoly on salvation because believing that would mean that everyone born prior to the creation of that church would have no chance. There must be something deeper that governs that issue.
And since I believe that premise, the theological disputes between various branches of Christianity seem like interesting squabbles to me, but not much more serious than that.
That's just my take, and I know it's not widely shared.