No. There is a good deal more inferential Scriptural evidence, as well as historical evidence from the very earliest days of the Church, but that verse is sufficient in itself.
I've seen MANY Protestant positions justifed based on a lot flimsier scriptural evidence.
Two good books:
"Upon This Rock"- Steve Ray
"Jesus, Peter and the Keys" by Scott Butler, Norman Dahlgren, and David Hess
Both of which have scores of references to scripture, history, and linguistics. The evidence is incontrovertible.
I'm sure that same answer is used to validate the "Hail Mary full of grace" seed of the whole Marian doctrine thing. Such "evidence" has never really materialized though I've read many RC apologetics that try to support Marianism.
And, by the way, the verse is in no way sufficient in itself. There is no support that anything that the Lord may have given to Peter at that time survived Peter.