Peter alone or Peter and his successors? You cannot have it both ways. If you are going to argue from the Scripture you need to actually pay attention to the Scripture says.
If you argue Peter and his successors, then you are in trouble, biblically speaking, since that error cannot be supported. You are back to your tradition as the supreme judge of all theology.
What we see in Mt. 16 is the new Davidic King bestowing the office of vice-regent. When an office-holder dies, the office does not die with him, but rather is bestowed upon someone else. Giving the keys to “Peter alone” is in the sense that the keys (and thus the office) belong to one man at any given time.