Him being given authority is not the same thing as founding the church upon him.
Or would you say that the kingdom if Israel was founded upon Saul? After all he was given the authority of king by God.
This is how the early Church fathers understood this. This was the same authority they used to put together the books in the Bible.
Again, authority is not equivalent to the founding.
They did not fall from the skies. They were sorted out after years of study, discussion, debate, and then by final authority of the successor to Peter as the “Word of God.”
Again, I'm not saying that Peter had no authority, or that things happened by magic [i.e. without study or work].
You must agree the profoundest of all authority is to authoritatively establish the Word of God in writing. In short, to assemble the books in the Bible. This is what the early Church fathers did using the sacred oral tradition, contemporary sources, and ritual. This deposit of authority was not to be spread around to every Tom, Dick and Harry who picks up the Bible and goes on a gallop of his own on how to interpret Scripture. This was to be done by virtue of Petrine authority. “Whatsoever, thou shall bind on earth….” You get the picture.