Even Peter himself isn't referred to as having papal authority among the early post-apostolic sources. Terence Smith explains:
"there is an astonishing lack of reference to Peter among ecclesiastical authors of the first half of the second century. He is barely mentioned in the Apostolic Fathers, nor by Justin and the other Apologists" (cited in Robert Eno, The Rise of the Papacy [Wilmington, Delaware: Michael Glazier, 1990], p. 15)
Concepts of Petrine supremacy (as well as a primacy of Paul or James in some places, for example) did develop over time. Cyprian, for example, a bishop who lived in the third century, believed in a primacy of Peter, but it was a non-jurisdictional primacy (On the Unity of the Church, 4), and Cyprian repeatedly denied, in multiple contexts, that the bishop of Rome or any other bishop has universal jurisdiction (Letter 51:21, Letter 54:14, Letter 67:5, Letter 71:3, Letter 72:26). The Roman Catholic scholar Robert Eno wrote:
"it is clear that he [Cyprian] did not see the bishop of Rome as his superior, except by way of honor...it is clear that in Cyprian's mind, one theological conclusion he does not draw is that the bishop of Rome has authority which is superior to that of the African bishops" (The Rise of the Papacy [Wilmington, Delaware: Michael Glazier, 1990], pp. 59-60)
Roman Catholic scholar William La Due:
"In the context of his life and his convictions reflected in his actions and his writings, Cyprian's position can be paraphrased as follows: Peter received the power of the keys, the power to bind and loose, before the other apostles received the same powers. This priority - in time - symbolizes the unity of episcopal power which is held by all in the same way. The only difference is that Peter was granted the power a short time before the others. It must be said that the impact of Cyprian's symbolism is not entirely clear. He was not a speculative theologian but a preacher, trained more as a lawyer than as a rhetorician. His meaning, from the context of his conduct as a bishop, seems quite unambiguous. And those who see in The Unity of the Catholic Church, in the light of his entire episcopal life, an articulation of the Roman primacy - as we have come to know it, or even as it has evolved especially from the latter fourth century on - are reading a meaning into Cyprian which is not there." (The Chair of Saint Peter [Maryknoll, New York: Orbis Books, 1999], p. 39)
Catholic scholar Klaus Schatz:
"He [Cyprian] does not rely on any specific responsibility of Stephen [bishop of Rome] as primate....Cyprian regarded every bishop as the successor of Peter, holder of the keys to the kingdom of heaven and possessor of the power to bind and loose. For him, Peter embodied the original unity of the Church and the episcopal office, but in principle these were also present in every bishop. For Cyprian, responsibility for the whole Church and the solidarity of all bishops could also, if necessary, be turned against Rome." (Papal Primacy [Collegeville, Minnesota: The Liturgical Press, 1996], p. 20)
Even the conservative Roman Catholic theologian Ludwig Ott acknowledged:
"St. Cyprian of Carthage attests the pre-eminence of the Roman Church...However, his attitude in the controversy regarding the re-baptism of heretics shows that he had not yet achieved a clear conception of the scope of the Primacy." (Fundamentals of Catholic Dogma [Rockford, Illinois: Tan Books and Publishers, Inc., 1974], p. 284)
Eastern Orthodox scholar Veselin Kesich:
"In his controversy with Bishop Stephen (254-257), Cyprian expressed the view that any bishop, whether in Rome or elsewhere, was included in Jesus' message to Peter. Like Tertullian, Cyprian is unwilling to accept the claim of exclusive authority for the Bishop of Rome on the basis of Mt 16:18-19....Peter is not superior in power to the other apostles, for according to Cyprian all of them are equal." (The Primacy of Peter, John Meyendorff, editor [Crestwood, New York: St. Vladimir's Seminary Press, 1992], p. 63)
Anglican scholar J.N.D. Kelly:
"Cyprian made plain, that each bishop is entitled to hold his own views and to administer his own diocese accordingly...[In Cyprian's view] There is no suggestion that he [Peter] possessed any superiority to, much less jurisdiction over, the other apostles...While he [Cyprian] is prepared, in a well-known passage, to speak of Rome as 'the leading church', the primacy he has in mind seems to be one of honour." (Early Christian Doctrines [San Francisco, California: HarperCollins Publishers, 1978], pp. 205-206)
In Cyprian we see an example of a father who thinks highly of Peter and the bishops of Rome without believing in a papacy. In fact, he contradicted the concept. With Cyprian in mind as an example of how Catholics often misrepresent the fathers to make them appear to have supported the papacy when they actually didn’t, let’s consider the earliest evidence cited by Catholic apologists.
Clement of Rome, the earliest church father and a Roman bishop, sent a letter to the Corinthian church to counsel them about a dispute involving the leadership of their church. Such letters were common in early Christianity (Ignatius' letter to Polycarp, Polycarp's letter to the Philippian church, etc.), and no jurisdictional superiority, much less papal authority, is implied by the sending of such a letter. To the contrary, the letter is written in the name of the church of Rome, not the bishop of Rome, and the letter makes many appeals to various authorities (scripture, Jesus, the apostles, the Holy Spirit, etc.), but never to any papal authority. Thomas Halton comments:
"Some scholars anachronistically saw in the epistle an assertion of Roman primacy, but nowadays a hermeneutic of collegiality is more widely accepted." (Encyclopedia of Early Christianity, Everett Ferguson, editor [New York: Garland Publishing, Inc., 1999], p. 253)
Other early sources, such as Ignatius and Dionysius of Corinth, commend the Roman church for virtues such as love and generosity, but say nothing of any jurisdictional primacy of the Roman bishop. Irenaeus speaks highly of the Roman church, but gives non-papal reasons for doing so. Roman Catholic scholar William La Due comments:
"It is indeed understandable how this passage [in Irenaeus] has baffled scholars for centuries! Those who were wont to find in it a verification of the Roman primacy were able to interpret it in that fashion. However, there is so much ambiguity here that one has to be careful of over-reading the evidence....Karl Baus' interpretation [that Irenaeus was not referring to a papacy] seems to be the one that is more faithful to the text and does not presume to read into it a meaning which might not be there. Hence, it neither overstates nor understates Irenaeus' position. For him [Irenaeus], it is those churches of apostolic foundation that have the greater claim to authentic teaching and doctrine. Among those, Rome, with its two apostolic founders, certainly holds an important place. However, all of the apostolic churches enjoy what he terms 'preeminent authority' in doctrinal matters." (The Chair of Saint Peter [Maryknoll, New York: Orbis Books, 1999], p. 28)
Similarly, Tertullian gives non-papal reasons for the importance of the Roman church (The Prescription Against Heretics, 36). Regarding Origen, the Catholic scholar Robert Eno explains that "a plain recognition of Roman primacy or of a connection between Peter and the contemporary bishop of Rome seems remote from Origen’s thoughts" (The Rise of the Papacy [Wilmington, Delaware: Michael Glazier, 1990], p. 43).
The first reference to a papacy or something similar to it is found in the Roman bishop Stephen, acting in his own interests, around the middle of the third century. Peter had been dead for nearly two centuries before the doctrine first appears. When Stephen asserted it, he was opposed by bishops in the West and East, such as Cyprian and Firmilian. Thus, the papacy was absent, including in contexts where we would expect it to be mentioned, for about the first two centuries of church history, then arose in Rome and gradually became more widely accepted in the West and sometimes to some extent in the East. But even in the West, the papacy was accepted only gradually and inconsistently. Some of the earliest ecumenical councils would either imply or explicitly state a rejection of the doctrine. The Catholic scholar Klaus Schatz summarizes:
"Rome did not succeed in maintaining its position against the contrary opinion and praxis of a significant portion of the Church. The two most important controversies of this type were the disputes over the feast of Easter [in the second century] and heretical baptism [in the third century]. Each marks a stage in Rome's sense of authority and at the same time reveals the initial resistance of other churches to the Roman claim." (Papal Primacy [Collegeville, Minnesota: The Liturgical Press, 1996], p. 11)
It’s important to recognize that the early sources had many opportunities to mention a papacy if they believed in such a concept. When men like Clement of Rome and Tertullian comment on issues of authority and the status of the Roman church without mentioning a papacy, the absence of the concept is significant. When men like Ignatius and Irenaeus write at length on issues of authority and Christian unity, without even once mentioning a papacy, that absence is significant. They explicitly and frequently mention offices such as bishop and deacon. They explicitly and frequently make appeals to Jesus, the Holy Spirit, the apostles, prominent churches, and other authorities. They explicitly and frequently discuss the Messiahship of Jesus, the virgin birth, the resurrection, the unique authority of the apostles, and other basic Christian doctrines, so it can’t be argued that they didn’t mention a papacy only because it was already known to and assumed by everybody. The fact that other concepts were known and assumed didn’t keep the early sources from explicitly and frequently mentioning those concepts. Why didn’t they mention a papacy?
They did sometimes mention a prominence of the Roman church. And, thus, Catholic apologists have attempted to transform the prominence of the Roman church into a jurisdictional primacy of the Roman bishop. But if the papacy is an oak tree, the prominence of the early Roman church is more like an apple seed than an acorn. It has to be manipulated if we want to transform it into an oak tree. If the seed is being manipulated so as to arrive at a desired unnatural conclusion, then it’s not comparable to an acorn naturally growing into an oak.
The early prominence of the Roman church doesn’t logically lead to a papacy. The churches in Jerusalem, Rome, Alexandria, and other cities have been prominent at different times in church history for different reasons, and none of them can claim an apostolic jurisdictional primacy for their bishop as a result. It would be sort of like arguing that since the city of Philadelphia was prominent during the time of the founders of America, then the founders must have intended whatever authority claims the mayor of Philadelphia makes hundreds of years after the founders have died. If Ignatius thinks highly of the virtues of the Roman church or Tertullian commends the Roman church because some of the apostles labored and suffered in Rome, it doesn’t logically follow that these church fathers would agree with a later claim of universal jurisdiction by the bishop of Rome.