I think you need to balance where St. John Chrysostom speaks of St. Peter as the rock with other passages where he refers to St. Peter's faith as the rock.
Your controversialist whom you quote regarding Homily 54 on St. Matthew's gospel omits another key passage in the same treatise. "And I say unto thee, Thou art Peter, and upon this rock will I build my Church;"that is, on the faith of his confession. Hereby He signifies that many were now on the point of believing, and raises his spirit, and makes him a shepherd. "And the gates of hell" shall not prevail against it."
It frequently seems the Ultramontane apologists cherry pick for the patristic texts that seem to support their thesis about papal power, while ignoring those that don't support their position.
Seeing as you brought up Pope St. Victor I.
Here is what Guettee says:
"Let us take the evidence of History. The question having been agitated, "there were synods and convocations of the Bishops on this question," says Eusebius, Euseb. Eccl. Hist. Book V. chap. xxiii. "and all unanimously drew up an ecclesiastical decree, which they communicated to all the Churches in all places. . . . There is an epistle extant even now of those who were assembled at the time; among whom presided Theophilus, Bishop of the Church in Cesarea and Narcissus, Bishop of Jerusalem. There is another epistle" [of the 'Roman Synod] "extant on the same question, bearing the name of Victor. An epistle also of the Bishops in Pontus, among whom Palmas, as the most ancient, presided; also of the Churches of Gaul, over whom Irenæus presided. Moreover, one from those in Osrhoene, and the cities there. And a particular epistle front Bacchyllus, Bishop of the Corinthians; and epistles of many others who, advancing one and the same doctrine, also passed the same vote."
It is evident that Eusebius speaks of the letter of the Roman synod in the same terms as of the others; he does not attribute it to Bishop Victor, but to the assembly of the Roman Clergy; and lastly, he only mentions it in the second place after that of the Bishops of Palestine.
Here is a point irrefragably established; it is that in the matter of Easter, the Church of Rome discussed and judged the question in the same capacity as the other churches, and that the Bishop of Rome only signed the letter in the name of the synod which represented that Church. The partisans of the Papal authority affirm that it was Victor who commanded the councils to assemble. This assertion is altogether false. Among the Roman theologians who make this false assertion, we will particularly name Darruel in his work entitled Du Pape et de ses Droits. This book sums up all the errors and exaggerations of the Romish theologians.
Several Oriental Bishops did not conform to the decision of the others. Polycrates of Ephesus, above all, protested against it. Euseb. Eccl. Hist. Book V. chap. xxiv. Then a lively discussion arose between him and Victor, Bishop of Rome, who seemed to think that the Bishop of Ephesus would be alone in his opinion, and advised him, in consequence, to ask the opinion of the other Bishops of his province. Polycrates complied, and those Bishops declared themselves in favor of his opinion; he wrote thus to Victor, who threatened to separate them from his communion.
This did not move Polycrates; he replied vigorously, saying to him particularly. "They who are greater than I have said 'we ought to obey God rather than men.' Upon this Victor, the Bishop of the Church of Rome, forthwith endeavoured πειρᾰται ; Euseb.loc.cit. to cut off the Churches of all Asia, together with the neighboring Churches, as heterodox, from the common unity. And he published abroad by letters, and proclaimed that all the brethren there were wholly excommunicated." Thus Eusebius.
It is difficult to believe that the partisans of the Roman pretensions can find in these words of Eusebius and in the conduct of Victor any proof in favor of their system. Without much effort, they might find in them a proof to the contrary. The expression of Eusebius, that "Victor endeavoured," etc., must first be noticed. It is clear that those who endeavour have not in themselves the power to do that which they have in view, otherwise the act would follow the will. Victor, however, did all he could in order that this excommunication should be recognized—he even pronounced it; but that act remained but an attempt, and had to be ratified by the other Churches in order to be valid. Victor did not have, then, as Bishop of Rome, the power to excommunicate other Churches, since the effect did not follow the sentence which he believed himself entitled to give in the name of the Western Churches, because of the importance of his See.
The Bishops, who would have submitted to his sentence, if they had recognized in him the Head of the Church, invested with universal authority, not only did not obey him, but strongly censured his conduct.
"But this," adds Eusebius, "was not the opinion of all the Bishops. They immediately exhorted him," [Victor] "on the contrary, to contemplate that course that was calculated to promote peace, unity, and love to one another."
Thus, instead of believing that unity consisted in union with Victor, the bishops exhorted him to observe better the true notions of unity. Many went even further. "There are also extant," continues Eusebius, "the expressions they used, who pressed upon Victor with much severity. Among these also was Irenæus, who, in the name of those brethren in Gaul, over whom he presided, wrote an epistle in which he maintains the duty of celebrating the mystery of the resurrection of our Lord only on the day of the Lord. He becomingly also admonishes Victor not to cut off whole churches of God who observed the tradition of an ancient custom." Irenæus endeavored to show to Victor that differences in practice, of which, he gives divers examples, are not inconsistent with Unity of Faith. "And when," adds Eusebius, "the blessed Polycarp went to Rome in the time of Anicetus, and they had a little deference among themselves likewise respecting other matters, they immediately were reconciled, not disputing much with one another on this head. For neither could Anicetus persuade Polycarp not to observe it, because he had always observed it" [a certain custom] "with John the disciple of our Lord, and the rest of the Apostles with whom he associated; and neither did Polyearp persuade Anicetus to observe, who said that he was bound to maintain the practice of the presbyters before him. Which thing being so, they communed with each other; and in the church Anicetus yielded to Polyearp the office of consecrating." And thus, though following different usages, all remained in the communion of the Church. "And not only to Victor, but likewise to the most of the other rulers of the churches, he sent letters of exhortation on the agitated question." Euseb. Eccl. Hist. Book V. chap. xxiv.
Thus Victor could not, of his own authority, cut off from the Church, in fact, those whom he had declared excommunicate; the other Bishops resisted him vigorously, and St. Irenæus, the great divine of the age, made war in his letters upon those which Victor had written to provoke the schism.
This discussion, invoked by the partisans of Papal pretensions in their favor, falls back upon them with all its weight, and with a force that can not in good faith be contested.
Anicetus did not invoke his authority against Polycarp, nor did Victor against Irenæus and the other Bishops. Polycarp and Irenæus reasoned and wrote as equals of the Bishop of Rome in Episcopal authority, and recognized but one rule—ancient tradition.
How were the Churches reünited in a common practice? Eusebius thus relates that happy result, which certainly was not due to the Bishop of Rome: Euseb. Eccl. Hist. Book V. chap. xxv.
"The Bishops, indeed, of Palestine, Narcissus and Theophilus, and Cassius with them, the Bishop of the Church at Tyre, and Clarus of Ptolemais, and those that came together with them, having advanced many things respecting the tradition that had been handed down to them by succession from the Apostles, regarding the Passover, at the close of the epistle use these words: 'Endeavor to send copies of the epistle through all the Church, that we may not give occasion to those whose minds are easily led astray. But we inform you also, that they observe the same day at Alexandria which we also do; for letters have been sent by us to them and from them to us, so that we celebrate the holy season with one mind and at one time.'"
Nevertheless, many Churches preserved the tradition of the Churches of Smyrna and Ephesus, and were not on that account regarded as schismatics, although Victor had separated himself from their communion.
The partisans of the Papal system attach much importance to the influence exercised by the Bishop of Rome in the question of Easter and some other matters: they transform that influence into authority. This is an untenable paralogism. It is not to be wondered at that the Bishop of Rome should have enjoyed from the first a high influence in religious questions; for he filled the first See of the West, and as Bishop of the Capital of the Empire, he was the natural link between East and West. It was then understood that the Catholic Church was not exclusively in any country; that the East possessed no more universal authority than the West. This is why certain heretics, born and condemned in the East, sought protection in the West, and above all at Rome, its representative. Thus it is, that even some saints—as Polycarp of Smyrna—went themselves to Rome to confer with the Bishop of that city upon religious questions.
But it is not possible conscientiously to study these facts from reliable documents without eliciting this truth: that the influence of the Bishop of Rome did not arise in an universal authority—that it did not even have its source in an authority recognized by all the Western Churches, but was simply derived from the importance of his See.
Rome was the centre of all communications between different parts of the Empire. The faithful crowded thither from all quarters—for political business or private interests—and thus her testimony as an Apostolic Church was strengthened by the faithful who came thither from all parts of the world, bringing the witness of all the Churches to which they severally belonged.
Such is the sense of a passage of St. Irenæus, of which the Roman theologians have made the strangest misuse. St. Iræneus, In Hæres. Lib. III. cap. iii. This great theologian, attacking the heretics who sought to corrupt the faithful at Rome, establishes against them the Catholic rule of faith, preserved everywhere and always." But," he adds, "as it would be very tedious to enumerate in such a work the succession of all the Churches, we will trace that of the very great and very ancient Church and known of all, which was founded and established at Rome by the two very glorious Apostles, Peter and Paul; which possesses a tradition that comes from the Apostles as much as the Faith declared to men, and which has transmitted it to us through the succession of her Bishops; by that, we confound all those who in any manner whatsoever, either through blindness or bad intention, do not gather where they should; for every Church, that is to say, the faithful who are from all places, are obliged to go toward that Church, because of the most powerful principality. In this Church, the tradition of the Apostles has been preserved by those who are of all countries." We must quote the text of St. Irenæus, that it may be compared with our translation, "Quoniam valde longum est, in hoc, tali volumine omnium eccelesiarum enumerare successiones; maximæ et antiquissimæ et omnibus cognitæ, a gloriosissimis duobus apostolis Petro et Paullo, Romæ fundatæ et constitutæ Ecclesiae, eam quam habet ab Apostolis Traditionem et annunciatam hominibus fidem, per successiones Episcoporum pervenientem usque ad nos, indicantes confundimus omnes eos, qui quoquomodo, vel per coecitatem et malam sententiam præterquam oportet colligunt. Ad hanc enim Ecclesiam, propter potentiorem principalitatem, necesse est omnem convenire ecclesiam, hoc est eos, qui sunt undique fideles; in qua semper ab his qui sunt undique, conservata est ea, quæ est ab Apostolis, Traditio."
The [Ultramontane] theologians choose a bad translation of this passage, in order to find in it an argument in favor of the papal sovereignty. Instead of saying that the faithful of the whole world were obliged to go to Rome, because it was the Capital of the Empire, the seat of government, and the centre of all business, civil and political, they translate convenire ad by the words, to agree with—which is a misinterpretation; they make potentiorem principalitatem refer to the Church of Rome, and they see in this its primacy, whereas these words are only used in a general manner, and nothing indicates that they do not solely designate the capital and principal city of the Empire. Again, they translate, maximæ, antiquissimæ, by greatest and most ancient, without reflecting that they thus attribute to St. Irenæus an assertion manifestly false; for, granting that the Church of Rome was the greatest of her day, she could not certainly be called the most ancient—every one knew that a great number of churches had been founded in the East before that of Rome. Moreover, their translation does not make the author say in conclusion, that the Apostolic tradition has been preserved at Rome, by those who were of all countries—(ab his qui sunt undique,) as the text requires, but like Pius IX, in his Encyclical Letter to the Christians of the East, "In all that the faithful believe," not reflecting that this is a misconstruction, and that they are thus attributing nonsense to the good Father.
In the text as we render it all things hang together. St. Irenæus after having established that only the universal Faith should be received, points out to the heretics of that city the Church of Rome, as offering to them an evidence the more convincing that Apostolic tradition had been there preserved by the faithful of the whole world.
How then could St. Irenæus, whose purpose it is to give the universal Faith as the rule for private belief, and who enlarges precisely upon this point in the chapter from which the text is taken, logically say what is attributed to him by the Popes and their theologians? He would then have argued thus: It is necessaryto adopt as the rule the belief of all the churches; but it suffices to appeal to that of the Church of Rome, to which there must be uniformity and submission, because of her primacy. St. Irenæus never expressed so unreasonable an opinion. He lays down as a principle the universal Faith as a rule, and he points out the Faith of the Church of Rome as true—thanks to the concourse of the faithful who assembled there from all parts, and who thus preserved there the Apostolic tradition. How did they preserve it? Because they would have protested against any change in the traditions of their own churches, to which they were witnesses at Rome. St. Irenæus does not give the pretended Divine authority of the Bishop of Rome, as the principle of the preservation of tradition in the Church of that city—but logically, he attributes that preservation to the faithful of other Churches who controlled her traditions by those of their own Churches, and who thus formed an invincible obstacle to innovation.
It was natural that the Bishop of the Capital of the Empire, precisely because of the faithful who there gathered from all parts, should acquire a great influence in religious matters, and even occasionally take the lead. But all the monuments, as also the circumstances attending, those transactions in which he took part, show that he enjoyed no authority superior to that of the other Bishops." The only direct reference in existence about St. Peter personally transferring his authority to the Bishops of Rome comes from the Clementine Homilies, which has been proven forgeries. The Pope is only St. Peter's successor as bishop of Rome in the same way that the Patriarch of Antioch is St. Peter's successor as Bishop of Antioch.
I just excerpted the sermon, its available online.
Here is what Guettee says:
I'm really not interested in what he says. He has no love for the truth. That was the point of what I wwas establishing. One could make similar points with numerous other "citations" of his, which he takes out of context, ignores, or wrongly understands. But why bother? The truth is well enough known.