Wrong...there were schisms occurring as early as the second century (Victor, Stephen, and Damascus - later the Acacian [AD 482], one under Patriarch Photios I of Canstantinople [AD 866], and continued through the Great Schism of 1054. It is a myth that there has been harmony amongst the patriarchs of the Early Church.
The discussion, however, is not about the adminitrative structure but the Church Fathers being Catholic, and in order to refute that you need to find a patristic teaching that is recognized either in the West or in the East, which would be contrary to Catholicism.