A good source for that are the letters of St. Ignatius of Antioch, as well as of course the letters of St. Paul to the Corinthians.
Well sure, we have the hierarchy of my pastor and staff members in my church. But in the very early days, did one Bishop control a whole region of churches? Was there a centralized Church government in Rome from the beginning? That just doesn't sound right to me.
One bishop ran the show in one large town. As the faith spread, a bishop had more and more presybters (elders, priests) helping them, and they delegated more and more duties to them.
The idea that one town would have more than one congregation (or parish), with one (or more) presbyters serving each one, developed later on.
In the early days, of course, there were only enough Christians for a single congregation in even the biggest towns, and few or none in the smaller towns and countryside. (Our word "pagan" comes from a Latin term equating roughly to "country bumpkin".)
The issue is whether the Early Church was hierarchical, not whether hierarchies also exist in Protestant settings. Also note that the quotes I gave you not simply set up admninistrative structures: they set up a liturgical and doctrinal structure as well, centered around the Eucharist and obedient to the bishop in all matters.
After St. Peter we had a short-lived papacy of St. Linus of which we know next to nothing; he was succeeded by St. Clement who was controlling things in Corinth over the heads of the local bishops, so definitely we had a papacy in the person of St. Clement, while the papacy of St. Peter is clear from the scripture.