If you scroll down a ways (quite a ways), you'll find one of the more interesting bits of evidence: on a tomb, in the Vatican necropolis near (what is believed to be) St. Peter's tomb, is the inscription:
Peter, pray for the holy Christian men buried near your body
The tomb dates no later than AD 192.