For three centuries after Christ died there was no Pope, no Vatican, and no ‘Bible’ and yet the first Christians were utterly devoted to God and worshipped him at the risk and cost of their lives.
There was a bible (scripture) available to the believers very early on.
The Old Testament was available before Christ was born.
Paul's letters were considered Scripture prior to 67 AD.
The books of the NT were all written prior to the end of the 1st century.
The NT as we know it was generally in place by then. We can see in the writings of the Early Church Fathers the references to these books.
But, you are correct in there was no pope, no vatican, no marian dogmas, no mass....in other words, we don't see the Roman Catholic church in the early church.