Church was a community center/retreat for early Christians.
Yes, Pastor was the care taker, he would show you how to avoid getting caught, how to get food, and maintain your faith in the face of these threats.
God is where 3 or more gather in his name. That wasn’t the point of the early church.
I think the Church and the Pastor is primarily tradition based, based on the early persecution, saints, and the martyrs. It was a system of keeping Christians safe and together....which is the hope in which it still exists today.
Think of your pastor as you local safe-house overseer, he is supposed to know his church, his area, his flock, and be prepared to keep you safe in your faith.