That same form of argument could as easily be used to claim the church shouldn’t exist in more than one location, because there would be different people leading each location who could grasp some point of Scripture differently.
We are united with Christ. If a priest or denomination is in-between you and Christ, look out. That’s a problem.
What does the Bible say about division?
https://www.gotquestions.org/Bible-division.html