You need to look at it in light of Matthew 16:19 (and 18:18): "And whatsoever thou shalt bind upon earth, it shall be bound also in heaven: and whatsoever thou shalt loose upon earth, it shall be loosed also in heaven."
In this you have Christ's promise to lend His Divine support to the acts of the Church.
I realize those are huge verses in Catholicism, along with John 20:22-23. I'm just not sure on what exactly is being transfered. I would say that only God has the power to forgive sins, and all binding and loosening comes from heaven, not from men. If one's position was that certain people were simply vessels for conveying all of this from God, without adding or subtracting anything based on their own opinions or humanity, then I would agree. However, if one's position was that God simply delegated His authority away from Himself, and let the individuals make human decisions based on their own opinions and knowledge, as a boss would delegate a task to a subordinate, then I would disagree.