The Our Father prayer nor the Unnmerciful Debtor parable say that ONLY debts/offenses against them are to be forgiven. Besides, these are forgiven for the benefit of the forgiver: the Father then forgives him who is himself merciful. Further, while you can speculate that a forgiving heart will also forgive sins against others (you add "or us"), that part is not really in the prayer or in the parable. They are really about forgiving "debtors" or "my brother sin against me", that is in both cases sins against personally the one who is to forgive.
The commandment in John 20:23, however, speaks clearly of forgiving sins and the select disciples are sent in the person of Jesus (v.21) to forgive sins. They are also breathed in the Holy Ghost (22). But they already had the instruction to forgive sin against themselves from the teaching of Christ prior to resurrection. No special task would seem necessary if the purpose of this episode was to reiterate the importance of forgiving attitude. This, therefore, was a task, -- not a general moral commandment, -- given by Resurrected Christ as a part of the mission of the Church that He outlined in the last chapters of the gospels.
What will be the effects of the forgiving of sins commanded in John 20? The sins of those thus forgiven will be forgiven. This is consistent with the overall prerogative of the Church to to things on earth with the effect promised in heaven (Matthew 18:18; it does not say there that every believer but specifically the Church that can do so). This however is different from the forgiving heart commandment where the forgiver shall be therefore forgiven his, the forgiver's, sin. Next, the sins can be either forgiven or retained. But the forgiving heart commandment said nothing about retaining debts/sins against the disciple; in fact, the power to also retain sins, likewise taking effect in heaven, has nothing to do with the merciful attitude. The confessor priests sent in John 20 therefore are to become judges of sin. Finally, who is it whose sins are to be forgiven or retained? The scripture says "whoever", -- in contrast to the forgiving hear tepisodes whare it says "our debtors" and "sin against me".
The episode in John 20:21-23 explains that a select people: once receiving the Holy Ghost in that particular manner, are to not merely be of merciful attitude but also are to become judges of sin of others, regardless of who is the sin against, if anyone, to forgive the sins against God. This is the only power that belongs exclusively to God that these men are given as they are sent in the person of Christ. This is the power given them as priests.
You, for some reason ask specifically where that power is given St. Peter. The answer would be that in Matthew 16:19 a sweeping power to bind and loose is given exclusively to Peter with the keys; it is that power, given initially St. Peter as pope that is later expanded to the entire Church (Matthew 18:18)
From your next post: Do you think St. Paul and those he taught administer the Eucharist?
The correct term is "offered" or "celebrated". Not only I think so, I know so from the Holy Scripture. In 1 Cor. 11 St. Paul corrects a certain attitude spread among the Corinthians regarding the celebration of the Lord's Supper, which we now call the Eucharist. St. Paul reminds them of the words of consecration spoken by Christ at the Last Supper, the same words the priest repeats today as he elevates the consecrated bread. Then St. Paul concludes:
[26] For as often as you shall eat this bread, and drink the chalice, you shall shew the death of the Lord, until he come. Therefore whosoever shall eat this bread, or drink the chalice of the Lord unworthily, shall be guilty of the body and of the blood of the Lord. [...] [29] For he that eateth and drinketh unworthily, eateth and drinketh judgment to himself, not discerning the body of the Lord.(1 Cor. 11)
The point he makes is twofold: that the consecrated bread and the chalice "show the death" of Christ and are to be "discerned" as "the body of the Lord" and (2) if abused, it is Christ's real body and blood that suffer the abuse.
This episode shows that St. Paul understood and taught the correct Catholic doctrine of real presence of Christ in the Eucharist, that is he understood the distinction that the Protestant don't understand: between a memorial meal to remember the sacrifice of Christ and the actual Sacrifice becoming visible at the Holy Mass.