Thats your interpretation, and welcome to it: but since your notion is not supported by the text (beinst Jesus was talking to St. Peter and the Apostles with him), your interpretation is not dispositive.
No, it is what the passage and the context says.
Of course He is speaking to the Apostles. He is speaking about their role in the millennial kingdom.
No interpretation needed, unless you want it to mean something else...
But of course that doesnt change the passage.
On the other hand, the authority of the Church to settle disputed questions (to bind and loose)--- NOW --- is supported by the text:
"If he refuses to listen even to the Church, regard him as you would a pagan or a tax collector." (Matthew 18:17)
and
(James of Jerusalem) "It is my judgment, therefore, that we should not make it difficult for the Gentiles who are turning to God." --- and in the same context --- "For it seemed good to the Holy Spirit, and to us, to lay upon you no greater burden than these necessary things...(Acts 15:28)