You are absolutely right. But those following the doctrine of Sola Scriptura typically do not believe anything that is not specifically stated in the Bible. What a shame, for there is much more to the Word of God than just what appears in the Bible.
There is the living Word of God as well as the written Word of God. The Church founded by Jesus Christ (and BTW is written in Sacred Scripture) is testimony to the living Word of God. If someone other than Jesus Christ had founded the Church, then it could be argued that Sacred Scripture is the only authority given by God.
That's why you have the argument...Jesus founded the church, not The Church...
And what is the 'living word of God' outside of the scriptures???