Very true, good quote. But you perhaps think that this verse means all you need is Scripture --- Scripture and nothing else --- but that's not what it says.
They also had the example and the oral preaching of the Apostles, and the actual practice of worship and governance of the first (apostle-founded) Churches, which sustained the whole Church for several generations before the whole NT was written, collected and and canonized.
Otherwise, you'd be an a position of saying there was "no Church" before the NT Scriptures were completed, which would leave you with "no Church" until about 90-95 AD, when the epistles of John were completed. This is obviously untenable. There were hundreds of local churches scattered all over the Middle East, Turkey, Greece, northern Africa, etc --- founded on the preaching, teaching, and example of first and second generation missionaries, in the absence of the NT canon.
St. John says, several times, that a whole lot more was said and done, than got written down; and St. Paul says to be guided by the preaching and by his example and that of the other apostles and evangelists in their midst.
COMPLETE means something different to Rome; eh?