The sin and excesses and false teaching that led to the reformation may well have moved that authority..
"And, since it is he that commits the government into their hand, they must administer it according to his will, for his glory; they must judge for him by whom they judge and decree justice, Prov. 8:15. And they may depend upon him to furnish them for what he calls them to, according to this promise: I will clothe him; and then it follows, I will strengthen him. Those that are called to places of trust and power should seek unto God for grace to enable them to do the duty of their places; for that ought to be their chief care. "
Matthew Henry
I would submit those keys were withdrawn. The Catholic church was no longer the sole voice of Chrisitanity..the voice was moved to the reformers.
Historically and logically. History shows an unbroken line of papal successors. Logically, if the Petrine office was destroyed, then the gates of hell would have prevailed against Christ's Church. Christ established the premier ecclesiastical office of His Church –the papacy or "vice-regency."