The problem wasn’t “hoarding”, it was the collapse of the Western Empire because of statism of the late Empire. The Eastern Empire had the same books and active scholars. As you mentioned, the Arab Muslims simply claimed achievements that weren’t theirs or built on the achievements of the Christian societies that they destroyed. The fading out of the “Golden Age” was just the fading out of the underlying Christian culture of the conquered lands. Muslims create nothing and can make nothing. BTW, the Arab conquerors often burned libraries because, as either the first or second caliph said, if a book is consistent with the Koran it is unnecessary, and if it isn’t it contains nothing of value.
'Statism' tends to last centuries. That was true in India, China, Egypt, Persia and the Roman Empire.
What killed the Roman Empire was Islam. It took over North Africa, which was the granary of the empire. It took over the Mediterranean Sea, which was the lifeblood of commerce for the Romans. All along the northern coast of the Mediterranean, and along the Atlantic coast up to England, folks started building defensively, moving uphill from the water, building more fort-like villas for protection from the Islamic raiders.
With commerce killed, there wasn't money left for big government.