One central AI is not a good idea. What is a good idea is creating something similar to the International Atomic Energy Agency to do the same thing it does for atomic energy and nuclear weapons. We need a strong, verifiable AGI non-proliferation treaty. It would be a pain. Everyone would lose their rights to unlimited numbers of hundred thousand dollar H 100 GPUs and the like, but it would be worth it in the end.
And just who is going to do this "verifying" AND (you don't mention it) ENFORCING??? Why it's the Global Thought Police!
Do you really think you can sell this crap here? With the potential for the excercise of power like that, do you really think for one second that no one will try to overstep those bounds because of a treaty?
So you would constrain ALL of those who would exercise constraint, under the presumption that, lacking the power to inflict appropriate punishmnet, whoever sits at the top of this fantasy heap will. Under treaty no less, supreme law of the land with no recourse, as if a President constrained to the Constitution and Laws authorized under Article I Section 8 could be authorized to negotiate such a power!!!
I will hopefully conclude you haven't thought about these things very deeply, so allow me to assist you.
There is an alternative to this thought process; either nobody has such power or most everybody does. Being a fan of the Second Amendment; to my mind; "arms" includes intelligent agents, with their miscreant directors to be punshed by the means advised on my tag line.