“Will it be sane? Able to respect truth and treat it logically?”
The reputable companies put a lot of effort into this. Even so, some even more conscientious researchers (e.g. Leopold Aschenbrenner) have left the industry based on similar concerns to your own.
https://www.axios.com/2024/06/23/leopold-aschenbrenner-ai-future-silicon-valley
(His 165-page paper is worth the time to read if you have some basic background).
One quick excerpt from your reference:
“9. Superintelligence “will be the United States’ most important national defense project.””
Will each country persue Superintelligence as their national defence project? Will their Si compete with our Si?
Will it promote its own country over others?
Seems like real Si would result in World Peace, convincing people that fighting is harmful, and cooperation is beneficial.
Will it decide that opponents to peace must be eliminated? (With Nukes, or some peaceful gas, disease, or starvation?) Will it have compassion?
How about the death penalty?
You have got me going. I’ll churn this for some time.