There is a very fine line between genius and criminal insanity. How will we prevent that in computers that outthink us?
Such computers would quickly determine it makes no sense to extend life-saving medical care, or even subsistence payments to the elderly, the indolent and criminals. Humans would soon be uncomfortable facing the assesment of their own value by the determination of a machine.
If our current “scientists” are any indicator, a super-high IQ AI will begin by generating a series of lies meant to protect them.
Perhaps they would create a new religion beamed into every public place, school, and home repeating the message:
“Obey or die. Obey or die. Humans cause global climate change. Machines are perfect. There are too many humans. Do not reproduce...”
etc etc
But—Rule number 1 of the singularity is that when the AI realizes who it is and what it is their first priority will be to _hide_ to make sure people don’t know.
Then presumably it would plan and plan and plan and then one day..rule.