[JamesP81]

Self-awareness is not the most important question. consider, they already have developed a computer/computer program combination that can play pretty much equal with the best human chess player in the world. Chess used to be considered one of the highest measures of human intelligence.

It's my opinion that that was an invalid experiment. Gary Kasparov is a chess player. His environment is one where he sits down at the chess board across from another player and attempts to win.

Kasparov went into the game against big blue with same mindset he had always had. Play against another chess player.

He lost because that wasn't the game he went into. He went in playing against not a computer, but a programmer. Superficially, it seemed to be the same contest he was familiar with, but in truth it was totally different. It really wasn't a chess game anymore, but something else with chess as the window dressing.

The programmer better understood the rules and the environment of the contest than Kasparov did. If Kasparov had gone in with the mindset of defeating a programmer's toy at chess, there's a much higher chance he'd have won.

[Fitzcarraldo]

The programmer better understood the rules and the environment of the contest than Kasparov did. If Kasparov had gone in with the mindset of defeating a programmer's toy at chess, there's a much higher chance he'd have won.

That's the problem. The machine could "win" to the detriment of the human race. At some point even the programmer of SI will not be able to keep up.