What you both said.
It’s odd how some things stick in memory. Many years ago — more than I want to remember — a college course (not in psychology) detoured through some of B.F. Skinner’s work. Skinner was still alive at that point and was a great faculty eminence at Harvard.
In one of his books, he posed a question to himself: how would he, on strict behaviorist grounds, account for human value judgements regarding, for example, a painting of a sunrise. In retrospect, I wonder how hard he had to work to come up with that example; he could have picked any question relating to meaning, values, purpose and artistic or moral judgment — the sorts of things that materialist reductionism has difficulty explaining.
Anyhow, I give him credit for posing the question and for acknowledging as he did so that it was a difficult question, and one for which he was dissatisfied with his answer. But the best he could offer on strictly materialist and behaviorist grounds was that our attraction to such a scene as a work of art might have something to do with “a metaphorical adumbration of the idea of survival value.”
I loved that statement enough to throw it back at the professor in a paper. Here was Skinner, a giant in the field and an uncompromising materialist, introducing three mentalist concepts — metaphor, idea and adumbration — in an attempt to explain an esthetic judgment.
Materialism founders on these sorts of questions. Thomas Nagel, another committed atheist, touched off a firestorm when he acknowledged the same problem a decade ago in “Mind and Cosmos: Why the Materialist Neo-Darwinian Conception of Nature Is Almost Certainly False.” The usual suspects tried to burn him at the stake for heresy. I do think there is another paradigm shift slowly building, as science encounters in several fields problems for which science cannot, in principle, offer satisfactory answers. That in no way lessens the power of science within its domain, but the culture will again be driven to the recognition that science cannot provide a satisfactory theory of everything. Science remains a subset of philosophy, not the other way around.
This paper heightened the awareness of subjective as opposed to objective views on reality.
Will AGI ever be able to have a subjective view? Or is the best we can hope for something like Dennett's idea that AGI and humans are merely very sophisticated adaptive filters in robot bodies?