Could a machine think?

The year is 2100. Geena is the proud new owner of Emit, a state-of-the-art robot. She has just unwrapped him, the packaging strewn across the dining room floor. Emit is designed to replicate the outward behaviour of a human being down to the last detail (except that he is rather more compliant and obedient). Emit responds to questions in much the same way humans do. Ask him how he feels and he will say he has had a tough day, has a slight headache, is sorry he broke that vase, and so on. Geena flips the switch at the back of Emit’s neck to ‘on’. Emit springs to life.

EMIT: How are you? Personally I feel pretty good. A little nervous about my first day, perhaps. But good. I’m looking forward to working with you.

GEENA: Now look, before you start doing housework, let’s get one thing straight. You don’t really understand anything. You can’t think. You don’t have feelings. You’re just a piece of machinery. Right?

EMIT: I am a machine. But of course I understand you. I’m responding in English aren’t I?

GEENA: Well, yes you are. You’re a machine that mimics understanding very well, I grant you that. But you can’t fool me.

GEENA: Because you have been programmed to respond to spoken commands. Outwardly you seem human. You look and behave as if you have the understanding, intelligence, emotions, sensations and so on that we human beings possess. But you’re a sham.

GEENA: Yes. I’ve been reading your user manual. Inside that plastic and alloy head of yours there’s a powerful computer. It’s programmed so that you walk, talk and generally behave just as a human being would. So you simulate intelligence, understanding and so on very well. But there is no genuine understanding or intelligence going on inside there.

GEENA: No. One shouldn’t muddle up a perfect computer simulation of something with the real thing. You can program a computer to simulate a thunderstorm but it’s still just that - a simulation. There’s no real rain, hail or wind inside the computer, is there? Climb inside and you won’t get wet. Similarly, you just simulate intelligence and understanding. It’s not the real thing.

Is Geena correct? It may perhaps be true of our present day machines that they lack genuine understanding and intelligence, thought and feeling. But is it in principle impossible for a machine to think? If by 2100 machines as sophisticated as Emit are built, would we be wrong to claim they understood? Geena thought so.

GEENA: No you don’t. You have no beliefs, no desires, and no feelings. In fact you have no mind at all. You no more understand the words coming out of your mouth than a tape recorder understands the words coming out of its loudspeaker.

GEENA: Hurting your feelings? I refuse to feel sorry for a lump of metal and plastic.

Geena explains why she thinks Emit lacks understanding. She outlines a famous philosophical thought experiment.

GEENA: The reason you don’t understand is that you are run by a computer. And a computer understand nothing. A computer, in essence is just a device for shuffling symbols. Sequences of symbols get fed in. Then, depending on how the computer is programmed, it gives out other sequences of symbols in response. Ultimately, that’s all any computer does, no matter how sophisticated.

GEENA: Yes. We build computers to fly planes, run train systems and so on. But a computer that flies a plane does not understand that it is flying. All it does is feed out sequences of symbols depending upon the sequences it receives. It doesn’t understand that the sequences it receives represent the position of an aircraft in the sky, the amount of fuel in its tanks, and so on. And it doesn’t understand that the sequences it puts out will go on to control the ailerons, rudder and engines of an aircraft. So far as the computer is concerned, it’s just mechanically shuffling symbols according to a program. The symbols don’t mean anything to the computer.

GEENA: Quite sure. I will prove it to you. Let me tell you about a thought experiment introduced by the philosopher John Searle way back in 1980. A woman is locked in a room and given a bunch of cards with squiggles on. These squiggles are in fact Chinese symbols. But the woman inside the room doesn’t understand Chinese - in fact, she thinks the symbols are meaningless shapes. Then she’s given another bunch of Chinese symbols plus instructions that tell her how to shuffle all the symbols together and give back batches of symbols in response.

EMIT: That’s a nice story. But what’s the point of all this symbol-shuffling?

GEENA: Well, the first bunch of symbols tell a story in Chinese. The second bunch asks questions about that story. The instructions for symbol-shuffling - her ‘programme’, if you like - allow the woman to give back correct Chinese answers to those questions.

GEENA: Right. Now the people outside the room are Chinese. These Chinese people might well be fooled into thinking that there was someone inside the room who understood Chinese and who followed the story, right?

GEENA: But in fact the woman in the room wouldn’t understand any Chinese at all, would she?

GEENA: She wouldn’t know anything about the story. She need not even know that there is a story. She’s just shuffling formal symbols around according to the instructions she was given. By saying the symbols are ‘formal’ I mean that whatever meaning they might have is irrelevant from her point of view. She’s simply shuffling them mechanically according to their shape. She’s doing something that a piece of machinery could do.

EMIT: I see. So you are saying that the same is true of all computers? They understand nothing.

GEENA: Yes, that’s Searle’s point. At best, they just simulate understanding.

GEENA: Of course. All computers, no matter how complex, function the same way. They don’t understand the symbols that they mechanically shuffle. They don’t understand anything.

GEENA: That’s right. Inside you there’s just another highly complex symbol-shuffling device. So you understand nothing. You merely provide a perfect computer simulation of someone that understands.

Emit is of course vastly more sophisticated than any current computer. Nevertheless, Geena believes Emit works on the same basic principle. If Geena is right then, on Searle’s view, Emit understands nothing.

Emit now asks why, if he doesn’t understand, what more is required for understanding?

EMIT: So what’s the difference between you and me that explains why you understand and I don’t?

GEENA: Yes. You are made out of the wrong kind of material. In fact, Searle doesn’t claim machines can’t think. After all, we humans are machines, in a way. We humans are biological machines that have evolved naturally. Now such a biological machine might perhaps one day be grown and put together artificially, much as we now build a car. In which case we would have succeeded in building a machine that understands. But you, Emit, are not such a biological machine. You’re merely an electronic computer housed in a plastic and alloy body.

Searle’s thought experiment does seem to show that no programmed computer could ever understand. But must a metal, silicon and plastic machine like Emit contain that sort of computer? No, as Emit now explains.

EMIT: Yes. That user’s manual is out of date. There’s no symbol-shuffling computer in here. Actually, I am one of the new generation of Brain-O-Matic machines.

EMIT: Yes. Inside my head is an artificial, metal and silicon brain. You are aware, I take it, that inside your head there is a brain composed of billions of neurones woven together to form a complex web?

EMIT: Inside my head there is exactly the same sort of web. Only my neurones aren’t made out of organic matter like yours. They’re metal and silicon. Each one of my artificial neurones is designed to function just as an ordinary neurone would. And these artificial neurones are woven together in just the same way as they are in a normal human brain.

EMIT: Now your organic brain is connected to the rest of your body by a system of nerves.

GEENA: That’s true. There’s electrical input going into my brain from my sense organs: my tongue, nose, eyes, ears and skin. My brain responds with patterns of electrical output that then moves my muscles around, causing me to walk and talk.

EMIT: Well, my brain is connected up to my artifical body in exactly the same manner. And, because it shares the same architecture as a normal human brain - my neurones are spliced together in the same way - so it responds in the same way.

GEENA: I see. I had no idea that such Brain-O-Matic machines had been developed.

EMIT: Now that you know how I function internally, doesn’t that change your mind about whether or not I understand? Don’t you now accept I do have feelings?

GEENA: No. The fact remains that you are still made out of the wrong stuff. You need a brain made out of organic material like mine in order genuinely to understand and have feelings.

EMIT: I don’t see why the kind of stuff out of which my brain is made is relevant. After all, there’s no symbol-shuffling going on inside me, is there?

GEENA: Hmm. I guess not. You are not a ‘computer’ in that sense. You don’t have a programme. So I suppose Searle’s thought experiment doesn’t apply. But it still seems to me that you are just a machine.

EMIT: But remember, you’re a machine too. You’re a meat machine, rather than a metal and silicon machine.

EMIT: But what’s your argument for saying that? In fact, I know that you’re wrong. I am inwardly aware that I really do understand. I know I really do have feelings. I’m not just mimicking all this stuff. But of course it is difficult for me to prove that to you.

EMIT: Right. But then neither can you prove to me that you understand, that you have thoughts and feelings and so on.

EMIT: Imagine we were gradually to replace the organic neurones in your brain with artificial metal and silicon ones like mine. After a year or so, you would have a Brain-O-Matic brain just like mine. What do you suppose would happen to you?

GEENA: Well, as more and more of the artificial neurones were introduced, I would slowly cease to understand. My feelings and thoughts would drain away and I would eventually become inwardly dead, just like you. For my artificial neurones would be made out of the wrong sort of stuff. A Brain-O-Matic brain merely mimics understanding.

GEENA: No, I suppose not. I would still behave in the same way, because the artificial neurones would perform the same job as my originals.

EMIT: Right. But then not even you would notice any loss of understanding or feeling as your neurones were replaced, would you?

EMIT: If you noticed a loss of understanding and feeling, then you would mention it, presumably, wouldn’t you? You would say something like: ‘Oh my God, something strange is happening, over the last few months my mind seems to have started fading away!’

EMIT: Yet you wouldn’t say anything like that, would you, because your outward behaviour, as you have just admitted, would remain just the same as usual.

EMIT: But then it follows that, even as your understanding and feeling dwindled toward nothing, you still wouldn't be aware of any loss.

EMIT: But then you’re not inwardly aware of anything that you would be conscious of losing were your neurones slowly to be replaced by metal and silicon ones.

EMIT: Then I rest my case: you think you’re inwardly aware of ‘something’ - understanding, feeling, whatever you will - that you suppose you have and I, being a ‘mere machine’, lack. But it turns out you’re actually aware of no such thing. This magical ‘something’ is an illusion.

GEENA: But I just know that there’s more to my understanding - and to these thoughts, sensations and emotions that I’m having - than could ever be produced simply by gluing some bits of plastic, metal and silicon together.

Geena is right that most of us think we’re inwardly aware of a magical and mysterious inner ‘something’ that we ‘just know’ no mere lump of plastic, metal and silicon could ever have. Mind you, it’s no less difficult to see how a lump of organic matter, such as a brain, could have it either. Just how do you build consciousness and understanding out of strands of meat? So perhaps what Geena is really ultimately committed to is the view that understanding, feeling and so on are not really physical at all.

But in any case, as Emit has just pointed out, the mysterious ‘something’ Geena thinks she is inwardly aware of and that she thinks no metal and plastic machine could have does begin to seem rather illusory once one starts to consider cases like the one Emit describes. For it turns out this inner ‘something’ is not something Geena could know about. Worse still, it could have no effect on her outward behaviour (for remember that Brain-O-Matic Geena would act in the very same way). As Geena’s thoughts and feelings, understanding and emotions both do affect her behaviour and are known to her, it seems Geena must be mistaken. Indeed, it seems it must be possible, at least in principle, for non-organic machines to have such mental states too.

GEENA: Look, I am happy to carry on the pretence that you understand me, as that is how you’re designed to function. But the fact remains you’re just a pile of plastic and circuitry. Real human beings are deserving of care and consideration. I empathize with them. I can’t empathize with a glorified household appliance.

GEENA: Right. Remind me to congratulate your manufacturers on the sophistication of your emotion simulator. Now hoover the carpet.

He stands still for a moment, and then slumps forward. A thin column of smoke drifts slowly up from the base of his neck.

The Chinese Room Argument appears in John Searle’s paper ‘Minds, Brains and ‘Programs’, which features as chapter 37 of:

which also contains many other fascinating papers and stories connected with consciousness. Highly recommended.

This is a chapter of The Philosophy Gym by Stephen Law, published by Headline, 3^rd March 2003.

I believe there is (was?) an experiement called the "Turing Test" in which the test of intelligence of a computer is judged on whether or not a human interacting with it can discern whether it's a human or machine by conversing with it. So far nothing has passed the test. I think that eventually we'll come close, but ultimatly fail because I don't believe that it's possible to create something more intelligent than ourselves. The creation can never surpass the creator.

So far nothing has passed the test.

The article assumes that the machine passes the test:
GEENA: No, I suppose not. I would still behave in the same way, because the artificial neurones would perform the same job as my originals.

So the question is: if a machine simulates humans perfectly can we say it thinks?

I don't believe that it's possible to create something more intelligent than ourselves.

I think using the word 'intelligence' when talking about computers is a little misleading. The question is not can a computer be more intelligent than its maker but can it simulate intelligence greater than it's maker in some kind of objectively discernible way? In a lot of ways they already are, their beating chess champions, solving math problems much faster than the average human could, performing tasks with a level of precision no human can match etc etc... Their 'brains' have a much faster processing speed than ours can ever have, fill both brains with the same data and they'll still be answering questions before we can. But potentially their 'brains' could hold many times as much data as our's can.

Can machines think? I don't know, in some ways thats a religious question. I do know that in the not very far future it will be impossible to prove that they don't.

I was hoping he'd hit on her. Wonder what the author would've done with that situation...
Wouldn't they have to perfect a quantum computer before they could make a truly AI machine? (A qc net would be able to operate as closely to the way we do as is possible.) *Shakes head* Naaaaah.

GEENA: No you don't. You have no beliefs, no desires, and no feelings. In fact you have no mind at all. You no more understand the words coming out of your mouth than a tape recorder understands the words coming out of its loudspeaker.

I'm confused here. Is she talking to a robot or a United States senator?

This is nearly a word-for-word copy (except for idiosyncratic spelling, i.e., 'neurone') of the arguements presented in Hofstadter and Dennet's book(s).

Only they make a much better job of it.

--Boris

Disclaimer: Opinions posted on Free Republic are those of the individual posters and do not necessarily represent the opinion of Free Republic or its management. All materials posted herein are protected by copyright law and the exemption for fair use of copyrighted works.