Coding consciousness
A very long post about the Turing Test
In his world-famous 1980 “Minds, brains, and programs”, the American philosopher John R. Searle has demonstrated in an elegant and unambiguous way that the execution of an algorithm (no matter how complex it is) isn’t a sufficient condition to have understanding. In his Gedankenexperiment, the Chinese Room Argument (CRA), he imagines to lock himself in a room with a large batch of Chinese writings (that to him are just a bunch of meaningless squiggles) and a book with rules written in English to correlate these symbols to others passed to him. No one, seeing this process from outside the room, would ever think that he is simply performing symbol manipulation and in any way he now understands Chinese more than when he entered the room in the first place. This thought experiment proves, as he rephrases several times in his work and here proposed in a simple and effective form, that “symbol manipulation by itself couldn’t be sufficient for understanding ” (Searle, 1980) or that, even simpler, “syntax is not by itself sufficient for, nor constitutive of, semantics”.
Syntax is not by itself sufficient for, nor constitutive of, semantics.
Obviously a lot of replies have been raised against this experiment and to almost all of them a precise response is given by Searle; however, the answer given to one of the most relevant, the so-called Systems Reply, is unsatisfactory and lacks of coherence, keeping the door open to some interesting consequences and further arguments.
The Systems Reply claims that even accepting that the mere execution of an algorithm doesn’t cause understanding, the executor (in the CRA the man locked inside the room, or, more generally, a Turing Machine (TM) going through the steps of a computer program) is just a part of the whole system and that this system understands Chinese (continuing to use the example made by Searle). The response of Searle to this reply follows two parallel directions: in the first one he hypothesises to “internalise” the whole system inside the individual (getting to a degenerate configuration in which the man is indeed the whole system) and claiming that even in this condition the system still doesn’t have understanding; in the second, merely touched to be honest but very relevant under a conceptual point of view, he argues how ridiculous would be to associate understanding to a man in conjunction with a piece of paper once correctly highlighted that the man alone wouldn’t have any. Being the second one more an analysis of the hypothetical consequences of ascribing understanding to the whole system, the focus has to be put on the first one, in order to understand where and why it lacks soundness even more so in the light of the consequences that Searle himself correctly connects to his arguments.
Without entering deeply in the classical philosophical “other-minds” problem (premise needed to even start reasoning on these arguments) understanding is, as consciousness, intentionality, etc. a “subjective character for which an objective nature can only be apprehended from many different points of view. The fact that in his argument Searle talks only about understanding and not about the others (consciousness, in the first place) is not a limitation for the subsequents steps; in a sense because as Harnad writes in 1991 “there is only one mind-body problem”, in another because the way in which the concept is discussed and further addressed in later writings is clearly exchangeable use-wise with the concept of consciousness.
In the attempt of a general and coherent description of these phenomena, different theories have been proposed during the decades; isn’t the aim of this post to precisely describe all of them, but with the double objective of making it the most self-contained possible and to avoid concept ambiguities in the next passages, it is useful to partition the universe in two: the first group of theories is composed by the ones that recognise themselves under the broad concept of Computationalism, and for which mental processes are entirely computational and describable by means of pure symbol manipulation. The entailments of these theories are largely debated but it is quite undisputed that accepting computationalism is to accept at least some shades of dualism and that therefore the mind and the body are distinct and divisible (concept on which, more than on others, Strong AI puts roots down). The latter, often referred to as Physicalism is concerned with the idea that human mental phenomena might be dependent on the physical/chemical properties of actual human brains. This second type of theories, putting the birth of intentional phenomena not in the act of computation but in the very fact of being “a certain sort of organism with a certain biological structure”, can survive to the CRA; in particular Biological Naturalism is precisely the theory supported by Searle both in the last paragraphs of his paper and throughout his career. Even though, in a sense, this specific theory places itself halfway between the two above partitions (considering its position about the existence of irreducible mental phenomena, not accepted by the pure physicalist), it certainly assigns to the actual human brain and to its specific nature the capability of producing intentionality. The physicalist theories, in addition to the properties listed by Searle, have the great advantage not to be affected by the abstract limitations powerfully described in the works of Gödel, Lucas and Penrose. Such limitations, originally directed to formal systems that are of sufficient complexity to express the basic arithmetic of the natural numbers and which are consistent, have been subsequently exploited to place some limits to the power of Turing Machines (or equivalent) in their ability to adequately model the human brain capabilities, attacking the very foundation of computationalism; “minds cannot be explained as machines” (Lucas, 1961).
And is in the light of this explanatory paragraph that we can better see why the internalisation process used by Searle to disprove the Systems Reply shows itself in its inadequacy. No details are given in the paper about how this process has to be conducted, and this lack cannot be excused by the regime of thought experiment within which all his conclusions are drawn. Talking about internalisation breaks the assumptions of hypothetical reasoning, and requires at least a theory of mind as a common ground, not differently from what happens with the necessity of a theory of computation for all the other claims. There is no guarantee, in fact, that the process of internalisation leads to a purely syntactic execution any more than the traditional learning process that has led to a mutually agreed actual understanding of the language used for the rules. If it is true, as Searle himself backs, that understanding is not a matter of execution but a consequence of some chemical properties of brains, what can be said about an internalisation process whose only given detail is the fact that exploits these very properties? Keeping these on the side is reasoning in a dual and computational way, using tools that he himself has proven to be wrong — or with the need to be heavily reviewed.
There is no guarantee, in fact, that the process of internalisation leads to a purely syntactic execution any more than the traditional learning process that has led to a mutually agreed actual understanding of the language used for the rules.
Furthermore, the internalisation process (and its analysis) breaks the wall of the other-minds problem, as if existed a “Searle’s little ‘periscope’ across the otherwise impenetrable other-minds barrier” (Hayes et. al, 1992) to testimony with no doubts the absence of understanding in the ability of speaking Chinese with Turing-indistinguishable performances. This argument, as risky as it is, adds another level of uncertainty about the whole response that in conjunction with the previous one, leads, if not to a full acceptance of the Systems Reply at least to a more convinced consideration of the possible consequences that would arise accepting it as true.
Where does understanding lie?
Accepting the Systems Reply is, simply, accepting the fact that the individual is part of a System and that the System understands the story. The consequences of yet so simple claim can be tricky to manage, and grow in two diverging directions: where do we put the boundaries of our System and to which artifacts of the System are we willing to assign understanding.
Accepting the Systems Reply is, simply, accepting the fact that the individual is part of a System and that the System understands the story.
Even though both issues will be investigated deeply later (with the former that comes as a direct consequence of the latter), it is useful to make a premise to talk about what will be now on referred as understanding with a focus on the extent with which we can investigate the concept. Debates about the nature and value of understanding occur across philosophy, but there are no reasons not to accept the definition given by Searle in the Notes of his paper. He defines understanding as the contemporary possession of mental states that are both valid and intentional. The core of the definition is of course around the concept of intentionality, that is “that feature of certain mental states by which they are directed at or about objects and states of affairs in the world”. The underlying problem, common to all the subjective experiences and related to the hard problem of consciousness, lies in the very nature of these phenomena, that makes definitions interesting but in a sense completely useless. Embracing fully the limits of our external point of view, and in the contemporary need of operationalising our search, we can only look for traces of understanding: signs through which, applying the same common sense arguments and assumptions of truthfulness that we apply in everyday social relationships, we can hypothesise the presence of understanding in others in the same way we associate it to us; or, similarly, signs that place ourselves in a condition such that we can hardly make sense of them without the ascription of intentionality.
The second consequence of the acceptance of the Systems Reply pointed out above is therefore slowly reshaping in a new form, that even if in a sense can be seen as a limitation, it grounds the concept in a domain in which it can be actually handled. It isn’t, as for Searle’s concerns that while a person doesn’t understand Chinese, somehow the conjunction of that person and bits of paper might understand Chinese, but the idea that the piece of paper — even though calling it like this is a voluntary underestimation of its content, the real deal here — is a trace of understanding and the proof — not formal, being impossible, but in a sort of maximum likelihood scenario — that somewhere understanding is present. An obvious issue that can’t be left out now is the idea, induced by this line of reasoning, that there is something in computation that has the power of “giving birth” to an interactive daemon able to exhibit understanding in a behavioral way — in clear distinction to artifacts only able to show passive traces of it. This idea isn’t new in literature, with a number of contributions large enough to deserve a parallel branch of theories called Virtual Mind Replies to CRA. Analogously to the Systems Reply, the VMR concedes that the operator doesn’t understand anything about Chinese but underlines that the important fact is if understanding is created, not if the operator is the one that actually understands.
The important fact is if understanding is created, not if the operator is the one that actually understands.
The act of computation creates new virtual entities which psychological traits depend entirely upon the program and the Chinese database. Even though this position requires at least some sort of computationalism and it’s not clear how this can cohabit with Gödel Theorems, it is relevant and indisputable that computation is a special matter because of its intrinsic characteristics. Our set of rules given to the individual locked inside the room is a medium not only for expressing representations but for bringing out the “representational activity” of certain machines. “The inherent procedural consequences of any computer program give it a toehold in semantics” where the meaning of a symbol “is to be sought by reference to its causal links with other phenomena”. This concept of representation in computation related to symbols’ execution (meaning as activity) will be subsequently better investigated with respect to the non-dictionary groundings needed to pass the Turing Test.
Considering these arguments, the first open issue of the Systems Reply, that is, where to set the boundaries of the System, has lost some of its initial interest: if we are now ready to accept anything as trace of understanding — and this doesn’t mean to accept everything, having the Turing Test as a silent watchdog — we can place a boundary ideally wherever we’d like to (outside of the executor, as Searle points out). Whichever System passes the Turing Test contains a piece of understanding, being the set of rules (or, equivalently, a computer program) or the programmer (if any) that has coded it, or the books in which he has studied, or the writers that have written it….
If this last paragraph seems unusual or somehow bold is instead very common in the everyday approaches to understanding and meaning. As Hayes points out: “I read a novel and as a result am cast into deep thinking about the nature of life and then decide to give it all up and become a monk, did the novel cause me to make that decision?” In a sense, the novel passes a very lightweight version of the Turing Test and as a result, I’ve no reason to think that whoever has written it doesn’t understand what he has written (traversing possibly the pyramid of copies of the same text).
Whichever System passes the Turing Test contains a piece of understanding, being the set of rules (or, equivalently, a computer program) or the programmer (if any) that has coded it, or the books in which he has studied, or the writers that have written it….
The idea that understanding isn’t in the book (considered as printed letters, papers, etc) but in the System is now more convincing, and the concept of System’s boundaries looks more familiar.
The adequacy of the Turing Test
A lot of the consequences drawn in Searle’s paper and in this one, are based on the assumption that the Turing Test is a valid test to at least speculate about the presence of understanding. In his original work, Turing has proposed this test in the direction of avoiding precise theoretical definitions of intelligence and in the need of an operative direction of research for the subsequent years. Even though the voluntary and advantageous escape from a definition of intelligence has raised some objections during the years is undoubtful that leveraging a behavioral approach similar to the one used in everyday life aspects makes the TT pretty much unassailable without entering in the same risky other-minds topics pointed out above. The notion of Turing indistinguishability is therefore not only a starting point to which respectfully look in the task of evaluating intelligent performances but, in a circular way, the only actual method we have to infer about understanding.
It is useful here to make a distinction between the two: the research of an intelligent behavior may be very domain dependent and can lead to some critical and pathological issues of the test when considered in its original version; acknowledging this, doesn’t in any way affect our line of research. We are not looking for performances (whatever is the method used to assess them and on which a lot of other discussions can be made) but, in a sense, for the reason why these performances are achieved.
We are not looking for performances but for the reason why these performances are achieved.
These concepts can only be operationalised by their nature; attacking the Turing Test is a distortion of the concepts in the extent of attacking their range of applicability. The Turing Test shows itself not only to be adequate for the task but until a new meaning will be attached to the concepts we are investigating, it is also the only possibility we have to detect traces of them.
“The only available basis for inferring a mind is Turing-indistinguishable performance capacity”. (Harnad, 1992)
Consequences and Conclusion
Some possible criticism to these positions have been already addressed in the post, and are mainly related to the acceptance of the claim that physical inanimate objects can exhibit traces of understanding. As already pointed out in the previous paragraphs, this potentiality has to be always seen under the light of the Turing Test’s framework and of the ability to execute the used formalism in an appropriate hardware-context. In partial agreement with Searle, it is true that this attribution is partially biased by “the fact that in artifacts we extend our own intentionality”, but the ability to extend our intentionality in an extent large enough to pass the Turing Test isn’t trivial and it’s the core of the whole reasoning.
The ability to extend our intentionality in an extent large enough to pass the Turing Test isn’t trivial and it’s the core of the whole reasoning.
Another possible criticism of these positions can be related to the recent discoveries in the field of Machine Learning (ML). Since the 1950s the Artificial Intelligence’s direction of research has been almost completely devoted to expert systems with a lot of domain specific properties used to develop efficient and effective heuristics to solve complex search problems. In recent years, the increased available computing power has allowed to brush up on some old approaches stored in the attic since the 60s and sketched by Turing in an ingenious and foresight passage of his paper (“The child programme and the education process”). Machine Learning is, simplifying the concept, the field involved in the automatic research of solutions for complex problems using data coming from the problems themselves. The crucial difference between the two paradigms isn’t only in the performances, but in the very source of information used to solve the tasks. If in the heuristics’ development process the human ability to understand the specific problem was required to allow a machine to solve it efficiently, ML doesn’t require it at all, being able to extract the relevant information from data without being programmed specifically to solve the problem in exam. If all the claims made in the previous passages about understanding and its presence in the System would not be philosophically affected by a hypothetical expert system able to pass the Turing Test, would be strange to attach some understanding to a Machine Learning system that has been able to accomplish this challenge without any human intervention. The elephant in the room, and not only in this specific niche, but in general in all the mainstream discussion about ML, is in the requirement of huge labeled corpora used to feed these systems. ML approaches to problem solving are invaluable and day-by-day more and more surprising but, in a sense, the human intervention in this processes is important now more than it was for the approaches used in the past. If tomorrow morning a ChatBot designed using state-of-the-art Deep Learning techniques was able to pass the Turing Test would be thank to hundreds of thousands labeled real dialogues between humans, written in a mutually understood language, and learned using ad- hoc algorithms. Understanding would still be in the System.
If tomorrow morning a ChatBot designed using state-of-the-art Deep Learning techniques was able to pass the Turing Test would be thank to hundreds of thousands labeled real dialogues between humans, written in a mutually understood language, and learned using ad- hoc algorithms.
In the light of all the discussions made, a necessary and sufficient condition to pass the Turing Test can be rephrased as the ability to manage a language and talk about the results of tasks performed by means of it in a human-indistinguishable way. The fact that humans themselves are able to accomplish this task by learning, could mean that Machine Learning as a general framework is the right one to design an executable computer program getting to the result of passing the Turing Test in the next years. A possible open question here, but is in a sense wider than the objective of this post, is if there exists a way for an individual to pass the original version of the Imitation Game — the classical one with a man and a woman in a room and the man trying to fool the interrogator about his gender — without the presence of a learning process attaching some groundings to the symbols that the teacher and the student are able to manage. The presence of understanding as the ability to teach (or, getting back to the ML case, in the ability to select proper examples and learning algorithms) can be an interesting line of discussion.
