Intelligence in Computing Machinery: Quest, approach and the future

A case for the necessity of holistic development of the science and engineering of intelligence

Photo by Alexandre Debiève on Unsplash

For some reason, we incline to make our lives better. Science and engineering endeavor to understand things and to build them, respectively, helping each other to take the next iteration. After many iterations now, we have become wiser and a bit more in control of what is going on with our lives and within ourselves, ultimately making us more civilized. The next iteration is tantalizingly the science and engineering of intelligence.

This problem of intelligence, both the science and engineering of intelligence, is considered the greatest problem of all sciences and might be the last iteration of science and engineering itself. Machine intelligence is necessary or imminent if the problem of intelligence is ever to be solved.

Intelligent machines, if made possible, could be an answer for a lot of questions. Complete automation of manufacturing and even design, personal assistance and care, enhanced human intelligence, and scientific discovery itself. This is way down the road. We even don’t know whether it is possible. But, that is what stirs the curious human mind.

Here, we see the quest for intelligence in computing machinery and several isolated approaches to building intelligent machines. We infer the rigorous interdisciplinary nature of the problem of intelligence and evident misalignment of interest of the investigators in the field. And all along, we plead for the holistic development of the science and engineering of intelligence that could help us with so many hurdles to give a clear view of the path towards intelligent machines.

1. The quest

The idea of engineering intelligence into a machine would trace back to Ada Lovelace, a mathematician and known as the first computer programmer, saying that ‘machines can do everything except think’. Partly arguing against this objection and others, the most significant earlier work on machine intelligence would be the seminal paper by Alan Turing called ‘Computing Machinery and Intelligence’(wiki, pdf), where he also talks how we should formulate the question of ‘does machines think?’. We could deduce that he reduced the problem of ‘thinking’ to an imitation of human conversation which could involve reasoning, problem solving, common sense and understanding of the world. He predicted that we would have a considerable thinking machine in about 50 years but, here we are about 70 years later still without a machine that we would all agree intelligent.

Optimism of an exciting venture

Then AI field emerged popularly from the summer research project at Dartmouth with participants John McCarthy, Marvin Minsky, Claude Shannon and so on. The entire field was very optimistic, with their projects being integrating logic, commonsense knowledge, general problem solving abilities into a machine which we now realize as far-fetched goals.

Marvin Minsky in his paper, ‘Steps toward Artificial Intelligence’( publication, pdf), with assertion and hope for AI which we no longer dare to possess, summarizes with the following- (This paper is a great and simple example of a science of computational intelligence).

The problems of heuristic programming — of making computers solve really difficult problems — are divided into five main areas: Search, Pattern-Recognition, Learning, Planning, and Induction. Wherever appropriate, the discussion is supported by extensive citation of the literature and by descriptions of a few of the most successful heuristic (problem-solving) programs constructed to date.

Investigators came up with different models mostly by symbolic interpretation of things and their transformation, that only worked up to the point of exploring the magnitude of the problem they are after, but none succeeded. And then, commercial rule-based expert systems came along that brought no leap in the scientific discovery. Even then there were too many approaches might be justified by the hardness and the holistic nature of the problem. The unrecoverable damage occurred not until after the discouragement by the magnitude of the problem.

That the connectionist (as opposed to the symbolic approach, with a distributed representation of objects and ability to learn from data statistically- for example, neural networks) approach began working a bit. Then, most of the investigators stopped doing science of intelligent machines and concentrated on their petty projects of narrow intelligence. Speech recognition, object detection and all types of pattern recognition tasks are immensely benefited. Not that it didn’t foster any good (a great many applications that served us well), but the science of intelligent machines moved no step further.

A problem that is underestimated

The problem of intelligence has always been underestimated, maybe because of its novelty, hardness, and holistic nature. Many investigators like Turing, Marvin Minsky, and presumably everyone worked in Artificial intelligence have predicted that intelligent machines would arrive earlier than now but, in fact, we are not so far-fetched from where they were towards building intelligent machines.

2. The approach

The problem attracted many approaches for making intelligent machines. At the dawn of the field, symbolic and connectionist approaches. Then, whether more innateness is needed in AI models to be intelligent, up to the point seeking hybrid approaches according to the narrow problem being solved, without any reference to the science of intelligent machines.

Investigators working in both the symbolic and the connectionist approaches foresaw the success in building intelligent machines, being skeptic about the other approach. The investigators in the former, because of the complexity of the problems like common sense and general problem solving and in the latter, because of the capability of their approach for better pattern recognition. See this theory on intelligent machines, ‘symbolic vs connectionist’, that talks about designing intelligence into machines and renders the debate to be transient and not the principal question to attend to.

“…Smart systems need both of those components, so the symbolic-connectionist antagonism is not a valid technical issue, but only a transient concern in contemporary scientific politics.”

Now the competing paradigms are of very different types like whether to use more innate machinery in the models or otherwise. Innate machinery is the self-contained (A priori) structure in a model that couldn’t be learned from data. In most cases, it enables and drives the learning. To give a familiar example of a type of innateness, the convolution or weight sharing in a Convolutional Neural Network could be seen as innate machinery to drive specified learning.

Now, there is a lot of discussion around innateness in intelligent machines from philosophy to linguistics, this debate (video) between Gary Marcus and Yan LeCun titled, ‘Does AI need more innate machinery’ would be fairly comprehensive and contemporary.

And whether to seek neuroscience for the ideas or build intelligent machines from scratch. Reinforcement learning is the ultimate answer. Neural networks are. Now, sub-domains like computer vision, natural language processing, robotics, generative modeling etc., are providing the escape from the hard problem of the science of intelligent machines, which is the only thing that helps every sub-domain leap forward in a greater time scale.

In the face of the greatest problem of all sciences, we are again underestimating the problem, but now, because of lack of the concern for the holistic development of the science of intelligence, taking the escape into different approaches or sub-domains inspired by the transient success of deep learning. A concrete science of intelligence would successfully discern what is the right approach in the time of ambiguity and also what is the right blend of approaches to move forward with every sub-domain. Without the science of intelligence, we can’t successfully engineer intelligence into a machine.

3. The future

A theory of the science and engineering of intelligence is necessary to lead the investigators in the related fields (which might concern with human intelligence or artificial intelligence like computer science, neuroscience, cognitive science, philosophy, psychology…) in the right path, or at least to provide the placeholders for different investigations and to enable building upon them in the next iteration.

David Marr, a prominent investigator of vision, expressed his view that investigations exploring different types of innate structures for the representations of objects would be productive for computer vision. See this for a review of categorized investigations from the perspective of visual intelligence- ‘Artificial Intelligence- A personal view’

“… Marr and Nishihara’s 3D representation theory asserts that the deep underlying structure is essentially that of a stick figure and that this representation is manipulated during the analysis of the image…”

So that the different cognitive tasks like reasoning, common sense and problem solving would be solved in the next iteration. This comes from the approach to the philosophy of mind that there is an A priori structure to enable perception that can’t be learned. This a better way to go about investigations than from solving one dataset to another. A holistic theory of intelligence would encourage and be realized by this kind of investigations.

Investigators in the field of deep learning say ( in the argument about required innateness for the model) that they are integrating all the aspects of A priori structure (innateness) into the architectures of the models so that it could learn everything from data. It would become a credible investigation in a scientific iteration, but only by discussing the importance and stance of their investigations with more attention on these aspects with reference to the science of intelligence rather than only defending them in debates. (debate on innateness mentioned above)

We couldn’t afford to only guess that our investigations would be the first step in solving the problem without even considering to fully understand the problem we are solving. We have to present our investigations as primitives for the coming generations, that are in turn investigated upon in the next iteration. We have to deal with the problem of intelligence with more scientific rigor beyond the scope of applied science.

Current investigative power of the field

Principal investigations being presented at prominent machine learning conferences like NIPS, ICML, ICLR etc., are new architectures and algorithms concerning neural networks, theoretical understanding of neural network models, novel approaches to improve optimization, regularization etc., and a few implementations inspired by neuroscience and cognitive science.

Not out of the concern for intellectual competence, but the current investigations of the field don’t actually contribute much to the science and engineering of intelligent machines. Most of the work is justified to be a contribution by saying ‘it is the first and necessary step’ towards machine intelligence without concerning about its role in the complete theory of intelligent machines. If in any perspective, these investigations seem to have a role in the science of intelligent machines then, we could easily see that it is only in few aspects but not holistic, making even those small contributions not constructive. The versatility and the value of applied machine learning might have an effect on the field along with some political and cultural influence.

We could see to an extent what contributes to the science of intelligent machines. For a simple measure, let’s see into

• How the five ingredients of intelligent machines proposed by Marvin Minsky (Search, Pattern-Recognition, Learning, Planning, and Induction from the above-mentioned paper ‘Steps toward Artificial intelligence’), their relationships and their combined integration are exploited in the investigations of the field.
• How the A priori structures (innate structures) for perception and understanding proposed by many philosophers of mind like Plato, Aristotle, and Immanuel Kant are justified/integrated into the neural network models that are designed for machine perception and understanding.
• How frequently and seriously does the machine learning community take input (theories to be implemented, insights for designing models, validation of models etc.,) from fields that have been investigating the human intelligence and the brain, for decades and even centuries like philosophy of mind, mathematical logic, neurophysiology, cognitive science, psychology, neuroscience and so on.

Apparently, these approaches are not mainstream. Maybe because more rewarding paths (through applied science) for intelligence which do not necessarily contribute to science exist. The problem with those paths is that they seldom succeed in our goal of making machines intelligent.

Influences on the pioneers of the field

Most of the investigators in the field would agree that AI as a field is in its infancy. In all the sciences, the investigators in an infant scientific field would dwell on the roots of the problem they are solving. Here it’s the problem of intelligence and the roots are the philosophy of mind, cognitive and behavioral science, psychology, linguistics, and so on.

The pioneers of the field are mindful of this. Marvin Minsky, one of the pioneers of AI, is greatly influenced by the psychologist Siegmund Freud, and by many philosophers that dated him. He spent most of his career theorizing about intelligent machines. His most significant work could be the ‘Society of Mind’, which is a computational model of a brain. It is built upon computational theory, mathematical logic, neuroanatomy, philosophy, cognitive and behavioral psychology to explain different aspects of intelligence.

John McCarthy extensively worked on integrating logic into machines, working at the intersection of philosophy and artificial intelligence. To get familiar with the ideas he borrowed from philosophy see this insightful paper, ‘philosophy of AI and AI of philosophy’.

Even some prominent investigators of the field appreciate the interdisciplinary nature of AI. Demis Hassabis, co-founder of Deepmind, makes a case for Neuroscience-inspired Artificial Intelligence, upholding the instances when inspirations from neuroscience helped AI, and describing the potential and necessity of this approach to engineer intelligence in to a machine.

To a young investigator of the field

Let’s assume in the near future some new-to-field investigator would like a machine to understand things in the world and reason about abstract ideas just by reading text input. Then, he wouldn’t probably succeed in building that model with the tools of AI/deep learning alone. He needs ideas from the investigations of linguistics, logic, common sense reasoning, epistemology, and some insight into how does brain actually understand from neuroscience, cognitive science etc..,

If one intends to work on the science and engineering of intelligence, he/she has to look beyond the work of investigators in this infant field to the influences of the pioneers itself. If we are not comprehensive of the problem we are solving, then we end up ‘inventing the wheel again’ or likely, solving not a problem at all.

The issues related to the aspects of intelligence, the science and engineering of it, should be the foundation of all the investigations in AI, but at the current scenario, they end up merely as a philosophical silver lining in popular talks summarizing the whole field.

References

[1] Alan M. Turing. Computing Machinery and Intelligence (pdf), Mind 1950.

[2] Marvin Minsky. Steps toward Artificial Intelligence (pdf), Proceeding of the IRE, 1961.

[3] Marvin Minsky. Symbolic vs Connectionist, Artificial Intelligence at MIT, Expanding Frontiers, Patrick H. Winston (Ed.), Vol.1, MIT Press, 1990.

[4] David Marr. Artificial Intelligence — a personal view, AIM 355 March 1976.

[5] Marvin Minsky. Society of Mind, Simon & Schuster, Inc. New York, NY, USA ©1986, ISBN:0–671–60740–5

[6] John McCarthy. The philosophy of AI and the AI of philosophy, Computer Science Department, Stanford University, 2006.

[7] Gary Marcus. Innateness, AlphaZero, and Artificial Intelligence, arXiv:1801.05667, 2018.

[8] Demis Hassabis, Darshan Kumaran, Christopher Summerfield, Mattew Botvinick. Neuroscience-Inspired Artificial Intelligence, Neuron, 2017.

This is the foundational article of the ‘Quest for Intelligence’ reporting. Upcoming article dwells on different A priori structures (innateness machinery) which enable perception and language understanding, that are proposed in philosophy and linguistics, and how they could be realized in the contemporary research of machine perception and language understanding.