On Why Machines Can Think
How can we think about thinking in the simplest way possible?
In the 17th century, René Descartes introduced a relatively new idea — the dictum “cogito ergo sum” (“I think, therefore I am”). This simple formulation served as the basis of Western philosophy and defined for centuries our ideas on what constitutes the essence of being a human.
Since then, our understanding of what it means to be a human evolved. Yet, for all intents and purposes, many still consider one’s capability to think as one of the most important hallmarks of humanity.
So, it comes as no surprise that the moment ChatGPT (and similar models) was released, we started being bombarded with articles discussing “whether it can think”.
For example, the New Yorker mused “What kind of mind does ChatGPT have?”; the Washington Post proclaimed “ChatGPT can ace logic tests. But don’t ask it to be creative.”; and the Atlantic concluded that “ChatGPT is dumber than you think”. A personal favourite of mine is this video of a comedian trying to explain what ChatGPT is to someone who’s working in HR.
As with any other complex topic that lends itself well to speculation, people are both over-exaggerating and under-representing the thinking capabilities of AI models. So, let’s unpack this.
To think is to reason
Thinking is a complex construct that has come to represent many different things. So, for simplicity sake, let’s presume that thinking is more or less synonymous with reasoning.
Reasoning is a much better defined concept that is, coincidentally, being increasingly thrown around as the future of AI. It’s also what Descartes (largely) meant when he was talking about thinking.
So instead of asking “Can AI think?”, let’s ask “Can AI reason?”.
The short answer is yes. The long answer — it can reason, but only in some ways.
Reasoning is not a monolithic concept. There are multiple ways, in which one reasons, depending on the type of tasks she’s trying to accomplish. So, in this post, we’ll first go through a brief primer on the three key reasoning types and examine how machines measure up. Then, we’ll explore why machines cannot perform common-sense reasoning and what question we need to answer before they can.
Primer on reasoning
Generally, there are three main types of reasoning we employ when “thinking”: deduction, induction, and abduction.
Deduction
Simply put, deduction is the ability to reach a conclusion from a given rule and a case that are assumed to be true.
Picture this: you fill a pan with water, turn on the stove, and pop in a thermometer. Because of things you’ve learned in school, you know that water (usually) boils at 100 °C. So, when someone tells you that the temperature has reached 100 °C, you can safely deduce that the water is boiling (you don’t actually have to see it with your own eyes to be “pretty sure” that it happens).
Here’s a useful structure to keep in mind.
1. Rule: water boils when it reaches 100 °C
2. Case: the temperature of the water is 100 °C
3. Result: the water in the pan is boiling
Thus, you reason from rule and case to a result.
Deduction is fundamental for our ability to do science. It’s also the type of reasoning that’s the easiest to reproduce by a machine.
By design, almost every machine carries out some form of deduction. Your simple non-glamorous calculator deduces answers every time you ask it how much 3+5 is. And it has zero AI in it.
If we put it in the same structure as the water example above, we get:
Rule: The calculator has been “provided” with the rule that 1+1 = 2
Case: You’ve asked the question 3+5 = ?
Result: Based on the rule, it can calculate/deduce that 3+5 = 8
Simple.
Induction
Induction is the ability to generalise rules from a given set of observations. It’s central for our ability to do science since it allows us to quantitatively identify new patterns/rules.
Let’s stick to the water-boiling example. Imagine you have never been told that water boils at 100 °C. So, every time you bring a pan of water to a boil, you put a thermometer in and measure the temperature — 100, 1.000, 10.000 times. Then, your friends do the same — and no matter how many times you do it, the temperature is always 100 °C. So, you can induce the rule: “water boils at 100 °C”.
1. Result: water is boiling
2. Case: whenever you put the thermometer in, it always shows 100 °C.
3. Rule: water boils at 100 °C.
And voila, you’ve identified quantitatively a new rule based on the pattern you observed. To do that, you reason from result and case to a rule.
This type of reasoning is not always correct, of course. Famously, Europeans thought all swans were white until they sailed to Australia. Also, we know that water doesn’t always boil at 100 °C (the atmospheric pressure plays a role, too).
Just because something happens to be correct 10.000 times, it doesn’t mean it will always be correct. Still, 10.000 times tends to be a safe bet.
Induction is much more challenging for machines. Your calculator, of course, cannot perform it. Machine learning models, however, can. In fact, that’s their primary objective: generalise from a set of given results.
Let’s take a simple example. Say, we have a supervised classification model that we’ll use for spam detection. First, we have the labelled training dataset — spam or not spam (a.k.a. the result). Within that dataset, we’ve compiled multiple cases for each result. Based on these, the model induces its own rules that can, later on, be applied to a case it has never seen before.
1. Result: spam or not spam
2. Case: large samples for both spam and not spam examples
3. Rule: emails with “these patterns and words” are likely to be spam (within a certain degree of probability)
Likewise, when dealing with unsupervised models such as recommendation systems, the process follows a similar beat. We first provide the model with a dataset about what people tend to buy when they go to the supermarket (result). Once we start with the model training, we’ll expect it to first cluster repeating patterns (cases) and then, induce its own rules that can be later on applied to similar contexts.
1. Result: the unlabelled data about people’s purchases
2. Case: the similar purchases the model found in the dataset (e.g., everyone who bought eggs also bought bacon).
3. Rule: people who buy eggs buy bacon, too (within a certain degree of probability)
In both cases, these rules aren’t necessarily intelligible by humans. As in, we know that a computer vision model “pays attention” to a certain part of an image, but we rarely know why. In fact, the more complex the model is, the lower our chances are of knowing what rules it uses.
So, here we go — machines can perform both induction and deduction.
Deduction and induction — the bedrock of science
It is a widely-held belief that the combination of induction and deduction is the driving force behind our ability to reason. And, as our examples show, contemporary ML models, even the simple ones, can already perform both.
They first utilise inductive reasoning to generate rules from a given dataset. Then, they apply these rules to new cases. For example, once we present a model with a previously unseen photo, it leverages its rules to deduce specific results (e.g., it can tell us that the photo we provided is upside down).
Still, the majority of data scientists will agree that even the most advanced ML models cannot reason. Why?
The water-boiling example can serve as a simple illustration on why relying solely on deduction and induction doesn’t quite cut it. True, we need them to generate a rule (“water boils at 100 °C”) and then falsify it in a diverse set of cases. However, this combination falls short in explaining how we guessed that the result of boiling has something to do with temperature.
Beyond that, additional limitations of induction and deduction also become apparent — they are somewhat constrained by a specific context and lack the capacity to fully encapsulate the human ability to transfer knowledge across domains. This is precisely where abduction comes in, offering a more comprehensive perspective on the cognitive processes that enable us to make intuitive leaps and connect insights across different realms.
Abduction
Abduction is the ability to generate new hypotheses from a single surprising observation (i.e., result). We do that every time we rely on our experiences to come to an explanation of sorts.
We go out and we see a wet street. We explain it away with the guess that it might’ve rained the night before. We don’t need to have seen 10.000 wet streets to know that when it rains, the street gets wet. Technically, we don’t even need to have encountered a wet street before — it’s enough for us to know that when water touches objects, it makes them wet.
This means that if we’re to go back to our water-boiling example, we’ll have a different way to reason:
1. Result: the water is boiling
2. Rule: water boils at 100 °C
3. Case: the temperature of the water must be 100 °C
We start from the result (as we do with induction), but we combine it with a rule we already know (based on our world knowledge and experience). The combination of the two allows us to come up with a case (i.e., the water is boiling because of changes in its temperature).
Abduction is the least reliable of the reasoning types. Chances are that the hypothesis you reached through abduction is not correct. For instance, the result of “wet street” might have had nothing to do with the rain — perhaps a pipe had bursted somewhere on the street during the night, or someone diligently sprayed the street with water. The rain, however, seems like a plausible explanation.
As such, abductive reasoning allows us to move through everyday situations without being stuck. As in, we don’t need 10.000 tries to make a simple decision.
To my knowledge, no AI model/algorithm to date has been able to perform abductive reasoning. Not in the ways I just described.
Those of you familiar with rule-based systems from the 1960s and 1970s, of course, can point at MYCIN, XCON and SHRDLU and claim that they’re capable of abduction. Others might bring up the examples of abduction cited by the Stanford AI index in 2022 and 2023 as one of the most promising areas for future research (i.e., abductive natural language inference).
So, if machines were able to do “abduction” in the 1970s, why are they still not able to do what I claimed abduction can do (i.e., common sense reasoning)?
Why abduction continues to be elusive
There are two high-level reasons why even state-of-the-art models can’t perform abduction: conflation and architecture.
Conflation: abduction is not the same as Inference to the best explanation (IBE)
Historically, in computer science, many have used the terms IBE and abduction interchangeably. Even ChatGPT will tell you that the two are the same, or that abduction is a sub-set of IBE (depending on how you ask it). The Stanford Encyclopedia of Philosophy echoes this sentiment, too. In fact, almost every paper in the larger field of computer science you’ll read about abduction, will tell you that it’s the same as IBE.
Yet, these are two very different constructs.
Generally, abduction covers the act of generating a novel case (where learnings can be transferred from one context to another). IBE, on the other hand, is a very special and more context-specific form of induction that doesn’t necessarily require you to identify patterns quantitatively (i.e., you don’t need to observe a pattern 10.000 times to formulate a rule). The exact ways in which these are different is a rather complicated philosophical discussion. If you want a deep-dive into that, I recommend this paper.
For the purposes of this post, however, what will help us is to think about them within the rule, case and result structure and use specific examples like MYCIN and the abductive natural language inference model the Stanford AI Index cites.
MYCIN was an early expert system developed in the 1970s at Stanford to assist doctors in diagnosing infectious diseases. It relied on a knowledge base where each rule was expressed in terms of condition (IF — i.e., the case) and a conclusion (THEN — i.e., the result). It then utilised a backward chaining inference mechanism, which allowed it to take a set of symptoms and patient data (result and case, respectively), and work backwards to identify and assign a heuristic certainty score from 0 to 1 to the rules that might explain the situation best. Namely, it reasoned from result and case to a rule (i.e., the pattern that inductive reasoning follows).
The work the Stanford AI index cites as an example of abductive natural language inference (either when generating a hypothesis or selecting the most plausible one) is a bit trickier. Still, it is not abduction. In fact, I’d argue, it resembles IBE, but it follows the same pattern as the other ML models we discussed thus far — induction, followed by deduction.
Some background; in 2020, Bhagavatula and colleagues*, trained a transformer model conditioned on a dataset they call ART (containing ∼20K narrative contexts defined by pairs of observations (O1, O2) and 200K explanatory hypotheses). After training, they provided the model with a set of two observations and asked it to generate a plausible hypothesis to match (see Figure 4).
As you can see from the figure, when a transformer model (GPT-2+COMeT embeddings) is presented with O1 (e.g., “Junior is the name of a 20+ year old turtle”), and O2 (e.g., “Junior is still going strong”), it can generate a plausible hypothesis (e.g., “Junior has been swimming in the pool with her friends”) that might explain why we think Junior is still going strong.
Why is this IBE and not abduction?
Let’s abstract ourselves from the underlying ML model for a bit and think about how a human might perform such reasoning task. First, we’re provided with a result: Junior is still going strong, and we’re told what the case is (i.e., Junior is a relatively old turtle). Then, from these, what we’d do is to try and find a potential (context-dependent) rule that can explain the case and the result. For example, we can induce that an old turtle that’s still going strong
- tends to play with its friends OR
- has a healthy appetite OR
- has good vitals
and so on.
We can then choose the most plausible (to us) rule and apply it to our case of “an old turtle”. This will allow us to hypothesise that Junior could have been swimming with her friends.
As already explained, the identifying of the potential rules from a limited set of observations is indicative of IBE and the act of drawing conclusions from these, tends to be a weak form of deduction.
We as humans understand that when one ages (be it a turtle or a human), their vitality tends to go down (arguably). This allows us to generate rules that are relatively ‘imbued with meaning”. A transformer model cannot do that. What it can do, however, is improve its predictions on the most probable combination of words that can follow the provided case and result (by applying induction and then deduction). The model has no underlying understanding that when Junior is having fun, she’s still going strong.
In fact, one might even go as far as to say that the work on abductive natural language inference is reminiscent of chain-of-thought prompting. Granted, the instructions are presented to the transformer in a different manner.
What all these instances highlight, hopefully, is that what computer science labels as abduction isn’t abduction after all. Instead, it appears to be a context-specific variant of induction.
Architecture: contemporary ML models are bound by induction
The second reason behind state-of-art models’ inability to carry out abduction lies in their architecture. By definition, ML models are an induction-generating machines. This inclination is further strengthen by their so-called inductive bias.
Inductive bias is an integral concept in ML referring to the inherent assumptions or preferences a model possesses regarding the types of functions it should learn. The bias helps guide the learning process by restricting the set of possible hypotheses, making learning more efficient and accurate.
For example, decision trees focus on hierarchical structures and simple decision boundaries. Support Vector Machines aim to find wide margins between classes. Convolutional Neural Networks emphasise translation invariance and hierarchical feature learning in images. Recurrent Neural Networks are biased towards sequential patterns, Bayesian Networks model probabilistic relationships, regularised linear models prefer simpler models by penalising large coefficients, and general transformers like GPT-4 are characterised by their ability to capture sequential dependencies and relationships in data. These biases shape the models’ behaviour and suitability for different tasks. They also make it difficult to transfer learnings from one context to another.
What we still need
OK, by now we discussed a primer on reasoning and we saw that machines can indeed reason. They perform both deduction and induction. However, what we tend to intuitively term as “thinking” is facilitated by abduction, which continues to be elusive due to conflation and architecture.
So, what do we need then?
How do we go about building something that can perform abductive reasoning?
Well, first of all, we need to be able to properly define what abduction is and describe how it works. Sadly, not much work has been done in this regard. Especially, when it comes to identifying how abduction relates to induction and deduction. Or how it can be operationalised by machines. The only thing scholars tend to agree on is that abduction comes first, followed by induction and deduction.
So, what is abduction?
Abduction is not a monolithic construct. I’ve personally came across around 10 different types, depending on the scientific field to which they pertain. Even the philosopher who introduced the notion of abduction, Charles Peirce, doesn’t refer to it in a consistent manner.
However, there are three main types that can describe the fundamental functions abduction serves. The exact functions and how they came to be are too complex to cover in this post. So, here are the cliff notes.
First, we have the most straightforward abduction type — explanatory. The one we discussed thus far. To employ it, we start with an observation (result) and a rule that is easy to identify. The combination of the two then enables us to make a conjecture about the case. This is well-illustrated in the water-boiling example.
Then, we have innovative abduction — a type of abduction which allows us to reason from a (desired) result to a pair of a case and a rule. Namely, we only know what result we want to create and then we need to gradually define a case-rule pairing that will allow us to achieve said result. This type of abduction is usually used to generate novel ideas.
Finally, we have, I think, one of the most interesting types of abduction — manipulative. We use it in situations where the only thing we know is parts of the result (desired or otherwise). Furthermore, the context in which this result “lives” is defined by multiple hidden interdependencies. So, it’s not possible to start looking for/generating a suitable case-rule pair right away. Instead, we need to better understand the result and how it relates to its environment, so that we can reduce the level of uncertainty.
That’s where the so-called thinking device/epistemic mediator comes in. This could take the form of e.g., a basic sketch, prototype, or 3D model, serving as a means to enhance our understanding of the problem. By manipulating this mediator within the target environment, we gain a deeper understanding of the context. Consequently, we become better equipped to explore potential combinations of rules and cases. Additionally, it allows us to establish associations that aid the transferring of knowledge from one domain to another. A simplified version of this kind of thinking is commonly applied in stereometry, for instance.
As I said, much work still needs to be done in explaining the relationships among these abduction types and their relatedness with other reasoning approaches. This endeavour is becoming increasingly critical, however, as it holds the potential to offer valuable insights into the transferability of insights across different domains. Especially, in light of the renewed interest in reasoning we see in the field — be it via IBE, “reasoning through simulation and examples”, or System-1 and System-2 thinking.
Amidst all that, it seems pertinent to understand how not to conflate the different types of reasoning that can be performed by a machine. Because, yes, machines can reason. They simply cannot perform the full reasoning spectrum.
*Other interesting work on IBE can be found in this paper (they do equate abduction with IBE, however).
