Hacking Hippocampus: the Next Frontier for Machine Learning and beyond…

“Deep inside the skull of every one of us there is something like a brain of a crocodile. Surrounding the R-complex is the limbic system or mammalian brain, which evolved tens of millions of years ago in ancestors who were mammal but not yet primates. It is a major source of our moods and emotions, of our concern and care for the young. And finally, on the outside, living in uneasy truce with the more primitive brains beneath, is the cerebral cortex; civilization is a product of the cerebral cortex.”

— Carl Sagan, Cosmos p.276–277

The knowledge of neuroscience about the human brain is still so incomplete that we can only get inspired in our work by the ideas of neuroscientists instead of basing it on a rock solid scientific proof. Yet the recent success in deep learning demonstrates that such an approach can work.

The robust performance of supervised learning matches or even outperforms already the ‘brain of a crocodile’ from the above quote of Carl Sagan. The ‘limbic system or mammalian brain’ is the next stop. At the core of it is the hippocampus, a single, curved cell layer of gray matter at the root of the cerebral cortex. The hippocampus is widely believed to mediate numerous cognitive functions because of its dense reciprocal axonal projections to and from cortex.

Mediator of Cognitive Functions

Imagine that your body is a drone that is piloted by two different people. One of them has a cockpit view (or ‘the first person’s view’ as they call it). This pilot can see the vertical and horizontal position of the drone on separate displays but the view ahead is the major channel of information for him. The first pilot can instantly compare the current cockpit view with the films and snapshots taken during previous missions. Even more importantly this pilot can turn on a cruise control and even an autopilot mode each time when it recognises a familiar path.

Another pilot flies the same drone by operating a small model of it on a 3D display. From time to time the second pilot takes a look at the drone from outside from a distance (they call it ‘in the line of sight’) but most of the time he only sees the position of the model of drone within the model of the environment. Both models (of the drone and of the environment) have been created and are instantly adjusted on the basis of comparison of records of previous missions and the new data inputs from all sensors of the drone.

Now imagine that the two pilots can’t talk to each other or exchange the information in any other way. They can only swap the controls of the drone between them by decision of an ideal observer. The ideal observer has no access to displays of either of the pilots. He can only measure with the ideal accuracy the probability of all events which have occurred, actually occur and are predicted to occur on those displays as well as the credibility of the information about those events. The ideal observer works as a switchboard swapping the controls between the two pilots. The first pilot is great in the situations where the high credibility information exists about either probable or improbable events but it gets entirely lost when the credibility of information is low. Then the second pilot comes in place. It navigates by landmarks and explores the environment to obtain more credible information.

Explore and observe the explorer

At this point hippocampus first came into our focus as the brain area responsible for making the exploration decisions — i.e. decisions about switching between passive and active learning sessions. “Hippocampal activity therefore directly corresponded with the eye-movement patterns, thus establishing tight linkage between hippocampal activity and specific eye-movement behavioral correlates of information processing that support exploration decisions,” as a group of researchers from Northwestern University pointed out in their article in Cell in 2014.

Researchers from California Institute of Technology in 2015 interpreted the role of hippocampus in the learning process from a different angle, “At the neural level, our findings indicate evidence for involvement of a very specific neural system for the range of learning rates that would support one-shot learning according to our model. Specifically, activity in the hippocampus was ramped up for high learning rates (90th percentile or more) relative to slower learning rates, in which, by contrast, the hippocampus showed no activity. Thus, the hippocampus appeared to be recruited in a switch-like manner, coming on only when one-shot learning occurred and being silent otherwise.”

The seahorse and the reptilian brain

Hippocampus has the form of a seahorse hence the name. It’s a part of a limbic system of a mammalian brain. Caudate nucleus is located in the oldest and smallest region of the human brain. It evolved hundreds of millions of years ago and is more like the entire brain of present-day reptiles. For this reason, it is often called the reptilian brain.

Researchers from the McGill University in Canada repeated in their experiments in 2013 the results of many other researchers before them, which showed a clear difference “between hippocampal-dependent spatial navigational strategies and caudate nucleus-dependent stimulus-response navigational strategies… The hippocampus is critical for allocentric spatial learning and memory, and the formation of a cognitive map, i.e. learning and memory for the relationships between environmental landmarks irrespective of the position of the observer, such that any target location can be reached in a direct path from any starting position… The striatum (caudate nucleus), in comparison, is critical for response learning and memory, and habit formation by making rigid stimulus-response associations.”

They also specified that, “The hippocampus and striatum (caudate nucleus) are also involved in decision making processes. The decision making process dependent on the hippocampus, involves projecting oneself into future situations to create expectations about action outcomes. In contrast, the decision making process dependent on the striatum, uses past experiences to associate actions with values.

Scientists from the University of Montreal during four years of research demonstrated “an inverse relationship between grey matter in the striatum and hippocampus.” As they stated, “There is a large amount of evidence that supports the hypothesis that the use of spatial strategies is associated with increased hippocampal grey matter and activity, while the use of response strategies is associated with increased grey matter and activity in the striatum…”

Hippocampus inspired neural network architecture

Recently researchers from DeepMind have proposed a predictive map theory inspired by the recent neuroscience research of hippocampus and their knowledge of reinforcement learning algorithms. They believe that “the predictive map theory can be translated into a neural network architecture.”

Even earlier than that researchers from the University of Lethbridge in Canada proposed in their paper (published in December 2016) the idea “that key features of processing in the hippocampus support a flexible Model-based Reinforcement Learning (MBRL) mechanism for spatial navigation that is computationally efficient and can adapt quickly to change.” They wrote, “We investigate this idea by implementing a computational MBRL framework that incorporates features inspired by computational properties of the hippocampus: a hierarchical representation of space, “forward sweeps” through future spatial trajectories, and context-driven remapping of place cells. We find that a hierarchical abstraction of space greatly reduces the computational load (mental effort) required for adaptation to changing environmental conditions, and allows efficient scaling to large problems. It also allows abstract knowledge gained at high levels to guide adaptation to new obstacles. Moreover, a context-driven remapping mechanism allows learning and memory of multiple tasks.”

Cognitive maps for the model of the world

Now it turns out that the role of hippocampus reaches far beyond spatial navigation. “Whereas the hippocampus is essential to spatial navigation via a cognitive map, its role derives from the relational organization and flexibility of cognitive maps and not from a selective role in the spatial domain. Correspondingly, hippocampal networks map multiple navigational strategies, as well as other spatial and nonspatial memories and knowledge domains that share an emphasis on relational organization. These observations suggest that the hippocampal system is not dedicated to spatial cognition and navigation, but organizes experiences in memory, for which spatial mapping and navigation are both a metaphor for and a prominent application of relational memory organization.” Howard Eichenbaum from Boston University wrote in his paper in April 2017.

Researchers from the Cognitive Neuroimaging Unit of the University of Paris in 2017 provided an even broader view on the subject stating that our brain implements “confidence-weighted learning algorithm, acting as a statistician that uses probabilistic information to estimate a hierarchical model of the world.”

Hacking the moral brain in reverse

This story began from my interest in fairy tales. In particular, I was fascinated by a phylogenetic research that had traced the origins of some of the most popular modern fairy tales back in time to the Bronze Age. Many books and articles by the most prominent researchers in folklore and storytelling: Vladimir Propp, Claude Levi-Strauss, Jack Zipes, Jerome Bruner, convinced me that magic folk tales have played a crucial role in the domestication of the human species.

Most researchers did agree on the point that fairy tales influence our brain implicitly although their studies at the same time were focused at the explicit language of fairy tales. Claude Levi-Strauss, for instance, suggested that fairy tales may carry a “metalanguage” atop of the ordinary language. Jerome Bruner while highlighting the implicit nature of the influence of a good story coined the termto domesticate uncertainty.”

The study by neuroscientists at the University of Southern California published in September 2017 shows that identification of distributed representations of stories in either English, Farsi, or Mandarin Chinese takes place in the same areas of the brain known as the default mode network. That network includes hippocampus. The research results demonstrated “that neuro-semantic encoding of narratives happens at levels higher than individual semantic units and that this encoding is systematic across both individuals and languages.”

We have hypothesised that a fairy tale as a dense string of events with randomly varied levels of probability and credibility shifts the balance in learning from relying on prior knowledge towards exploring newly obtained information. We called the implicit brain coding contained in fairy tales the Brain Refresh Button. It is not language or culture sensitive. It enhances people’s ability to individually exercise reason irrespectively of the cultural context.

Yale Psychology Professor Paul Bloom and his team in the research of the morality of babies discovered that the human brain, most probably, has a hardwired set of basic (or naive) moral principles and moral emotions. Naive moral principles and emotions need to be calibrated to the real life and to the modern world. “This is a domain in which there is a fascinating interplay between innate capacities, cultural learning, and the individual exercise of reason,” as professor Bloom puts it.

We believe that good fairy tales play the key role in the calibration of the naive moral principles and emotions. Among other things they utilise the Brain Refresh Button to do so. Now we are making tools that will rejuvenate the implicit code of good fairy tales because the vast majority of stories around us is bad. They have been modified to address only to the reptilian part of human brains that maximises short term reward for automatic reactions.

Can consciousness be hacked?

Researchers from the University of Bern in 2015 presented a concept that “hippocampus is a place of interaction between unconscious and conscious memory.”

A group of researchers from the University Hospital Schleswig-Holstein and the University of Kiel in Germany in a paper published in 2011 announced the results of their research that had provided evidence that human hippocampal CA1 neurons are essential for the retrieval of autobiographical episodic memory and that they are important for autonoetic consciousness.

To be continued…