Facebook AI and Inria 👍🏻 investigate languages 🗣 ️ and brain🧠; Rutgers Uni🇺🇸 pushes neuromorphic computations forward with a biological backprop algorithm; 🇩🇪 🇺🇸 investigate over brain states with Information Theory 👩 💻 revealing a high info content for social tasks!

Why you should care about Neuroscience?
Neuroscience is the root for nowadays artificial intelligence 🧠🤖. Reading and being aware of the evolution and new insights in neuroscience not only will allow you to be a better "Artificial Intelligente" guy 😎 , but also a finer neural network architectures creator 👩 💻 !
This month a lot of exciting publications! First, FacebookAI and Inria and CNRS propose a new model-based approach to investigate over language basis in the Brain, bringing a new research methodology that is comparable with classical and heavy-data needed experiments. Second, a fantastic improvement for neuromorphic computation from Rutgers University, which defines a biologically inspired backprop that satisfies neuromorphic principles. Finally, a collaboration between Pennsylvania Uni, Santa Fe Uni, Julich Research centre and RWTH Aachen Uni to define how the connectome is energy-efficient and optimized to deal with high information content states.
Model-based Analysis of Brain Activity Reveals the Hierarchy of Language in 305 Subjects
Charlotte Caucheteux, Alexandre Gramfort, Jean-Remi King, Paper
As we saw many times in this series, the bases of language, how languages are structured across brain regions, how we learn languages is hot topic in neuroscience research. To get a deeper understanding of the language processes within the brain, neuroscientists usually run a standard experiment procedure, successfully applied by pioneers as Lerner, employing functional magnetic resonance imaging (fMRI). fMRI records the brain response signal after a stimulus (e.g. listening to a story, to a specific paragraph, or scrambled sounds). Then, a correlation called Inter Subject Correlation (ISC) is computed for a single subject brain voxel with the average brain voxel activity from the rest of the subjects. The final brain map defines regions that are specifically activated for language stimuli. Although Lerner’s technique has proved to be successful, fMRI-ISC requires a huge amount of data to obtain meaningful results, scaling up as the number of subjects times the number of stimuli. In this paper, researchers from Facebook AI, Inria and CNRS propose a model-based Lerner experiment, which works with fewer data and can achieve the same performance as ISC studies.
Fig.1 compares the two methodologies. If on one side the brains-to-brain correlation, Lerner’s approach, requires a lot of recordings from different subjects to achieve an explanation of the brain’s temporal receptive field (TRFs), the model-to-brain correlation exploits the power of deep-learning model such as GPT-2 to achieve the same performance. GPT models have been studied extensively and they share many similarities with the human brain in structuring the mapping for regular speech and modified stimuli responses. The hierarchy of language processing is recovered and predicted accurately enough, by analysing and extracting latent features from the eighth GPT-2 layer. The eighth layer is chosen because it is an intermediate layer of transformers that proved to encode relevant linguistic features.

Fig.2 reports the experiment results, where each colour depicts a specific stimulus. The model-based approach replicates the same Lerner’s experiment performance just using 7 subjects listening to a 7 minutes story and scrambled narratives. As further proof of convergence, the authors have extended the model to 305 subjects listening to a 4-hour story, showing how the model was already effective.
This paper is an excellent example to enforce the idea that naturalistic stimuli and deep neural networks form a powerful couple for language bases studies. If on one side we need to understand the limits of some deep learning models, on the other we can think of new solutions for these models, to employ them routinely with in-vivo studies.

BioGrad: Biologically Plausible Gradient-Based Learning for Spiking Neural Networks
Guangzhi Tang, Neelesh Kumar, Ioannis Polykretis, Konstantinos P. Michmizos, Paper
Neuromorphic Computing is an emerging computing paradigm. This hardware is brain-inspired and allows to easily spin-up neural network architectures. One of the most promising neural network architecture is the spiking neural network (SNN), that have shown energy-efficient, massively parallel and low-latency solution for AI problems on neuromorphic hardware. Despite their success, SNN has a hurdle to solve, which is the use of backpropagation. Backpropagation has been widely used in traditional neural network applications, however, it is not inspired by the human brain. This leads research to find a replacement for backprop, which can satisfy the three neuromorphic principles:
- spike-based computation: each computation is spike-based and should offer energy-efficient solutions (which backprop doesn’t)
- local information processing: neuromorphic networks should enable asynchronous computations and massive parallelism – very hard to be achieved by backprop
- rapid online computations: networks should not require information from future time steps (low-latency solution)
For these reasons, Rutgers researchers proposed in this paper s Biologically Plausible Gradient-based learning is called BioGrad for SNN. Fig.1 shows the building block of this algorithm, which satisfies neuromorphic computational principles. The core of the algorithm is to use a multi-compartment neuron model, which is made of a spiking somatic part and a non-spiking apical area. In particular, the somatic region integrates the feed-forward presynaptic inputs into its membrane voltage, spiking every time the voltage is higher than a threshold. The apical compartment receives top-down feedback from the error neurons, integrating its membrane voltage. A periodic sleep phase is introduced, so that weights were updated in an unsupervised manner with local Hebbian rule and random inputs.

The experimental setup was tested on the MNIST dataset and compared with biologically inspired methodologies (tab.1). BioGrad achieves an accuracy of 98.13% on MNIST dataset, which is better or comparable with other methodologies, which, however, do not satisfy neuromorphic principles. BioGrad has also been tested against standard backprop algorithms, with stochastic gradient descent or Adam optimiser, achieving the same accuracy level.
Finally, to show that BioGrad algorithm can be used directly on-chip, authors have deployed the model on Intel’s Loihi processor. The training was done on a single hidden layer, for hardware limitations, with 100 hidden units on MNIST. The final test accuracy was 93.32% consuming 400 times less energy per training sample than on GPU.
The Information Content of Brain States is Explained by Structural Constraints on State Energetics
Leon Weninger, Pragya Srivastava, Dale Zhou, Jason Z. Kim, Eli J. Cornblath, Maxwell A. Bertolero, Ute Habel, Dorit Merhof, Dani S. Basset, Paper
What does it happen when a piece of information is processed in the brain? We supposed this information triggers a signal propagation, which travels along the structural connectome of the brain and induces changes. However, how much do we know about these changes? How is the transition between different states? In this great collaboration between the University of Pennsylvania, the RWTH Aachen Uni, the Research Centre Julich and Santa Fe Institute authors have investigated the information content from fMRI datasets and proved the following hypotheses:
- The brain will show different levels of information content, based on specific cognitive functions
- The structural connectome is organised to support transitions to observe brain states, with an efficient energy consumption
- The energy required to reach high information content states is greater than the energy required to reach low information content states.

Fig. 5a shows the analysis from the mean information content for all the observed tasks on 596 subjects. Interestingly, the social activity task shows a significantly higher information content than all other tasks. On the other side, gambling and emotional task show low information content. Furthermore, authors have investigated the distribution of information content across brain regions, as shown in fig. 5b. Working memory, emotion and relational task show a positive skew, indicating an anisotropic regional distribution in the brain, with some regions highly contributing more than others. Overall, the remaining task shows a Gaussian profile.

The skew analysis led the authors to investigate the energy requirement for high information states, which should be far away from the mean state and should require higher energies than states with low information content. Indeed, from statistical analyses, high information states (like social tasks) require higher energy to drive the brain into these states. In particular, the structural connectome appears swell structured for reaching high information content states, satisfying the final paper hypothesis.
To sum up, the authors have run a thorough statistical analysis on fMRI to detect the brain information content. Social tasks show the highest information content, while emotional and gambling tasks have a low weight. Analysing the brain as a whole, information content is sensitive to task differences across brain regions and this could be used as a pivotal assumption for understanding altered brain dynamics, dysfunctions and psychopathology. Finally, this paper proves how information theory can be easily applied to Neuroscience and how many discoveries about brain energy constraints and connectome we could have, to further expand artificial neural networks in the close future.
I hope you like this review on October 2021 Neuroscience arxivg.org
papers. Please, feel free to send me an email for questions or comments at: [email protected]