The world’s leading publication for data science, AI, and ML professionals.

Cognitive Computational Modelling for Spatio-Temporal fMRI in Ventral Temporal Cortex

Decoding and Understanding the Human Brain

Glass Brain Visualizations (Image By Author)
Glass Brain Visualizations (Image By Author)

Materials

In this series of articles, we will go over the following articles

  • Introduction to Cognitive Computational Modelling of Human Brain (Part-I)
  • Discovery Neuroimaging Analysis (Part-II)
  • Functional Connectivity and Similarity Analysis of Human Brain(Part-III)
  • Unsupervised Representation Learning of Distributed Regions in Human Brain (Part-IV)
  • Spatio-Temporal Fmri Decoding with Machine Learning and Deep Learning (Part-V)

All related materials are hosted on my Github page. Don’t forget to check it out. If you are a paper lover, you can read the paper version of this series of articles that can also be found in my repo.

Cognitive-Computational-Modelling-for-Spatio-Temporal-fMRI-in-Ventral-Temporal-Cortex

Before diving into technical sides, prior knowledge on

  • Computational Neuroscience
  • Basics of Unsupervised Learning and Machine Learning
  • Simple Neural Networks (MLPs, CNNs, … )

will be quite helpful. But, no worries. These will be high-level, hands-on, and code-first articles. So, let’s get started.

In this article, we’ll briefly discuss multi-voxel pattern analysis (MVPA) on the ventral temporal cortex of the human brain. Multi-voxel pattern analysis & recognition are currently exploited to investigate the information contained in distributed patterns of neural activity to infer the functional role of brain areas and network¹.

From the Machine Learning perspective, it is a directly supervised classification problem. Here, input is generally fMRI data from the neuroscientific experiments. Output is a probability distribution over the categories of interest. In our case, we’ll try to decode the visual stimuli category viewed by human subjects (human faces, cats, chairs, shoes, … ).

MVPA is considered as a supervised classification problem where a classifier attempts to capture the relationships between spatial pattern of fMRI activity and experimental conditions¹.

In our case, fMRI data will be 4D time-series data consist of space and time components.

Table of Content

  1. In part-I, we’ll discuss visual decoding of distributed regions of the human ventral temporal cortex in the context of computational cognitive Neuroscience. Then, we’ll understand why we need to construct spatio-temporal decoding algorithms to perform cognitive tasks. (Part-I)
  2. In part-II, we investigated the functional architecture of the object vision pathway in the human brain by functional magnetic resonance imaging (fMRI) methods to decode the visual stimuli viewed by a human subject. We conduct state-of-the-art explanatory echo-planar, region-of-interest (RoI), statistical map, anatomical and glass brain pre-analysis to discover block-designed 4-D time-series fMRI dataset, namely Haxby dataset, from the study on the face and object representation. (Part-II)
  3. In part-III, to understand geodesic relation in ventral temporal masked fMRI samples, we performed functional connectivity analysis based on the correlation, precision, and partial correlation, and similarity analysis based on the cosine, Minkowski, and euclidean distances. (Part-III)
  4. In part-IV, manifold learning and dimensionality reduction methods are performed on the per subject ventral temporal masks to extract latent representations of spatio-temporal masks that will help further decoding of fMRI. (Part-IV)
  5. In part-V, end-to-end machine learning algorithms from perceptrons to FREMs are developed to categorize the stimuli based on distributed and overlapping regions in the ventral temporal cortex. We further constructed cognitive neural networks, precisely MLPs, 2D and 3D CNNs by taking the advantage of interactions between different streams of visual representations. (Part-V)

Links of Articles

  1. Published Articles

Introduction to Cognitive Computational Modelling of Human Brain (Part-I)

2.

Discovery Neuroimaging Analysis (Part-II)

3.

Functional Connectivity and Similarity Analysis of Human Brain (Part-III)

4.

Unsupervised Representation Learning on Distributed Regions in the Human Brain (Part-IV)

  1. On the Way (Coming soon…)
  2. Placeholder for Part-V

Further Reading

The following list of references is utilized in my research for both machine learning and neuroscience sides. I highly recommend copy-and-paste the references and review them in brief.

References

[1] J. L. Ba, J. R. Kiros, and G. E. Hinton. Layer normalization, 2016.

[2] L. Buitinck, G. Louppe, M. Blondel, F. Pedregosa, A. Mueller, O. Grisel, V. Niculae, P. Prettenhofer, A. Gramfort, J. Grobler, R. Layton, J. VanderPlas, A. Joly, B. Holt, 10 and G. Varoquaux. API design for machine learning software: experiences from the scikit-learn project. In ECML PKDD Workshop: Languages for Data Mining and Machine Learning, pages 108–122, 2013.

[3] X. Chu, Z. Tian, Y. Wang, B. Zhang, H. Ren, X. Wei, H. Xia, and C. Shen. Twins: Revisiting the design of spatial attention in vision transformers, 2021.

[4] K. Crammer, O. Dekel, J. Keshet, S. Shalev-Shwartz, and Y. Singer. Online passive aggressive algorithms. 2006.

[5] K. J. Friston. Statistical parametric mapping. 1994.

[6] C. G. Gross, C. d. Rocha-Miranda, and D. Bender. Visual properties of neurons in inferotemporal cortex of the macaque. Journal of neurophysiology, 35(1):96–111, 1972.

[7] S. J. Hanson, T. Matsuka, and J. V. Haxby. Combinatorial codes in ventral temporal lobe for object recognition.

[8] J. Haxby, M. Gobbini, M. Furey, A. Ishai, J. Schouten, and P. Pietrini. "visual object recognition", 2018.

[9] R. A. Heckemann, J. V. Hajnal, P. Aljabar, D. Rueckert, and A. Hammers. Automatic anatomical brain mri segmentation combining label propagation and decision fusion. NeuroImage, 33(1):115–126, 2006.

[10] D. Hendrycks and K. Gimpel. Gaussian error linear units (gelus), 2020.

[11] S. Huang, W. Shao, M.-L. Wang, and D.-Q. Zhang. fmribased decoding of visual information from human brain activity: A brief review. International Journal of Automation and Computing, pages 1–15, 2021.

[12] R. Koster, M. J. Chadwick, Y. Chen, D. Berron, A. Banino, E. Duzel, D. Hassabis, and D. Kumaran. Big-loop recurrence ¨ within the hippocampal system supports integration of information across episodes. Neuron, 99(6):1342–1354, 2018.

[13] E. Maor. The Pythagorean theorem: a 4,000-year history. Princeton University Press, 2019.

[14] K. A. Norman, S. M. Polyn, G. J. Detre, and J. V. Haxby. Beyond mind-reading: multi-voxel pattern analysis of fmri data. Trends in cognitive sciences, 10(9):424–430, 2006.

[15] A. J. O’toole, F. Jiang, H. Abdi, and J. V. Haxby. Partially distributed representations of objects and faces in ventral temporal cortex. Journal of cognitive neuroscience, 17(4):580–590, 2005.

[16] F. Pedregosa, G. Varoquaux, A. Gramfort, V. Michel, B. Thirion, O. Grisel, M. Blondel, P. Prettenhofer, R. Weiss, V. Dubourg, J. Vanderplas, A. Passos, D. Cournapeau, M. Brucher, M. Perrot, and E. Duchesnay. Scikit-learn: Machine learning in Python. Journal of Machine Learning Research, 12:2825–2830, 2011.

[17] R. A. Poldrack. Region of interest analysis for fmri. Social cognitive and affective neuroscience, 2(1):67–70, 2007.

[18] M. Poustchi-Amin, S. A. Mirowitz, J. J. Brown, R. C. McKinstry, and T. Li. Principles and applications of echo-planar imaging: a review for the general radiologist. Radiographics, 21(3):767–779, 2001.

[19] R. P. Reddy, A. R. Mathulla, and J. Rajeswaran. A pilot study of perspective taking and emotional contagion in mental health professionals: Glass brain view of empathy. Indian Journal of Psychological Medicine, page 0253717620973380, 2021.

[20] S. M. Smith, K. L. Miller, G. Salimi-Khorshidi, M. Webster, C. F. Beckmann, T. E. Nichols, J. D. Ramsey, and M. W. Woolrich. Network modelling methods for fmri. Neuroimage, 54(2):875–891, 2011.

[21] K. Tanaka. Inferotemporal cortex and object vision. Annual review of neuroscience, 19(1):109–139, 1996.

[22] M. S. Treder. Mvpa-light: a classification and regression toolbox for multi-dimensional data. Frontiers in Neuroscience, 14:289, 2020.

[23] M. P. Van Den Heuvel and H. E. H. Pol. Exploring the brain network: a review on resting-state fmri functional connectivity. European neuropsychopharmacology, 20(8):519–534, 2010.

[24] G. Varoquaux, A. Gramfort, J. B. Poline, and B. Thirion. Brain covariance selection: better individual functional connectivity models using population prior. arXiv preprint arXiv:1008.5071, 2010.

[25] Y. Wang, J. Kang, P. B. Kemmer, and Y. Guo. An efficient and reliable statistical method for estimating functional connectivity in large scale brain networks using partial correlation. Frontiers in neuroscience, 10:123, 2016.

[26] S. Wold, K. Esbensen, and P. Geladi. Principal component analysis. Chemometrics and intelligent laboratory systems, 2(1–3):37–52, 1987.

[27] S. Wold, K. Esbensen, and P. Geladi. Principal component analysis. Chemometrics and intelligent laboratory systems, 2(1–3):37–52, 1987.


Related Articles