
Materials
This is the second article of the series, namely "Cognitive Computational Modelling for Spatio-Temporal fMRI in Ventral Temporal Cortex". If you want to check out the whole series, go to the following link.
Cognitive-Computational-Modelling-for-Spatio-Temporal-fMRI-in-Ventral-Temporal-Cortex
I will introduce the topic of discovery neuroimaging analysis, their use case in the research of brain decoding. Let’s get started.
All related materials are hosted on my GitHub page. Don’t forget to check it out. If you are a paper lover, you can read the paper version of this series of articles that can also be found in my repo.
Cognitive-Computational-Modelling-for-Spatio-Temporal-fMRI-in-Ventral-Temporal-Cortex
In this article, we utilize state-of-the-art explanatory neuroimaging technologies such as echo-planar, region-of-interest (RoI), statistical map, anatomical and glass brain methods, to visualize and pre-analyze the visual structure of fMRI samples.
As always in the case ML problem, it is crucial to understand, analyze and visualize the data before processing.
Here, we utilize the "Nilearn" framework that is statistical computing and neuroimaging platform built on top of "Scikit-Learn" to enable researchers to manage, mask, preprocess and decode neuroscientific data. In the previous article, I showed how to install and import Nilearn framework. Don’t worry. I will repeat the process in this article also. Let’s get started.
Discovery fMRI Analysis by Neuroimaging
I performed pre-analysis based on the neuroimaging technologies to visualize the dataset. To accomplish that, I utilized the Echo-planar Averaging for 4-D visualization, region of interest (RoI), statistical map, anatomic, glass brain visualization tools embedded in statistical learning and neuroimaging framework.
From now on, I introduce the different technologies and visualization tools with their simple yet effective implementations in Python. Fasten the belts!
But, before that, in the case you missed the last article (part-I), I provide installation guidelines as follows.
Now, we can start the actual visualization process.
4-D Visualization of fMRI Volume and EPI
Echo-planar imaging is a very fast magnetic resonance (MR) imaging technique capable of acquiring an entire MR image in only a fraction of a second [18]. In single-shot echo-planar imaging, all the spatial-encoding data of an image can be obtained after a single radio-frequency excitation [18]. Here, we visualize the EPI for subjects of the Haxby experiment from cuts of frontal, axial, and lateral regions. EPI visualization provides realistic insights from the activated regions in the brain that plays a crucial role in spatio-temporal brain decoding. I provide the same image as at the beginning of the article for the sake of simplicity.


Region of Interest Analysis
A common way to analyze fMRI data is performing the region of interest (RoI) analysis that involves the extraction of signals from specified areas. The most theoretically agnostic use of ROI analysis is to simply explore the underlying signal behind a whole-brain voxel-wise analysis [17]. After extracting statistically meaningful areas, we can perform severity of correction for multiple statistical tests instead of a large number of voxels in the brain. Further, most ML decoders are performed on the RoI’s of subjects instead of whole brain medium.

Statistical Maps
Statistical Parametric Mapping refers to the construction and assessment of spatially extended statistical processes used to test hypotheses about functional imaging data [5]. Generally, the prior step in statistical fMRI is to create a thresholded statistical map, representing the regions that are active (above a threshold) [17]. Hence, it is useful in examining differences in brain activity recorded during neuroscientific experiments.

Direct fMRI Visualizations
Simple and compact visualization of fMRI data is quite an important topic in the context of neuroimaging since it enables researchers to view cortical brain activity. So, I directly plot the temporally averaged fMRI data of subject 2 to further visualization.

Anatomic Visualizations
I visualized the anatomical structure of the fMRI (by default 3 cuts: Frontal, Axial, and Lateral) obtained by the temporally averaged fMRI data of subject 2 to generate insights before decoding.

Glass Brain
The Glass Brain is a state-of-the-art real-time brain visualization technology that is created on the Unity 3D game engine and powered by NVIDIA’s GPU computing [19]. Its inputs include an individual’s brain structure, both tissue, and fiber tract architecture, obtained from high-resolution MRI-DTI brain scans. Real-time brain activity and functional interactions among networks are superimposed on the brain structure using high-density EEG (electroencephalography) [19]. Here, I project frontal, axial, and lateral sides of temporally averaged fMRI data of subject 5.


If you want to go further and visualize the data in an interactive 3D fashion, here is the code.
Yeah! That’s it for this article. I covered the most common visualization techniques for fMRI data in depth.
Congratulations! You completed the second article and took a step through cognitive computational approaches for decoding the human brain.
In the next article, we’ll perform functional connectivity and similarity analysis to further capture statistical properties for the human brain.
Links of Articles
- Published Articles
Introduction to Cognitive Computational Modelling of Human Brain (Part-I)
2.
3.
Functional Connectivity and Similarity Analysis of Human Brain (Part-III)
4.
Unsupervised Representation Learning on Distributed Regions in the Human Brain (Part-IV)
- On the Way (Coming soon…)
- Placeholder for Part-V
Further Reading
The following list of references is utilized in my research for both machine learning and neuroscience sides. I highly recommend copy-and-paste the references and review them in brief.
References
[1] J. L. Ba, J. R. Kiros, and G. E. Hinton. Layer normalization, 2016.
[2] L. Buitinck, G. Louppe, M. Blondel, F. Pedregosa, A. Mueller, O. Grisel, V. Niculae, P. Prettenhofer, A. Gramfort, J. Grobler, R. Layton, J. VanderPlas, A. Joly, B. Holt, 10 and G. Varoquaux. API design for machine learning software: experiences from the scikit-learn project. In ECML PKDD Workshop: Languages for Data Mining and Machine Learning, pages 108–122, 2013.
[3] X. Chu, Z. Tian, Y. Wang, B. Zhang, H. Ren, X. Wei, H. Xia, and C. Shen. Twins: Revisiting the design of spatial attention in vision transformers, 2021.
[4] K. Crammer, O. Dekel, J. Keshet, S. Shalev-Shwartz, and Y. Singer. Online passive aggressive algorithms. 2006.
[5] K. J. Friston. Statistical parametric mapping. 1994.
[6] C. G. Gross, C. d. Rocha-Miranda, and D. Bender. Visual properties of neurons in inferotemporal cortex of the macaque. Journal of neurophysiology, 35(1):96–111, 1972.
[7] S. J. Hanson, T. Matsuka, and J. V. Haxby. Combinatorial codes in ventral temporal lobe for object recognition.
[8] J. Haxby, M. Gobbini, M. Furey, A. Ishai, J. Schouten, and P. Pietrini. "visual object recognition", 2018.
[9] R. A. Heckemann, J. V. Hajnal, P. Aljabar, D. Rueckert, and A. Hammers. Automatic anatomical brain mri segmentation combining label propagation and decision fusion. NeuroImage, 33(1):115–126, 2006.
[10] D. Hendrycks and K. Gimpel. Gaussian error linear units (gelus), 2020.
[11] S. Huang, W. Shao, M.-L. Wang, and D.-Q. Zhang. fmribased decoding of visual information from human brain activity: A brief review. International Journal of Automation and Computing, pages 1–15, 2021.
[12] R. Koster, M. J. Chadwick, Y. Chen, D. Berron, A. Banino, E. Duzel, D. Hassabis, and D. Kumaran. Big-loop recurrence ¨ within the hippocampal system supports integration of information across episodes. Neuron, 99(6):1342–1354, 2018.
[13] E. Maor. The Pythagorean theorem: a 4,000-year history. Princeton University Press, 2019.
[14] K. A. Norman, S. M. Polyn, G. J. Detre, and J. V. Haxby. Beyond mind-reading: multi-voxel pattern analysis of fmri data. Trends in cognitive sciences, 10(9):424–430, 2006.
[15] A. J. O’toole, F. Jiang, H. Abdi, and J. V. Haxby. Partially distributed representations of objects and faces in ventral temporal cortex. Journal of cognitive neuroscience, 17(4):580–590, 2005.
[16] F. Pedregosa, G. Varoquaux, A. Gramfort, V. Michel, B. Thirion, O. Grisel, M. Blondel, P. Prettenhofer, R. Weiss, V. Dubourg, J. Vanderplas, A. Passos, D. Cournapeau, M. Brucher, M. Perrot, and E. Duchesnay. Scikit-learn: Machine learning in Python. Journal of Machine Learning Research, 12:2825–2830, 2011.
[17] R. A. Poldrack. Region of interest analysis for Fmri. Social cognitive and affective neuroscience, 2(1):67–70, 2007.
[18] M. Poustchi-Amin, S. A. Mirowitz, J. J. Brown, R. C. McKinstry, and T. Li. Principles and applications of echo-planar imaging: a review for the general radiologist. Radiographics, 21(3):767–779, 2001.
[19] R. P. Reddy, A. R. Mathulla, and J. Rajeswaran. A pilot study of perspective taking and emotional contagion in mental health professionals: Glass brain view of empathy. Indian Journal of Psychological Medicine, page 0253717620973380, 2021.
[20] S. M. Smith, K. L. Miller, G. Salimi-Khorshidi, M. Webster, C. F. Beckmann, T. E. Nichols, J. D. Ramsey, and M. W. Woolrich. Network modelling methods for fmri. Neuroimage, 54(2):875–891, 2011.
[21] K. Tanaka. Inferotemporal cortex and object vision. Annual review of neuroscience, 19(1):109–139, 1996.
[22] M. S. Treder. Mvpa-light: a classification and regression toolbox for multi-dimensional data. Frontiers in Neuroscience, 14:289, 2020.
[23] M. P. Van Den Heuvel and H. E. H. Pol. Exploring the brain network: a review on resting-state fmri functional connectivity. European neuropsychopharmacology, 20(8):519–534, 2010.
[24] G. Varoquaux, A. Gramfort, J. B. Poline, and B. Thirion. Brain covariance selection: better individual functional connectivity models using population prior. arXiv preprint arXiv:1008.5071, 2010.
[25] Y. Wang, J. Kang, P. B. Kemmer, and Y. Guo. An efficient and reliable statistical method for estimating functional connectivity in large scale brain networks using partial correlation. Frontiers in neuroscience, 10:123, 2016.
[26] S. Wold, K. Esbensen, and P. Geladi. Principal component analysis. Chemometrics and intelligent laboratory systems, 2(1–3):37–52, 1987.
[27] S. Wold, K. Esbensen, and P. Geladi. Principal component analysis. Chemometrics and intelligent laboratory systems, 2(1–3):37–52, 1987.