
This is the fourth article of the series, namely "Cognitive Computational Modelling for Spatio-Temporal fMRI in Ventral Temporal Cortex". If you want to check out the whole series, go to the following link.
Cognitive Computational Modelling for Spatio-Temporal fMRI in Ventral Temporal Cortex
I will introduce the topic of unsupervised representation learning on distributed regions in the human brain and its use case in the research of brain decoding. Let’s get started.
All related materials are hosted on my GitHub page. Don’t forget to check it out. If you are a paper lover, you can read the paper version of this series of articles that can also be found in my repo.
Cognitive-Computational-Modelling-for-Spatio-Temporal-fMRI-in-Ventral-Temporal-Cortex
Unsupervised & Manifold Learning in Human Brain
Functional MRI data are very high-dimensional if one considers all the voxels or surface coordinates acquired with standard imaging parameters. As in our dataset, with the structure of 4D time-series image data, we have a curve of dimensionality problem. Hence, dimension reduction and manifold learning algorithms can reduce the dimensionality of fMRI space by preserving geodesic relations in the lower space representations. We performed PCA, LDA, ICA, NNMF, and MDS as dimension reduction algorithms. Besides, t- SNE, UMAP, ISOMAP, LLE, and Spectral Embedding are performed to generate lower-dimensional manifolds of the fMRI space. Let’s start discovering the intersection of unsupervised learning and the human brain. I performed many unsupervised learning algorithms as it further helps to understand geodesic information underlying the human brain and gives prior information on whether the neural activities in distributed regions are decodable or not.
Let’s install and import all necessary packages. Please refer to previous articles (part-I) for the dataset understanding. Note that this article is not for answering the following questions (which are previously answered)
- How fMRI data is structured? (part-I)
- Why spatio-temporal masking is performed on the human brain? (Part-II)
Here are the pip commands for installing everything we need.
We installed everything (even more) we need. Let’s import them as follows.
We are ready to go! Let’s fetch the Haxby dataset.
Next thing is to prepare, mask, standardize and convert fMRI data to NumPy matrix form as follows.
Preprocessing part is done. So, we can move to the actual processes we want to perform. In this code snippet, "masks" are the masked regions in the human brain that we want to perform cognitive tasks on it and categories are just their labels. We do not use whole voxels for representation learning purposes. Our main idea is to extract and visualize the latent variables in the human brain where the distributed and overlapping patterns of neural activity are happening.
Finally, we can start the actual business as follows. In the following figures, different colors represent different categories. (i.e., blues belong to class 1, oranges belong to class 2 etc.)
To properly visualize the latent space, I utilized the plotly python package. The following code performs 2D and 3D visualization in an interactive fashion.
Dimension Reduction: PCA
PCA is a linear unsupervised dimension reduction algorithm and it computes principal vectors to change the basis of the representation [26]. PCA is a used algorithm in a broad range of topics from image compression to decorrelation of texts. Here, we performed PCA on RoI’s of subject 5 and visualized it as follows.


Dimension Reduction: LDA
LDA is a supervised Dimensionality Reduction algorithm and it is a generalization of Fisher’s linear discriminant, aims to find linear subspace that characterizes the original data space. Since it is supervised, it is a powerful paradigm in representation learning. Here, we performed LDA on RoI’s of subject 5 and visualized it. From the figures, we can see that LDA is outperforming other methods by uniquely separating geodesic distances in the manifolds.

Dimension Reduction: ICA
ICA is a computational approach for separating multi-variate signals into their additive components. It is a natural paradigm for unsupervised dimensionality reduction. Here, we performed ICA on RoI’s of subject 5, and visualized it as follows.

Dimension Reduction: NNMF
NNMF is an iterative non-negative factor analysis to decompose a non-negative matrix into its linear subspaces. It is useful in extracting natural linear subspaces of original data samples. Here, we performed NNMF on RoI’s of subject 5 and visualized it.

Manifold Learning: MDS
MDS is a classical approach for extracting non-linear sub-spaces of the original data space by preserving geodesic distance in the manifold. Lower dimensional embedding is obtained to represent original data in the manifold. Here, we performed MDS on RoI’s of subject 5 and visualized it.

Manifold Learning: t-SNE
T-SNE is an iterative statistical approach for producing non-linear embedding of the original data space by preserving small pairwise distances or localized similarities. It minimizes the Kullback-Leibler divergence between the joint probabilities of the low-dimensional embedding and the high-dimensional data. Here, we performed t-SNE on RoI’s of subject 5 and visualized it.

Manifold Learning: UMAP
UMAP is a recent approach for non-linear embedding, and it generally outperforms t-SNE by a significant margin. It is very similar to t-SNE but it also preserves the global geodesic structure of the data. Here, we performed UMAP on RoI’s of subject 5 and visualized it.

Manifold Learning: ISOMAP
ISOMAP map is also a non-linear embedding algorithm through isometric mapping for accurately estimating the intrinsic geometry of manifold by preserving geodesic distances in the manifold. Here, we performed ISOMAP on RoI’s of subject 5 and visualized it.

Manifold Learning: LLE
LLE is a topology-preserving non-linear dimension reduction algorithm, trying to preserve neighbor structure in the manifold, and it is generally outperforming ISOMAP in terms of optimization and speed thus it has very practical uses in literature. Here, we performed LLE on RoI’s of subject 5 and visualized it.

Manifold Learning: Spectral Embedding
Spectral embedding is also a non-linear embedding algorithm that forms an affinity matrix and applies spectral decomposition to the laplacian graph. Here, we performed Spectral embedding on RoI’s of subject 5 and visualized it.

That’s it for this article. We covered unsupervised representation learning on distributed regions in the human brain and its use case in the research of brain decoding. Congratulations! You completed the fourth article and took a step through cognitive computational approaches for decoding the human brain.
In the next article, we’ll perform comprehensive decoding algorithms from classical ML algorithms to neural networks.
Links of Articles
- Published Articles
Introduction to Cognitive Computational Modelling of Human Brain (Part-I)
2.
3.
Functional Connectivity and Similarity Analysis of Human Brain (Part-III)
4.
Unsupervised Representation Learning on Distributed Regions in the Human Brain (Part-IV)
- On the Way (Coming soon…)
- Placeholder for Part-V
Further Reading
The following list of references is utilized in my research for both Machine Learning and neuroscience sides. I highly recommend copy-and-paste the references and review them in brief.
References
[1] J. L. Ba, J. R. Kiros, and G. E. Hinton. Layer normalization, 2016.
[2] L. Buitinck, G. Louppe, M. Blondel, F. Pedregosa, A. Mueller, O. Grisel, V. Niculae, P. Prettenhofer, A. Gramfort, J. Grobler, R. Layton, J. VanderPlas, A. Joly, B. Holt, 10 and G. Varoquaux. API design for machine learning software: experiences from the scikit-learn project. In ECML PKDD Workshop: Languages for Data Mining and Machine Learning, pages 108–122, 2013.
[3] X. Chu, Z. Tian, Y. Wang, B. Zhang, H. Ren, X. Wei, H. Xia, and C. Shen. Twins: Revisiting the design of spatial attention in vision transformers, 2021.
[4] K. Crammer, O. Dekel, J. Keshet, S. Shalev-Shwartz, and Y. Singer. Online passive aggressive algorithms. 2006.
[5] K. J. Friston. Statistical parametric mapping. 1994.
[6] C. G. Gross, C. d. Rocha-Miranda, and D. Bender. Visual properties of neurons in inferotemporal cortex of the macaque. Journal of neurophysiology, 35(1):96–111, 1972.
[7] S. J. Hanson, T. Matsuka, and J. V. Haxby. Combinatorial codes in ventral temporal lobe for object recognition.
[8] J. Haxby, M. Gobbini, M. Furey, A. Ishai, J. Schouten, and P. Pietrini. "visual object recognition", 2018.
[9] R. A. Heckemann, J. V. Hajnal, P. Aljabar, D. Rueckert, and A. Hammers. Automatic anatomical brain mri segmentation combining label propagation and decision fusion. NeuroImage, 33(1):115–126, 2006.
[10] D. Hendrycks and K. Gimpel. Gaussian error linear units (gelus), 2020.
[11] S. Huang, W. Shao, M.-L. Wang, and D.-Q. Zhang. fmribased decoding of visual information from human brain activity: A brief review. International Journal of Automation and Computing, pages 1–15, 2021.
[12] R. Koster, M. J. Chadwick, Y. Chen, D. Berron, A. Banino, E. Duzel, D. Hassabis, and D. Kumaran. Big-loop recurrence ¨ within the hippocampal system supports integration of information across episodes. Neuron, 99(6):1342–1354, 2018.
[13] E. Maor. The Pythagorean theorem: a 4,000-year history. Princeton University Press, 2019.
[14] K. A. Norman, S. M. Polyn, G. J. Detre, and J. V. Haxby. Beyond mind-reading: multi-voxel pattern analysis of fmri data. Trends in cognitive sciences, 10(9):424–430, 2006.
[15] A. J. O’toole, F. Jiang, H. Abdi, and J. V. Haxby. Partially distributed representations of objects and faces in ventral temporal cortex. Journal of cognitive Neuroscience, 17(4):580–590, 2005.
[16] F. Pedregosa, G. Varoquaux, A. Gramfort, V. Michel, B. Thirion, O. Grisel, M. Blondel, P. Prettenhofer, R. Weiss, V. Dubourg, J. Vanderplas, A. Passos, D. Cournapeau, M. Brucher, M. Perrot, and E. Duchesnay. Scikit-learn: Machine learning in Python. Journal of Machine Learning Research, 12:2825–2830, 2011.
[17] R. A. Poldrack. Region of interest analysis for fmri. Social cognitive and affective neuroscience, 2(1):67–70, 2007.
[18] M. Poustchi-Amin, S. A. Mirowitz, J. J. Brown, R. C. McKinstry, and T. Li. Principles and applications of echo-planar imaging: a review for the general radiologist. Radiographics, 21(3):767–779, 2001.
[19] R. P. Reddy, A. R. Mathulla, and J. Rajeswaran. A pilot study of perspective taking and emotional contagion in mental health professionals: Glass brain view of empathy. Indian Journal of Psychological Medicine, page 0253717620973380, 2021.
[20] S. M. Smith, K. L. Miller, G. Salimi-Khorshidi, M. Webster, C. F. Beckmann, T. E. Nichols, J. D. Ramsey, and M. W. Woolrich. Network modelling methods for fmri. Neuroimage, 54(2):875–891, 2011.
[21] K. Tanaka. Inferotemporal cortex and object vision. Annual review of neuroscience, 19(1):109–139, 1996.
[22] M. S. Treder. Mvpa-light: a classification and regression toolbox for multi-dimensional data. Frontiers in Neuroscience, 14:289, 2020.
[23] M. P. Van Den Heuvel and H. E. H. Pol. Exploring the brain network: a review on resting-state fmri functional connectivity. European neuropsychopharmacology, 20(8):519–534, 2010.
[24] G. Varoquaux, A. Gramfort, J. B. Poline, and B. Thirion. Brain covariance selection: better individual functional connectivity models using population prior. arXiv preprint arXiv:1008.5071, 2010.
[25] Y. Wang, J. Kang, P. B. Kemmer, and Y. Guo. An efficient and reliable statistical method for estimating functional connectivity in large scale brain networks using partial correlation. Frontiers in neuroscience, 10:123, 2016.
[26] S. Wold, K. Esbensen, and P. Geladi. Principal component analysis. Chemometrics and intelligent laboratory systems, 2(1–3):37–52, 1987.
[27] S. Wold, K. Esbensen, and P. Geladi. Principal component analysis. Chemometrics and intelligent laboratory systems, 2(1–3):37–52, 1987.