Alzheimer Diagnosis with Deep Learning: Data Preprocessing
Data preprocessing for MRI ahead of model implementation to predict Alzheimer’s disease
This article is part of a series in which I expose the research, implementation and conclusions that were extracted from my Master’s Thesis in Artificial Intelligence. While a previous article addressed the exploratory phase of the thesis (literature review), this article presents the first part of the descriptive phase (data preprocessing). One last article will describe model implementation.
The entire project can be found on the following GitHub repository: https://github.com/oscardp96/TFM-Alzheimer-Diagnosis
Thanks for reading
In a previous article, the current state of the art in Alzheimer’s computerized diagnosis was analyzed. From that review, a few interesting conclusions could be extracted for the process of implementing a classification system based on MRI. These conclusions are summarized into the following points:
- Convolutional Neural Networks are the obvious choice for model implementation. There is not much to discuss about if any other machine learning or deep learning model should be used instead. The discussion focuses mainly on how to train the CNN.
- Image registration is really important. Multiple medical images adapted to a reference atlas will represent the same anatomical structures on the same regions of the image. This would help a CNN identify areas of interest.
- Skull-stripping can be a very useful technique for eliminating noise. The information of the skull is irrelevant for Alzheimer’s diagnosis, so eliminating that information should simplify model training.
- A decision must be made about the structure of the images of the dataset. Multiple image types can be used, being MRI and PET the most common. Also, the images dimensionality can be 4D (time series) or 3D, but can be converted to 2D, they can be augmented, patches can be extracted from them, etc.
In this experiment, two neural networks will be trained: a 2D InceptionV3 network pre-trained with ImageNet weights and a 3D ResNet.
Data
Data used in preparation of this article were obtained from the Alzheimer’s Disease Neuroimaging Initiative (ADNI) database (adni.loni.usc.edu). As such, the investigators within the ADNI contributed to the design and implementation of ADNI and/or provided data but did not participate in analysis or writing of this report. A complete listing of ADNI investigators can be found at: https://adni.loni.usc.edu/wp-content/uploads/how_to_apply/ADNI_Acknowledgement_List.pdf
From the data provided by ADNI, more than 3000 T1-weighted MRI images were obtained. The sample consist of patients between 55 and 92 years old and were assigned to one of three possible labels: AD, MCI and NC. The complete distribution can be seen in Table 1.
Of the whole sample, 15% was kept outside the training, to be used as a test set. At the same time, another 15% was separated from the remaining training set to be used as a validation set for evaluating the quality of the model as the training progresses (with each epoch).
Tools
Preprocessing biomedical data can be quite complex, but in this experiment, the procedure was carried out using libraries and Open Source tools applied indistinctly for all images, seeking the best possible results, but without treating each image separately. This is a good way to test the extent to which the tools available today make it possible to build systems of this type, as individual image processing may not be affordable for data scientists with no in-depth experience in radiology. Having stated that, the following tools were used for image preprocessing:
- The Deep Learning Toolkit for Medical Imaging (DLTK) [1] is a TensorFlow extension that provides useful methods and models to enable deep learning on biomedical images. Although this framework was not used for model implementation, it was really useful for image preprocessing, providing basic methods for whitening and spatial normalization of the images. Also, this framework allowed for easy and fast experimentation, although much of the methods used were not applied in the final work.
- SimpleITK was used for transforming the .nii images provided by ADNI into numpy arrays.
- SimpleElastix [2] was used for registrating the ADNI images to the MNI 305 Atlas [3]-[6]. This is a medical image registration library built on SimpleITK.
- For skull-stripping, the FSL BET [7] tools implemented into Python by the Nipype [8] library were used. Although FSL BET by itself works as a command line tool for UNIX systems, Nipype methods allow to execute it from a Python environment.
Spatial Normalization
The spatial normalization preprocessing step aims to ensure that the spatial structure of all the images of the dataset are as similar as possible. Originally, the images from ADNI had a considerable variety of shapes, and different sets from within the database had been preprocessed beforehand with different techniques. Due to this, images needed to be resampled to a common isotropic resolution, and needed to be registered to a common atlas.
Thus, in a first step, images were resampled to an isotropic resolution of 2mm³, meaning that every voxel would represent 2mm³ of space in the “real” world. For that, a very simple method was created, based on a code snippet included here [1]. Notice that the method takes images in SimpleITK format, not as numpy arrays.
Once an image is resampled, it can be registered to a reference image. An atlas is basically an image like any other, and it can be managed in the same way. It should also be resampled to the same isotropic resolution as the rest of the images:
Resampling the MNI 305 atlas to the isotropic resolution of 2mm³ resulted in an image with a final resolution of 78x110x86. Therefore, after registration, every image in the dataset would have that same resolution.
For image registration, the following method was implemented, using SimpleElastix. It is worth mentioning that affine transformation was used, since non-rigid transformation was much slower and tended to distort the final image.
After defining the resampling and registration methods, and having loaded the atlas, a complete preprocessing flow can be built. Originally, the ADNI database mixes all images, ignoring the labels (AD/MCI/NC), and instead provides a description file (.csv) in which the label can be indexed using the image ID. Therefore, this preprocessing flow should, for every image:
- Load the image and extract its label.
- Resample and registrate the image to the atlas.
- Write the image into a new disk location, organized on a folder that allows to easily identify its label.
For this last step, a destination folder should be prepared:
Finally, the main preprocessing method can be implemented:
With this method, all that is left would be walking the original database to preprocess all the images. This can be done using simple loops.
Skull-stripping
After spatial normalization, a skull-stripping step has been carried out to remove irrelevant information from the images and leave only the brain tissue. As previously mentioned, FSL BET was used for this, running on Nipype. The key here was to find the appropriate value for the main hyper-parameter: the fractional intensity threshold. It basically measures the aggressiveness of the algorithm in removing image components that do not represent the brain tissue. A very small value would leave some irrelevant information in the image, while a very big value would remove some brain tissue.
The problem here was that, since not all MRI data is obtained and preprocessed in the same way before being stored in the ADNI database, finding the ideal fractional intensity threshold for the entire dataset would not be feasible. At the same time, keeping a simple approach was a key aspect of this experiment. So after multiple tests, a threshold of 0.2 was used, since it was able to keep a correct balance for most images, without being perfect for most of them.
Once the skull-stripping method was implemented, the spatially normalized images could be iterated to generate a new skull-stripped dataset.
Fig. 1 finally shows three cuts of an MRI completely preprocessed. It is resampled to an isotropic resolution of 2mm³, registered to the MNI 305 atlas, and skull-stripped. In this image, there is hardly any trace of the skull and no brain tissue has been lost, but in other cases one of these situations may occur because the fractional intensity threshold of 0.2 does not work equally well with all images.
Preparing a 2D Dataset
The current state of the dataset would allow the training of a 3D Convolutional Neural Network, which is one of the objectives that were established at the beginning of this article. The other objective was to fine-tune an InceptionV3 network pre-trained with Imagenet, which would require a bidimensional dataset.
For the 3D to 2D transformation, a simple algorithm has been implemented, inspired from the work at [9]. It involves making multiple axial (horizontal) cuts and placing them on the same plane to construct a two-dimensional image. More specifically, 16 different slices were taken from each 3D image, and placed on a “matrix” of 4x4 images. The resulting two-dimensional image is replicated three times to adopt RGB dimensions. Thus, 16 slices of 110x86 size result in an image of 440x344x3.
Prior to extracting the cuts, the 3D images are normalized to have zero mean and unit standard deviation. In other words, a whitening procedure is applied. Subsequently, the 16 slices are selected between the axial heights of 28 and 60, which after multiple tests were found to represent sufficient information. The slices with a height greater than 60 showed the upper part of the brain very far away, and a height less than 28 showed mostly regions of the brainstem. The 16 slices are distributed at equal distances from each other, i.e., one axial slice is extracted from every two. A result example can be seen in Fig. 2.
The following code presents the method that creates the 2D image:
That method was used inside another method, that loaded the 3D image and applied the whitening process. This method had to be run on all the images of the dataset. For the sake of brevity, this article is not going to present this last loop-based code, but it can be checked out here.
TFRecords database and train/test split
As previously stated, Google Colaboratory was used for model training. This platform has two limiting factors:
- It has a time limit when it comes to model training, so it was important for the training process to be as fast as possible.
- It offers 12GB of RAM, which is clearly not enough to load all 3000 images on memory, especially the ones from the 3D dataset.
There is no real solution for the first problem, it is always going to be a limiting factor. However, the second can be addressed by building a TFRecords database. TFRecords is a solution offered by Tensorflow that allows to save the dataset in a special format, using files that can be read much faster by tensors. This requires duplicating the information on the hard drive to save it in this format, but it is also much faster than using Python’s native generators.
The process is very generic, so for the sake of brevity, the code will not be explained in this article. It can be checked out here.
Future steps
In this article, the process for building both a 3D and 2D MRI dataset was described. All images would be ready to be used for training deep learning models, which is something that will be addressed in a future article.
References
[1] M. Rajchl, S. I. Ktena, and N. Pawlowski, An Introduction to Biomedical Image Analysis with TensorFlow and DLTK (2018), Medium.com
[2] K. Marstal, F. Berendsen, M. Staring, and S. Klein, SimpleElastix: A user-friendly, multi-lingual library for medical image registration (2016), Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops, p. 134–142
[3] A. C. Evans, An MRI-based stereotactic atlas from 250 young normal subjects (1992), Soc. neurosci. abstr
[4] D. L. Collins, P. Neelin, T. M. Peters, and A. C. Evans, Automatic 3D intersubject registration of MR volumetric data in standardized Talairach space (1994), Journal of Computer Assisted Tomography, vol. 18, no 2, p. 192–205
[5] A. C. Evans et al., Anatomical mapping of functional activation in stereotactic coordinate space (1992), Neuroimage, vol. 1, no 1, p. 43–53
[6] A. C. Evans, D. L. Collins, S. R. Mills, E. D. Brown, R. L. Kelly, and T. M. Peters, 3D statistical neuroanatomical models from 305 MRI volumes (1993), IEEE conference record nuclear science symposium and medical imaging conference, p. 1813–1817
[7] S. M. Smith, Fast robust automated brain extraction (2002), Human Brain Mapping, vol. 17, no 3, p. 143–155
[8] K. Gorgolewski et al., Nipype: a flexible, lightweight and extensible neuroimaging data processing framework in python (2011), Frontiers in neuroinformatics, vol. 5, p. 13
[9] Y. Ding et al., A Deep learning model to predict a diagnosis of alzheimer disease by using 18F-FDG PET of the brain (2018), Radiology, vol. 290, no 2, p. 456–464
Thanks again for reading. Let’s continue this conversation on LinkedIn: https://www.linkedin.com/in/oscar-darias-plasencia/
