Explainable Deep Learning in Breast Cancer Prediction
Understanding Convolutional Neural Network Prediction Results in Healthcare
Advanced machine learning models (e.g., Random Forest, deep learning models, etc.) are generally considered not explainable [1][2]. As described in [1][2][3][4], those models largely remain black boxes, and understanding the reasons behind their prediction results for healthcare is very important in assessing trust if a doctor plans to take actions to treat a disease (e.g., cancer) based on a prediction result. In [2], I used the Wisconsin Breast Cancer Diagnosis (WBCD) tabular dataset to present how to use the Local Interpretable Model-agnostic Explanations (LIME) method to explain the prediction results of a Random Forest model in breast cancer diagnosis.
In this article, I use the Kaggle Breast Cancer Histology Images (BCHI) dataset [5] to demonstrate how to use LIME to explain the image prediction results of a 2D Convolutional Neural Network (ConvNet) for the Invasive Ductal Carcinoma (IDC) breast cancer diagnosis.
1. Preparing Breast Cancer Histology Images Dataset
The BCHI dataset [5] can be downloaded from Kaggle. As described in [5], the dataset consists of 5,547 50x50 pixel RGB digital images of H&E-stained breast histopathology samples. These images are labeled as either IDC or non-IDC. There are 2,788 IDC images and 2,759 non-IDC images. Those images have already been transformed into Numpy arrays and stored in the file X.npy. Similarly the corresponding labels are stored in the file Y.npy in Numpy array format.
1.1 Loading Data
Once the X.npy and Y.npy files have been downloaded into a local computer, they can be loaded into memory as Numpy arrays as follows:
X = np.load('./data/X.npy') # images
Y = np.load('./data/Y.npy') # labels (0 = Non IDC, 1 = IDC)The following are two of the data samples, the image on the left is labeled as 0 (non-IDC) and the image on the right is labeled as 1 (IDC).
1.2 Shuffling Data
In the original dataset files, all the data samples labeled as 0 (non-IDC) are put before the data samples labeled as 1 (IDC). To avoid artificial data patterns, the dataset is randomly shuffled as follows:
indices = np.arange(Y.shape[0])
np.random.shuffle(indices)
indices = list(indices)
X = X[indices]
Y = Y[indices]1.3 Transforming Dataset
The pixel value in an IDC image is in the range of [0, 255], while a typical deep learning model works the best when the value of input data is in the range of [0, 1] or [-1, 1]. The class Scale below is to transform the pixel value of IDC images into the range of [0, 1].
class Scale(BaseEstimator, TransformerMixin):
def __init__(self):
pass
def fit(self, X, y):
return self
def transform(self, X):
X1 = X.copy()
X1 = X1 / 255.0
return X11.4 Dividing Dataset for Model Training and Testing
The dataset is divided into three parts, 80% for model training and validation (1,000 for validation and the rest of 80% for training) , and 20% for model testing.
X_train_raw, X_test_raw, y_train_raw, y_test_raw = train_test_split(X, Y, test_size=0.2)X_train = X_train_raw.copy()
X_val = X_train[:1000]
X_train = X_train[1000:]
X_test = X_test_raw.copy()y_train = y_train_raw.copy()
y_val = y_train[:1000]
y_train = y_train[1000:]
y_test = y_test_raw.copy()
2. Training 2D ConvNet Model
The BCHI dataset [5] consists of images and thus a 2D ConvNet model is selected for IDC prediction.
2.1 Creating 2D ConvNet
Similarly to [5], the function getKerasCNNModel() below creates a 2D ConvNet for the IDC image classification.
def getKerasCNNModel():
batch_size = BATCH_SIZE
epochs = EPOCH_SIZE
img_rows, img_cols = X_train.shape[1], X_train.shape[2]
input_shape = (img_rows, img_cols, 3)
model = Sequential()
model.add(Conv2D(16, kernel_size=(3,3), activation='relu', input_shape=input_shape))
model.add(MaxPooling2D(pool_size=(2, 2)))
model.add(Dropout(0.25))
model.add(Conv2D(32, (3,3), activation='relu'))
model.add(MaxPooling2D(pool_size=(2, 2)))
model.add(Dropout(0.25))
model.add(Flatten())
model.add(Dense(128, activation='relu'))
model.add(Dropout(0.5))
model.add(Dense(1, activation='sigmoid'))
model.compile(loss= keras.losses.binary_crossentropy,
optimizer=keras.optimizers.rmsprop(),
metrics=['accuracy'])
return model2.2 Creating Pipeline Component
The class KerasCNN is to wrapper the 2D ConvNet model as a sklearn pipeline component so that it can be combined with other data preprocessing components such as Scale into a pipeline.
class KerasCNN(BaseEstimator, TransformerMixin):
def __init__(self, X_val=None, y_val=None):
self._model = getKerasCNNModel()
self._batch_size = BATCH_SIZE
self._epochs = EPOCH_SIZE
self._X_val = X_val / 255.0
self._y_val = y_val
def fit(self, X, y):
self.history = self._model.fit(X, y,
batch_size=self._batch_size,
verbose=1,
epochs=self._epochs,
validation_data=(self._X_val, self._y_val))
return self
def transform(self, X):
return X def predict_proba(self, X):
y_pred = self._model.predict(X)
return y_pred
def evaluate(self, X, y):
return self._model.evaluate(X,y)
3. Explaining Model Prediction Results
As described before, I use LIME to explain the ConvNet model prediction results in this article.
3.1 Setting Up a Pipeline
Similarly to [1][2], I make a pipeline to wrap the ConvNet model for the integration with LIME API.
from sklearn.pipeline import Pipelinesimple_cnn_pipeline = Pipeline([
('scale', Scale()),
('CNN', KerasCNN(X_val=X_val, y_val=y_val))
])
3.2 Training the ConvNet Model
The ConvNet model is trained as follows so that it can be called by LIME for model prediction later on.
simple_cnn_pipeline.fit(X_train, y_train)3.3 Selecting LIME Explainer
As described in [1][2], the LIME method supports different types of machine learning model explainers for different types of datasets such as image, text, tabular data, etc. The LIME image explainer is selected in this article because the dataset consists of images.
The 2D image segmentation algorithm Quickshift is used for generating LIME super pixels (i.e., segments) [1].
from lime import lime_image
from lime.wrappers.scikit_image import SegmentationAlgorithmexplainer = lime_image.LimeImageExplainer() segmenter = SegmentationAlgorithm(‘quickshift’, kernel_size=1, max_dist=200, ratio=0.2)
3.4 Explaining Model Prediction
Once the ConvNet model has been trained, given a new IDC image, the explain_instance() method of the LIME image explainer can be called to generate an explanation of the model prediction.
An explanation of an image prediction consists of a template image and a corresponding mask image. These images can be used to explain a ConvNet model prediction result in different ways.
Explanation 1: Prediction of Positive IDC (IDC: 1)
Figure 3 shows a positive IDC image for explaining model prediction via LIME.
The code below is to generate an explanation object explanation_1 of the model prediction for the image IDC_1_sample (IDC: 1) in Figure 3.
In this explanation, white color is used to indicate the portion of image that supports the model prediction (IDC: 1).
explanation_1 = explainer.explain_instance(IDC_1_sample,
classifier_fn = simple_cnn_pipeline.predict_proba,
top_labels=2,
hide_color=0,
num_samples=10000,
segmentation_fn=segmenter)Once the explanation of the model prediction is obtained, its method get_image_and_mask() can be called to obtain the template image and the corresponding mask image (super pixels):
from skimage.segmentation import mark_boundariestemp, mask = explanation_1.get_image_and_mask(explanation_1.top_labels[0],
positive_only=True,
num_features=20,
hide_rest=True)
plt.imshow(mark_boundaries(temp, mask))
Figure 4 shows the hidden portion of given IDC image in gray color. The white portion of the image indicates the area of the given IDC image that supports the model prediction of positive IDC.
The code below is to show the boundary of the area of the IDC image in yellow that supports the model prediction of positive IDC (see Figure 5).
temp, mask = explanation_1.get_image_and_mask(explanation_1.top_labels[0],
positive_only=True,
num_features=20,
hide_rest=False)
plt.imshow(mark_boundaries(temp, mask))
Explanation 2: Prediction of non-IDC (IDC: 0)
Figure 6 shows a non-IDC image for explaining model prediction via LIME.
The code below is to generate an explanation object explanation_2 of the model prediction for the image IDC_0_sample in Figure 6. In this explanation, white color is used to indicate the portion of image that supports the model prediction of non-IDC.
explanation_2 = explainer.explain_instance(IDC_0_sample,
classifier_fn = simple_cnn_pipeline.predict_proba,
top_labels=2,
hide_color=0,
num_samples=10000,
segmentation_fn=segmenter
)Once the explanation of the model prediction is obtained, its method get_image_and_mask() can be called to obtain the template image and the corresponding mask image (super pixels):
temp, mask = explanation_2.get_image_and_mask(explanation_2.top_labels[0],
positive_only=True,
num_features=20,
hide_rest=True)
plt.imshow(mark_boundaries(temp, mask))Figure 7 shows the hidden area of the non-IDC image in gray. The white portion of the image indicates the area of the given non-IDC image that supports the model prediction of non-IDC.
The code below is to show the boundary of the area of the IDC image in yellow that supports the model prediction of non-IDC (see Figure 8).
temp, mask = explanation_2.get_image_and_mask(explanation_2.top_labels[0],
positive_only=True,
num_features=20,
hide_rest=False)
plt.imshow(mark_boundaries(temp, mask))
Conclusion
In this article, I used the Kaggle BCHI dataset [5] to show how to use the LIME image explainer [3] to explain the IDC image prediction results of a 2D ConvNet model in IDC breast cancer diagnosis. Explanations of model prediction of both IDC and non-IDC were provided by setting the number of super-pixels/features (i.e., the num_features parameter in the method get_image_and_mask()) to 20.
I observed that the explanation results are sensitive to the choice of the number of super pixels/features. Domain knowledge is required to adjust this parameter to achieve appropriate model prediction explanation. Quality of the input data (images in this case) is also very important for a reasonable result. Accuracy can be improved by adding more samples.
A Jupyter notebook with all the source code used in this article is available in Github [6].
References
[1] M. T. Ribeiro, S. Singh, and C. Guestrin, “Why Should I Trust You?” Explaining the Predictions of Any Classifier
[2] Y. Huang, Explainable Machine Learning for Healthcare
[3] LIME tutorial on image classification
[4] Interpretable Machine Learning, A Guide for Making Black Box Models Explainable
[5] Predicting IDC in Breast Cancer Histology Images
[6] Y. Huang, Jupyter notebook
DISCLOSURE STATEMENT: © 2020. Opinions expressed in this article are those of the author and do not necessarily represent those of Argonne National Laboratory.
