Medical Image Segmentation [Part 2] — Semantic Segmentation of Pathological Lung Tissue with Dilated Fully Convolutional Networks with Interactive Code

Gif from this website

So this is my second post on segmentation on Medical images. Today we will cover this paper “Semantic Segmentation of Pathological Lung Tissue with Dilated Fully Convolutional Networks”. And I feel bit lazy today again, so I’ll just use auto differentiation to train my network.

Network Architecture (Basic Building Block / Full Architecture)

Image from paper

So above image shows the basic building block of this network, it is composed of convolution operation, normalization, addition and activation function (ReLU). Now since we have saw the basic building block, lets take a look at the full network architecture.

The paper really does a great job describing the full architecture, the network has 13 layers in total with variety of combination of concatenation as well as dropout layers.

Experimental Set Up / Difference from Paper

Right Image → Original Image
Middle Image → Ground Truth Binary Mask
Right Image → Ground Truth Mask Overlay on top of Original Image

There are couple of differences from the original paper and those differences are…..
1. Data Set: I am going to use the segmentation data from Kaggle “Ultrasound Nerve Segmentation”

2. Normalization Layer / Batch size: The original paper used Instance Normalization Layer, however as seen below I will use batch normalization. Also, the original paper used a batch size of 1, I will use a batch size of 2.

3. Different number of Feature Map: The original paper reported to use 32 feature maps for each layer, due to hardware limitations I will only use 3 feature maps. ( So in the concatenation operation we would have 1 + 3 * 10 = 31 feature maps)

Result (Original Network)

Even if I trained the original network with variety of hyper parameters I wasn’t able to get any good results. The generated mask was just an pitch black image. So I decided to make three changes to the network.

Modification of Original Network

Red Line → Added Residual Connection on each layer

The first change that I made was adding an additional residual connection on each layer, for more fluent information flow.

Green Line → Changed Drop Out rate

The second thing I changed was the drop out rate, I decided not to use the drop out layer at all.

Purple Line → Changed Cost Function and direct Connection

The final change was directly linking the final layer to the cost function as well as changing the cost function to Mean Square Error function. ( The paper used the softmax with cross entropy loss function.

Results (Modified Network)

Right Image → Original Image
Middle Image → Ground Truth of Binary Mask
Left Image → Generated Binary Mask from Network

The network did an okay job segmenting the region of interest. However, one thing I noticed was the square looking artifacts that was presented in the generated mask. This phenomena sometimes called the Checkerboard artifacts are well know to happen even when using the deconvolution operation and it is explained in this post. Below are some images when over laying the ground truth mask or generated mask to the original image.

Results in GIF

Order of Presented Images → 1. Original Image → 2. Ground Truth Binary Mask → 3. Generated Binary Mask → 4. Ground Truth Mask overlay on Original Image → 5. Generated Mask overlay on Original Image.

Above is a GIF that I made from resulted segmentation, please take note of the order when viewing the GIF, and below is compilation of how the network did overtime. As training continuous (seen by epoch) we can see that the generated mask becomes more precise, however we can see that the checker board artifacts are always present.

Interactive Code / Transparency

For Google Colab, you would need a google account to view the codes, also you can’t run read only scripts in Google Colab so make a copy on your play ground. Finally, I will never ask for permission to access your files on Google Drive, just FYI. Happy Coding!

To access the code for original model please click here.
To access the code for modified model please click here.

• * NOTE **: I did not want to host Kaggles Data on my github, since I might break their data usage policy. So this code cannot be run directly online.
• In my hopes to make this experiment more transparent, I uploaded all of my command output to my github, if you wish to view it please click here.

Final Words

The original network is much more complex what I implemented. It even have 4D SoftMax layer. (Click here if you wish to check out 4D Soft Max Layer.). They really did an amazing job segmenting lung regions.

If any errors are found, please email me at jae.duk.seo@gmail.com, if you wish to see the list of all of my writing please view my website here.

Meanwhile follow me on my twitter here, and visit my website, or my Youtube channel for more content. I also did comparison of Decoupled Neural Network here if you are interested.

Reference

1. intact-project/LungNet. (2018). GitHub. Retrieved 3 April 2018, from https://github.com/intact-project/LungNet
2. Anthimopoulos, M., Christodoulidis, S., Ebner, L., Geiser, T., Christe, A., & Mougiakakou, S. (2018). Semantic Segmentation of Pathological Lung Tissue with Dilated Fully Convolutional Networks. arXiv preprint arXiv:1803.06167.
3. Ultrasound Nerve Segmentation | Kaggle. (2018). Kaggle.com. Retrieved 3 April 2018, from https://www.kaggle.com/c/ultrasound-nerve-segmentation
4. Ioffe, S., & Szegedy, C. (2015). Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift. Arxiv.org. Retrieved 3 April 2018, from https://arxiv.org/abs/1502.03167
5. Ulyanov, D., Vedaldi, A., & Lempitsky, V. (2016). Instance Normalization: The Missing Ingredient for Fast Stylization. Arxiv.org. Retrieved 3 April 2018, from https://arxiv.org/abs/1607.08022
6. Mean squared error. (2018). En.wikipedia.org. Retrieved 3 April 2018, from https://en.wikipedia.org/wiki/Mean_squared_error
7. Odena, A., Dumoulin, V., & Olah, C. (2016). Deconvolution and Checkerboard Artifacts. Distill, 1(10). doi:10.23915/distill.00003
8. Roelants, P. (2018). Peter’s Notes. Peterroelants.github.io. Retrieved 3 April 2018, from http://peterroelants.github.io/posts/neural_network_implementation_intermezzo02/