Computer Vision Feature Extraction 101 on Medical Images — Part 1: Edge Detection / Sharpening / Blurring / Emboss / Super Pixel

Gif from this website

So today, I just wanted to review some of the core concepts in computer vision, and I wish to focus on the application rather than theory. If you are new to computer vision I strongly recommend watching this video series that I linked below to get the theory. (The video series is long, but it is really good.)

Also, here is the list of publicly available data that we are going to use.
1. Breast Cancer Cell from UCSB Bio-Segmentation Benchmark data set
2. Cellular 2D from UCSB Bio-Segmentation Benchmark data set
3. DRIVE: Digital Retinal Images for Vessel Extraction
4. Ultrasound Nerve Segmentation from kaggle Data set
5. Brain MRI from pixabay
6. Brain MRI from pixabay
7. Natural Image from pixabay

Please note, this post is for me to review some of the fundamental concepts in computer vision.

2D Convolution

Image from this website

“convolution is a mathematical operation on two functions (f and g) to produce a third function, that is typically viewed as a modified version of one of the original functions, giving the integral of the pointwise multiplication of the two functions as a function of the amount that one of the original functions is translated” — Wiki Page

2D convolution operation is in the heart of computer vision, and since this is the main operation that we are going to use please make sure you understand the concept. If you need help please click here to check out a step by step example of 2D Convolution operation by Song Ho Ahn.

Original Images

Top Left → Breast Cancer Cell from UCSB Bio-Segmentation Benchmark
Top Middle → Cellular 2D from UCSB Bio-Segmentation Benchmark data set
Top Right → DRIVE: Digital Retinal Images for Vessel Extraction
Bottom Left → Ultrasound Nerve Segmentation from kaggle Data set
Bottom Middle → Brain MRI from pixabay
Bottom Right → Brain MRI from pixabay

Before we do anything to these images, lets actually take a look at what each image looks like, finally below is how our natural image look like.

Identity Kernel

Lets start off simple, identity kernel does not change the given image, since it is the identity kernel. And as seen below, the images look exactly the same, except for the fact that all images are now in black and white.

Edge Detection (Horizontal)

Now lets see how much change there is in the Horizontal direction of each images. One very interesting observation that struck me was the natural image, we can see that the light from the lighting beam is nowhere to be found.

Edge Detection (Vertical)

Now lets see the changes that were made in the vertical direction for each images. We can observe that the light beam have returned in the natural image.

Edge Detection (Gradient Magnitude)

Since we now have change in the horizontal direction as well as vertical direction we can take the square root of of summed of square value to get the gradient magnitude. Please see below screen shot for the equation from this wiki page.

Image from this website

Edge Detection (Gradient Direction)

Since we now have change in the horizontal direction as well as vertical direction, we can obtain the gradient direction by using the arc-tangent function. Please see below for the equations.

Image from this website

And when we visualize the raw results, we get some thing like below. Very strange looking images….

Sobel Filters ( Gradient Magnitude)

Now lets take a sobel filter, and get the gradient magnitude. I personally, can’t see a huge difference from the filters that we have used.

Sobel Filters ( Gradient Direction )

Again, for gradient directions I wasn’t able to see a huge difference between the filters that we used and the sobel filter.

Gaussian Blur

We also can blur the images, by using Gussian filter, if we visualize this filter it would look something like below. This operation is usually called Gaussian blur.

Image from this website

Sharpening the Edges

We also can sharpen the image, by first finding the edges of the image, and adding that value to the original image. This will sharpen the edges of the image.

“ To sharpen the image is very similar to finding edges, add the original image, and the image after the edge detection to each other, and the result will be a new image where the edges are enhanced, making it look sharper.” — Lode’s Computer Graphics Tutorial

Emboss

To give the images a shadow effect we also can use a emboss filter and add a bias of 128. I actually didn’t know about this filter, until I saw it from Lode’s Computer Graphics Tutorial. The results look very similar to edge detection in horizontal direction.

“ An emboss filter gives a 3D shadow effect to the image, the result is very useful for a bumpmap of the image. It can be achieved by taking a pixel on one side of the center, and subtracting one of the other side from it. Pixels can get either a positive or a negative result. To use the negative pixels as shadow and positive ones as light, for a bumpmap, a bias of 128 is added to the image.” — Lode’s Computer Graphics Tutorial.

Super Pixel

Finally, for fun lets make super pixels using the Simple Linear Iterative Clustering that is implemented in skimage library.

Interactive Code

For Google Colab, you would need a google account to view the codes, also you can’t run read only scripts in Google Colab so make a copy on your play ground. Finally, I will never ask for permission to access your files on Google Drive, just FYI. Happy Coding!

Please note, I didn’t wanted to abuse any data policy. So for Google Colab I found the below image, which is labeled for reuse. To access the online code please click here.

Image from this website

Final Words

I wanted to make this post for long period of time, feels good to know that I finally did it. I hope to make another post about HOG features or SIFT features soon.

If any errors are found, please email me at jae.duk.seo@gmail.com, if you wish to see the list of all of my writing please view my website here.

Meanwhile follow me on my twitter here, and visit my website, or my Youtube channel for more content. I also did comparison of Decoupled Neural Network here if you are interested.

Reference

1. Taking over for Aaron. (2018). YouTube. Retrieved 9 April 2018, from https://www.youtube.com/watch?v=2S4nn7S8Hk4&list=PLtizWl5sTV3d4uQ6PvzXKrlkp_3XOCotN
2. Ahn, S. (2018). Example of 2D Convolution. Songho.ca. Retrieved 9 April 2018, from http://www.songho.ca/dsp/convolution/convolution2d_example.html
3. Kernel (image processing). (2018). En.wikipedia.org. Retrieved 9 April 2018, from https://en.wikipedia.org/wiki/Kernel_(image_processing)
4. Image embossing. (2018). En.wikipedia.org. Retrieved 9 April 2018, from https://en.wikipedia.org/wiki/Image_embossing
5. Rosebrock, A. (2014). Accessing Individual Superpixel Segmentations with Python. PyImageSearch. Retrieved 9 April 2018, from https://www.pyimagesearch.com/2014/12/29/accessing-individual-superpixel-segmentations-python/
6. Bio-Segmentation | Center for Bio-Image Informatics | UC Santa Barbara. (2018). Bioimage.ucsb.edu. Retrieved 9 April 2018, from https://bioimage.ucsb.edu/research/bio-segmentation
7. DRIVE: Download. (2018). Isi.uu.nl. Retrieved 9 April 2018, from http://www.isi.uu.nl/Research/Databases/DRIVE/download.php
8. Ultrasound Nerve Segmentation | Kaggle. (2018). Kaggle.com. Retrieved 9 April 2018, from https://www.kaggle.com/c/ultrasound-nerve-segmentation
9. Free Image on Pixabay — Mri, Magnetic, X Ray, Skull, Head. (2018). Pixabay.com. Retrieved 9 April 2018, from https://pixabay.com/en/mri-magnetic-x-ray-skull-head-782457/
10. Free Image on Pixabay — Sunset, Lighthouse, Dawn, Dusk, Sun. (2018). Pixabay.com. Retrieved 9 April 2018, from https://pixabay.com/en/sunset-lighthouse-dawn-dusk-sun-3120484/
11. Free Image on Pixabay — Mri, Magnetic, X Ray, Skull, Head. (2018). Pixabay.com. Retrieved 9 April 2018, from https://pixabay.com/en/mri-magnetic-x-ray-skull-head-782459/
12. DICOM in Python: Importing medical image data into NumPy with PyDICOM and VTK. (2014). PyScience. Retrieved 9 April 2018, from https://pyscience.wordpress.com/2014/09/08/dicom-in-python-importing-medical-image-data-into-numpy-with-pydicom-and-vtk/
13. Matplotlib plots: removing axis, l. (2018). Matplotlib plots: removing axis, legends and white spaces. Stackoverflow.com. Retrieved 9 April 2018, from https://stackoverflow.com/questions/9295026/matplotlib-plots-removing-axis-legends-and-white-spaces
14. scipy.signal.convolve2d — SciPy v1.0.0 Reference Guide. (2018). Docs.scipy.org. Retrieved 9 April 2018, from https://docs.scipy.org/doc/scipy/reference/generated/scipy.signal.convolve2d.html
15. matplotlib, R. (2018). Removing white space around a saved image in matplotlib. Stackoverflow.com. Retrieved 9 April 2018, from https://stackoverflow.com/questions/11837979/removing-white-space-around-a-saved-image-in-matplotlib
16. numpy.arctan — NumPy v1.14 Manual. (2018). Docs.scipy.org. Retrieved 9 April 2018, from https://docs.scipy.org/doc/numpy/reference/generated/numpy.arctan.html
17. Spatial Filters — Gaussian Smoothing. (2018). Homepages.inf.ed.ac.uk. Retrieved 9 April 2018, from https://homepages.inf.ed.ac.uk/rbf/HIPR2/gsmooth.htm
18. scipy.ndimage.filters.gaussian_filter — SciPy v0.15.1 Reference Guide. (2018). Docs.scipy.org. Retrieved 9 April 2018, from https://docs.scipy.org/doc/scipy-0.15.1/reference/generated/scipy.ndimage.filters.gaussian_filter.html
19. Gaussian blur. (2018). En.wikipedia.org. Retrieved 9 April 2018, from https://en.wikipedia.org/wiki/Gaussian_blur
20. Module: segmentation — skimage v0.14dev docs. (2018). Scikit-image.org. Retrieved 9 April 2018, from http://scikit-image.org/docs/dev/api/skimage.segmentation.html#skimage.segmentation.slic