The world’s leading publication for data science, AI, and ML professionals.

Land Cover Image Classification Using a TensorFlow CNN in Python

EuroSAT land cover image classification using a TensorFlow convolutional neural network


Introduction

Land cover is the detected bio-physical overlay on the Earth’s surface, including materials like grass, forest, pastures, and water. Various methods exist for assimilating land cover information. A primary process involves using remotely sensed imagery in combination with field surveys.

Photo by NASA on Unsplash
Photo by NASA on Unsplash

Dataset

EuroSAT dataset is composed of tiles from Sentinel-2 satellite images.² The dataset is made publicly available to download.

Figure 1 shows fifteen randomly selected samples from the collection, depicting eight of the ten possible categories. The two remaining unseen classes are HerbaceousVegetation and River.

Figure 1 - EuroSAT Sample Training Data (Image By Author)
Figure 1 – EuroSAT Sample Training Data (Image By Author)

From Figure 1, there are apparent visual differences between each land cover category. Therefore, a reasonable a priori assumption expects potential confusion between the Forest, Pasture and AnnualCrop categories. Also, Highway and River images can tend to be similar. In general, however, there appear to be suitable distinguishing attributes for a neural network to learn the appropriate feature mappings sufficiently.

Figure 2 shows the count distribution of each of the _ten land cover classe_s. In total, there are 27,000 labelled and geo-referenced images.

Geo-referencing means that the coordinate system of an aerial photo is related to a ground system of geographic coordinates. Therefore, a Geographic Information System (GIS) can ‘place’ the image in its appropriate real-world location.³

Figure 2— Training Data Count Distribution (Image By Author)
Figure 2— Training Data Count Distribution (Image By Author)

Machine learning tasks often utilise three data subsets: train, test, and validation. With the train/validation/test percentage splits varying depending on the size of the available dataset:

  • Training (60%): labelled data used for carrying-out gradient descent to tune the parameters of the CNN through the backpropagation algorithm
  • Validation (20%): labelled data used to determine the model performance metrics after each training cycle
  • Testing (20%): unseen labelled data used to assess model efficacy

Gist 1 shows the Keras utility function image_dataset_from_directory, which loads the image files provided the directory structure matches the documentation.

After loading the images, inspect the data and determine useful variables. For example, find the names and number of **** classes using:

class_names = train_dataset.class_names
num_classes = len(class_names)

The Keras function used above loads the data in batches. Retrieve a single volume from the BatchDataset and examine the input and output shapes along with the images value range as shown in Gist 2.

As specified in GIst 2, the batch size = 32. Therefore, the input image_batch is a (32, 64, 64, 3) tensor, where the images are 64 (height) x 64 (width) x 3 (RGB channels). Each image has a corresponding integer label for class identification, and so the label_batch is shape is (32).


Preprocessing

Data preprocessing refers to preparing the raw data for building and training machine learning models. Later a Convolutional Neural Network (CNN) model performs the EuroSAT land cover classification.

The input images are tensors interpretable as NumPy arrays of 8-bit unsigned integers (0 through 255 decimal). Therefore, rescaling the values from 0–255 to 0–1 **** improves performance and training stability.

Keras offers a [Rescaling](https://www.tensorflow.org/api_docs/python/tf/keras/layers/Rescaling) preprocessing layer, which modifies the input values to a new range. Define this layer using the Python code below.

rescaling_layer = layers.experimental.preprocessing.Rescaling(
    scale=1. / 255,
    input_shape=(img_height, img_width, 3)
)

Every value of the input image is multiplied by scale.


Model Architecture

With the preprocessing complete and data in an acceptable format for CNN training, the next step is to define the network architecture. Figure 3 graphically illustrates the six-layer model architecture.

When calculating the depth of a CNN network, we only consider the layers that have tunable/trainable weights/parameters.⁵

Figure 3 - Convolution Neural Network Architecture for Land Cover Classification (Image By Author)
Figure 3 – Convolution Neural Network Architecture for Land Cover Classification (Image By Author)

Basic Tensorflow CNN models commonly employ the same elements. These layers include:

  • [Conv2D](https://www.tensorflow.org/api_docs/python/tf/keras/layers/Conv2D) layer slides kernels, e.g. eight filters with size (3×3), across the input image at a given stride to extract features suitable for solving the classification problem. Feature extraction occurs through the convolution operation, or more accurately, cross-correlation. Filters are initialised randomly during model compilation. The filter parameters are learned through backpropagation to extract the essential input image features when training the network.

  • [MaxPooling2D](https://keras.io/api/layers/pooling_layers/max_pooling2d/) layer operates on groups of pixels, e.g. (2×2) grids, __ and filters values by selecting the maximum. Pooling _downsample_s the input image size while keeping feature information. _Paddin_g can maintain or slow the reduction of the spatial dimensions; however, "valid" means no padding.
  • [Dense](https://keras.io/api/layers/core_layers/dense/): "Just your regular densely-connected NN layer".

Gist 3 gives the Python code defining the Tensorflow Sequential CNN model.

The model compilation model.compile()involves defining:

  • optimiser='adam': Adam is a stochastic gradient descent (SGD) optimisation method, which essentially updates the trainable parameters based on the backpropagated outputs of the loss function.
  • loss='**[SparseCategoricalCrossentropy](https://www.tensorflow.org/api_docs/python/tf/keras/losses/SparseCategoricalCrossentropy)‘: Computes the cross-entropy loss** between the labels and predictions. SparseCategoricalCrossentropy loss applies as the label classes count ≥ two, provided as integers. As the model does not contain a softmax layer as the last layer, from_logits=True is necessary to indicate that the probabilities are not normalised.⁴
  • metrics=[‘accuracy‘]: Calculates how often predictions equal labels and is used to evaluate model performance during training.

See this article to determine how to derive the binary (0 or 1) cross-entropy loss function for classification applications.

Derivation of the Binary Cross-Entropy Classification Loss Function

Figure 4 is the output from model.summary(). The layers, output shapes and parameter counts are equivalent to those presented in Figure 3.

Figure 4 - CNN Model Summary (Image By Author)
Figure 4 – CNN Model Summary (Image By Author)

Training

Training can commence using the Python code below as the train/validation data is in the TensorFlow BatchDatasets structure and the model architecture defined.

# number of training cycles
epochs = 15
# trains the model for 15 iterations on the dataset
history = model.fit(
    train_dataset,
    validation_data=validation_dataset,
    epochs=epochs
)
# save model
model.save(f'{os.getcwd()}/land_cover_model')

Many network hyperparameters, such as the learning rate, number of epochs, batch size, and the number of hidden layers and units, are tuned through trial and error. Experiment with different hyperparameters to try and improve classification performance.

Figure 5 is a snippet of the console log output during fitting. It displays the loss for both the training and validation datasets.

Figure 5 - Verbose Output from Model Fitting (Image By Author)
Figure 5 – Verbose Output from Model Fitting (Image By Author)

Figure 6 plots the training metrics of interest, the training and validation accuracies, and the training and validation losses. Over time the network learns the suitable Conv2D filter weights through backpropagation to minimise the loss, and as the loss decreases, the model’s accuracy increases.

The final training and validation accuracies are approximately 90% and 86%, respectively.

Figure 6 - Plot of Training and Validation Metrics (Image By Author)
Figure 6 – Plot of Training and Validation Metrics (Image By Author)

Prediction

After training the land cover classification network, further testing is often valuable. As mentioned earlier, a saved portion of the dataset (around 20%) remains for additional evaluation on unseen data.

Firstly the dataset is loaded using the same code as in Figure 3 except with the updated path variable pointing to the test folder. Load pre-trained Keras models from disk using the following commands:

# load keras model and print summary
model = keras.models.load_model('path/to/model')
model.summary()

With the model loaded into memory, it is possible to make predictions. Gist 4 details Python code for iterating over the BatchDataset of test images and making a prediction.

For each inference model.predict() returns an array of probabilities of class likelihood associated with each class. The highest probability index is the model’s integer label prediction, and argmax() extracts this position.

Displayed in Figure 7 are predictions from the CNN.

Figure 7 - CNN Land Cover Class Predictions (Image By Author)
Figure 7 – CNN Land Cover Class Predictions (Image By Author)

Evaluation

A confusion matrix is one metric available that evaluates classification accuracy. With a list of the model predicted classes, y_pred and a list of the actual categories, the y_true, SciKit-Learn Python library has a built-in function to compute the matrix quickly.

Figure 8 is the confusion matrix generated from predictions on the test dataset.

Figure 8 - Land Cover Confusion Matrix (Image By Author)
Figure 8 – Land Cover Confusion Matrix (Image By Author)

Each row of the matrix denotes the actual classes, while each column describes the predicted types.

  • True Positive (TP): identifies a correctly predicted class. TPs lie across the matrix diagonal at the actual and predicted values intersection.
  • True Negative (TN): specifies rightly predicted negative classes. TNs are the sum of values of all columns and rows, excluding the current class.
  • False Positive (FP): a mistakenly predicted positive class, the sum of the current column minus the TP value
  • False Negative (FN): an incorrectly predicted negative class, the corresponding class row summation besides the TP value

Taking the River class as an example, Figure 9 depicts the metrics breakdown: TP = 15, TN = 350, FP = 5 and FN = 10.

Figure 9 - Confusion Matrix Breakdown for River Class (Image By Author)
Figure 9 – Confusion Matrix Breakdown for River Class (Image By Author)

Initially, foreseen was confusion between the Highway and River groups. According to the confusion matrix, the model incorrectly labelled five rivers as highways (false negatives). Similarly, two highways are incorrectly classified as rivers (false positives).

Overall, the prediction accuracy is around 86%, as exhibited in Figure 8 and is around the anticipated value determined during training. Accuracy improvements are certainly possible. The EuroSAT paper⁷ reports performance metrics values into the high nineties range. Experiment with the CNN network architecture and hyperparameter tuning to enhance results.


Conclusion

This article has demonstrated implementing a convolutional neural network in Python for satellite imagery land cover classification.

Find all Python code used below. Thanks for reading.

Join Medium with my referral link – Andrew Joseph Davies


Python Code for CNN Training (Gist 5)

Python Code for CNN Predictions (Gist 6)


References

[1] Eurosat: A novel dataset and deep learning benchmark for land use and land cover classification. Patrick Helber, Benjamin Bischke, Andreas Dengel, Damian Borth. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 2019 [2] Eurosat | Tensorflow Dataset (February 2022) [3] Georeferencing – Teaching with GeoPads [4] What does from_logits=True do in SparseCategoricalcrossEntropy loss function?Data Science Stack Exchange (April 2020) [5] Counting CNN Layers. Ranmal Dewage – StackOverflow [6] EuroSAT Publicly Available Dataset – GitHub.com [7] Introducing EuroSAT: A Novel Dataset and Deep Learning Benchmark for Land Use and Land Cover Classification. Patrick Helber, Benjamin Bischke, Andreas Dengel. 2018 IEEE International Geoscience and Remote Sensing Symposium, 2018.


Related Articles