Ultra TinyML: Machine Learning for 8-bit Microcontroller

How to create a gesture recognition system with Arduino and Neuton TinyML

Leonardo Cavagnis

Published in

Towards Data Science

11 min readFeb 21, 2022

“A mistake repeated more than once is a decision” — P. Coelho

TinyML is a sub-field of Machine Learning that studies the way to run ML models on small and low-powered devices.

In this article, I will show an easy way to get started with TinyML: implementing a Machine Learning model on an Arduino board while creating something cool: a gesture recognition system based on an accelerometer.

Gesture recognition is a process that attempts to recognize human gestures through the use of mathematical algorithms.

To make the experiment simpler, the system is designed to recognize only two gestures: a punch and a flex movement.
This, in the data science field, is called binary classification.

But… Why “Ultra” TinyML?

The biggest challenge of this experiment is trying to run the prediction model on a very tiny device: an 8-bit microcontroller.
To achieve this, you can use Neuton.

Neuton is a TinyML framework. It allows to automatically build neural networks without any coding and with little machine learning experience and embed them into small compute devices.
It supports 8, 16, and 32-bit microcontrollers, unlike TensorFlow Lite TinyML framework which only supports 32-bit. You don’t need a powerful machine to use Neuton, it is an online software tool which runs on a web browser.
Using the free plan, you can train and download an unlimited number of models.

…Let’s start!

The experiment is divided in three steps:

Capture the training dataset
Train the model using Neuton
Deploy and run the model on Arduino

Experiment process flow. Image by author.

The hardware system

The gesture recognition system is composed by:

A microcontroller: Arduino Mega 2560
An accelerometer sensor: GY-521 Module

Arduino Mega 2560 is a microcontroller board based on the ATmega2560: a low-power 8-bit MCU with 256KB flash memory, 32 general purpose working registers, UART interfaces, 10-bit A/D converter, and many other peripherals.
Mega 2560 board is designed for complex projects: it has a larger space than other Arduino boards (Uno, Nano, Micro, etc.). This makes it ideal for machine learning applications where there are large amounts of data to process.

The GY-521 module is built around the InvenSense MPU-6050: a sensor containing, in a single IC, a 3-axis MEMS accelerometer and 3-axis MEMS gyroscope.
Its operation is very precise since it contains for each channel an accurate digital converter. It is able to capture the values of the X, Y and Z axes at the same time. Communication with MCU takes place using the I2C interface.

Below, the connection circuit designed with Fritzing software:

Connection circuit: Arduino Mega2560 and GY-521. Image by author.

GY-521 is powered with the 5V and GND pins of Arduino Mega power section, while for data communication the I2C pins are used (Pin 20 and Pin 21).
The remaining pins are optional and not useful for this application.

To verify if the GY-521 module is correctly supplied, connect the USB cable of the Arduino board and check if the LED mounted on the sensor board is turning on.

After verifying the sensor power supply, check if the I2C communication is working properly by downloading the Adafruit MPU6050 Arduino library and opening the “plotter” example.

Add a library in Arduino IDE. Image by author.

Adafruit MPU6050 Library. Image by author.

Upload the example sketch on the Arduino board, open the “Serial Plotter” within the Tools menu, set 115200 in the baud drop-down menu and “shake” the sensor board. The expected result will be the following:

Serial Plotter of MPU6050 plotter example. Image by author.

Now, the system is ready to collect accelerometer and gyroscope data.

Capture training data

The first step to build the predictive model is to collect enough motion measurements.
This set of measures is called training dataset and it will be used to train the Neuton neural network builder.

The easiest way to achieve this is to repeat several times the same two motions (punch and flex), by capturing acceleration and gyroscope measurements and storing the result in a file.
To do this, you create an Arduino sketch dedicated to sensor data acquisition. The program will acquire the measurements of each motion and will print the sensor measurements output on the serial port console.

You will perform at least 60 motions: 30 for the first movement (punch) and 30 for the second one (flex). For each motion, you will acquire 50 acceleration and 50 gyroscope measures in a 1 second time window (Sampling time: 20ms —50Hz). In this experiment, 60 motions are enough. By increasing the number of motion measurements, you can improve the predictive power of the model. However, a large dataset can lead to an over-fitted model. There is no “correct” dataset size, but a “trial and error” approach is recommended.

The serial port output of the Arduino sketch will be formatted according to Neuton training dataset requirements:

CSV format: it is a database file format. Each line of the file is a data record consisting of one or more fields, separated by commas.
At least 50 data records (for binary classification task means at least 25 data records per group).
First row must contain the column names (e.g., ax0, ay0, az0, gx0, gy0, gz0, …).
A target variable column must be present. Each row must be assigned to a specific target group (in this experiment: ‘0’ for punch, ‘1’ for flex).
Use a dot as a decimal separator.

Below, Arduino program for dataset creation:

IMU Sensor initialization and CSV header generation:

#define NUM_SAMPLES 50Adafruit_MPU6050 mpu;void setup() {
  // init serial port
  Serial.begin(115200);
  while (!Serial) {
    delay(10);
  }  // init IMU sensor
  if (!mpu.begin()) {
    while (1) {
      delay(10);
    }
  }
  
  // configure IMU sensor
  // [...]  // print the CSV header (ax0,ay0,az0,...,gx49,gy49,gz49,target)
  for (int i=0; i<NUM_SAMPLES; i++) {
    Serial.print("aX");
    Serial.print(i);
    Serial.print(",aY");
    Serial.print(i);
    Serial.print(",aZ");
    Serial.print(i);
    Serial.print(",gX");
    Serial.print(i);
    Serial.print(",gY");
    Serial.print(i);
    Serial.print(",gZ");
    Serial.print(i);
    Serial.print(",");
  }
  Serial.println("target");
}

Acquisition of 30 consecutive motions. The start of a motion is detected if the acceleration sum is above a certain threshold (e.g., 2.5 G).

#define NUM_GESTURES    30#define GESTURE_0       0
#define GESTURE_1       1
#define GESTURE_TARGET  GESTURE_0 
//#define GESTURE_TARGET  GESTURE_1 void loop() {
  sensors_event_t a, g, temp;
  
  while(gesturesRead < NUM_GESTURES) {
    // wait for significant motion
    while (samplesRead == NUM_SAMPLES) {
      // read the acceleration data
      mpu.getEvent(&a, &g, &temp);
      
      // sum up the absolutes
      float aSum = fabs(a.acceleration.x) + 
                   fabs(a.acceleration.y) + 
                   fabs(a.acceleration.z);
      
      // check if it's above the threshold
      if (aSum >= ACC_THRESHOLD) {
        // reset the sample read count
        samplesRead = 0;
        break;
      }
    }
  
    // read samples of the detected motion
    while (samplesRead < NUM_SAMPLES) {
        // read the acceleration and gyroscope data
        mpu.getEvent(&a, &g, &temp);
  
        samplesRead++;
  
        // print the sensor data in CSV format
        Serial.print(a.acceleration.x, 3);
        Serial.print(',');
        Serial.print(a.acceleration.y, 3);
        Serial.print(',');
        Serial.print(a.acceleration.z, 3);
        Serial.print(',');
        Serial.print(g.gyro.x, 3);
        Serial.print(',');
        Serial.print(g.gyro.y, 3);
        Serial.print(',');
        Serial.print(g.gyro.z, 3);
        Serial.print(',');        // print target at the end of samples acquisition
        if (samplesRead == NUM_SAMPLES) {
          Serial.println(GESTURE_TARGET);
        }
        
        delay(10);
    }
    gesturesRead++;
  }
}

Firstly, run the above sketch with the serial monitor opened and GESTURE_TARGET set to GESTURE_0. Then, run with GESTURE_TARGET set to GESTURE_1. For each execution, perform the same motion 30 times, ensuring, as far as possible, that the motion is performed in the same way.

Copy the serial monitor output of the two motions in a text file and rename it to “trainingdata.csv”.

Example of training dataset in CSV format. Image by author.

Train the model with Neuton TinyML

The process of training a model involves providing a Machine Learning algorithm with training data to learn from. It is the phase where you try to fit the best combination of weights and bias to a ML algorithm to minimize a loss function.

Neuton performs training automatically and without any user interaction.
Train a Neural Network with Neuton is quick and easy and is divided into three phases:

Dataset: Upload and validation
Training: Auto ML
Prediction: Result analysis and model download

Dataset: Upload and validation

First, create a new Neuton solution and name it (e.g., Gesture Recognition).

Neuton: add new solution. Image by author.

Upload CSV training dataset file.

Neuton: upload CSV file. Image by author.

Neuton validates CSV file according to dataset requirements.

Neuton: dataset validation. Image by author.

If the CSV file meets the requirements a green check will appear, otherwise an error message will be shown.

Neuton: validated dataset. Image by author.

Select the column name of the target variable (e.g., target) and click “Next”.

Neuton: target variable. Image by author.

Neuton: dataset content preview. Image by author.

Training: Auto ML

Now, let’s get to the heart of training!

Neuton analyzes the content of the training dataset and defines the ML task type. With this dataset, the binary classification task is automatically detected.

Metric is used to monitor and measure the performance of a model during training. For this experiment, you use the Accuracy metric: it represents how accurately class is predicted. The higher the value, the better the model.

Enable TinyML option to allow Neuton to build a tiny model for microcontroller.

In the TinyML settings page, select “8-bit” in the drop-down menu and enable “Float datatype support” option. This, because the microcontroller used in the experiment is an 8-bit MCU with floating point number support.

Neuton: TinyML settings. Image by author.

After pressing the “Start Training” button, you will see the process progress bar and the percentage of completion.

Neuton: Training started. Image by author.

The first step is the data preprocessing. It is the process of preparing (cleaning, organizing, transforming, etc.) the raw dataset to make it suitable for training and building ML models.
After data preprocessing completion, model training starts. Process can take a long time; you can close the window and come back when the process is finished. During training you can monitor the real-time model performance by observing model status (“consistent” or “not consistent”) and Target metric value.

Neuton: Data preprocessing complete. Image by author.

Upon training is complete, the “Status” will change to “Training completed”. Model is consistent and has reached the best predictive power.

Neuton: Training completed. Image by author.

Prediction: Result analysis and model download

That’s all… Model is ready!

After the training procedure is completed, you will be redirected to the «Prediction» section.
In this experiment, the model has reached an accuracy of 98%. It means that from 100 predicted records, 98 had been assigned to the correct class… that’s impressive!

Moreover, the size of the model to embed is less than 3KB.
This is a very small size, considering that the Arduino board in use is 256KB memory size and a typical memory size for an 8-bit microcontroller is 64KB÷256KB.

To download the model archive, click on the “Download” button.

Neuton: Prediction tab. Image by author.

Deploy the model on Arduino

Now, it’s time to embed the resulting model into the microcontroller.

The model archive downloaded from Neuton includes the following files and folders:

/model: the neural network model in a compact form (HEX and binary).
/neuton: a set of functions used to perform predictions, calculation, data transferring, result management, etc.
user_app.c: a file in which you can set the logic of your application to manage the predictions.

First, you modify the user_app.c file adding functions to initialize model and run inference.

/*
 * Function: model_init
 * ----------------------------
 *
 *    returns: result of initialization (bool)
 */
uint8_t model_init() {
   uint8_t res;
 
    res = CalculatorInit(&neuralNet, NULL);
 
    return (ERR_NO_ERROR == res);
}/*
 * Function: model_run_inference
 * ----------------------------
 *
 *   sample: input array to make prediction
 *   size_in: size of input array
 *   size_out: size of result array
 *
 *   returns: result of prediction
 */
float* model_run_inference(float* sample, 
                           uint32_t size_in, 
                           uint32_t *size_out) {
   if (!sample || !size_out)
      return NULL;   if (size_in != neuralNet.inputsDim)
      return NULL;   *size_out = neuralNet.outputsDim;   return CalculatorRunInference(&neuralNet, sample);
}

After that, you create the user_app.h header file to allow the main application using the user functions.

uint8_t model_init();
float*  model_run_inference(float* sample, 
                            uint32_t size_in, 
                            uint32_t* size_out);

Below, the Arduino sketch of main application:

Model initialization

#include "src/Gesture Recognition_v1/user_app.h"void setup() {
   // init serial port and IMU sensor
   // [...]   // init Neuton neural network model
   if (!model_init()) {
      Serial.print("Failed to initialize Neuton model!");
      while (1) {
        delay(10);
      }
   }
}

Model inference

#define GESTURE_ARRAY_SIZE  (6*NUM_SAMPLES+1) void loop() {
   sensors_event_t a, g, temp;
   float gestureArray[GESTURE_ARRAY_SIZE]  = {0};   // wait for significant motion
   // [...]   // read samples of the detected motion
   while (samplesRead < NUM_SAMPLES) {
      // read the acceleration and gyroscope data
      mpu.getEvent(&a, &g, &temp);      // fill gesture array (model input)
      gestureArray[samplesRead*6 + 0] = a.acceleration.x;
      gestureArray[samplesRead*6 + 1] = a.acceleration.y;
      gestureArray[samplesRead*6 + 2] = a.acceleration.z;
      gestureArray[samplesRead*6 + 3] = g.gyro.x;
      gestureArray[samplesRead*6 + 4] = g.gyro.y;
      gestureArray[samplesRead*6 + 5] = g.gyro.z;
    
      samplesRead++;
    
      delay(10);      // check the end of gesture acquisition
      if (samplesRead == NUM_SAMPLES) {
         uint32_t size_out = 0;
      
         // run model inference
         float* result = model_run_inference(gestureArray,  
                                             GESTURE_ARRAY_SIZE, 
                                             &size_out);         // check if model inference result is valid
         if (result && size_out) {
            // check if problem is binary classification
            if (size_out >= 2) { 
               // check if one of the result has >50% of accuracy
               if (result[0] > 0.5) {
                  Serial.print("Detected gesture: 0"); 
                  // [...]
               } else if (result[1] > 0.5) {
                  Serial.print("Detected gesture: 1"); 
                  // [...]
               } else { 
                  // solution is not reliable
                  Serial.println("Detected gesture: NONE");
               } 
            }
         }
     }
   }
}

Model in action!

Project and code are ready!

/neuton_gesturerecognition
 |- /src
 | |- /Gesture Recognition_v1
 |   |- /model
 |   |- /neuton
 |   |- user_app.c
 |   |- user_app.h
 |- neuton_gesturerecognition.ino

Now, it’s time to see the predictive model in action!

Verify that the hardware system is correctly setup
Open the main application file
Click on the “Verify” button and then on the “Upload” one
Open the Serial Monitor
Grab your hardware system in the hand and perform some motions.

For each detected motion, the model will try to guess what type of movement is (0-punch or 1-flex) and how accurate the prediction is. If the accuracy of the prediction is low (0.5), the model does not make a decision.

Below, an example of model inference execution:

Serial monitor output of Neuton gesture recognition system. Image by author.

…Already done?

Doing Machine learning with Neuton is simple and fast. Model accuracy and performance achieved on a low-power 8-bit microcontroller are impressive!
Neuton is suitable for fast-prototyping development. It allows user to focus on the application, avoiding wasting time in complex and manual statistical analysis.

Here, you can find all Arduino sketches described in this article!