The world’s leading publication for data science, AI, and ML professionals.

Introducing OpenHAC- an open source toolkit for digital biomarker analysis and machine learning

The new video analysis platform for human activity classification research and model creation.

Open Human Activity Classification (OpenHAC) is an open source user interface for digital biomarker extraction, analysis, and exploration with tools to build, assess, and deploy machine learning classifiers. It is built on existing software packages used to quantify behavioral characteristics and assemble machine learning frameworks. My goals with this project are 1) to make digital biomarker research accessible to all researchers, developers, scholars, and citizen-scientists to advance digital measurement of health and create access to novel tools in human activity phenotyping, and 2) to create a community of individuals working toward building clinically and academically meaningful technological solutions for human health.


Digital Biomarker Extraction

Digital biomarker features available in OpenHAC currently include facial activity, occulomotion, patterns of movement, body key points, and heart rate.

Face-head-gaze

Face, head, and gaze data rely on OpenFace for their measurements, which are fed forward to OpenDBM to derive clinically and academically relevant behavioral characteristics.

OpenFace: https://github.com/TadasBaltrusaitis/OpenFace

OpenDBM: https://github.com/AiCure/open_dbm

Data collection

Output

Body pose

Body points include localizations on the head, arms, and legs – 15 points in total. OpenHAC relies on OpenCV’s image processing and the pretrained models used in the OpenPose library.

OpenCV: https://opencv.org/

OpenPose: https://github.com/CMU-Perceptual-Computing-Lab/openpose

Data collection:

Output

Heart rate

OpenHAC’s heart rate measurement is built upon the work of habom2310. Using the heart rate function will extract beats per minute and accuracy per frame.

Heart-rate-measurement-using-camera: https://github.com/habom2310/Heart-rate-measurement-using-camera

Data collection

Output


Human Activity Classification

Through OpenHAC it is also possible to create, analyze, and extract new digital biomarker features with it’s Machine Learning classification tools. Combining behavioral characteristics with manual classifications, a user can create effective classifiers for behavioral manifestations such as pain, drowsiness, activity level, and atypical movement – among many others.

OpenHAC uses the PyCaret library, powered by Scicit-Learn, to compare, create, save, load, and deploy machine learning models. The process of creating human activity classifiers is accessible and straightforward with OpenHAC’s graphical user interface.

Setup data and compare multiple models

OpenHAC uses PyCaret’s functions to initialize a training environment that creates a pipeline. It can then train and evaluate performance of all estimators in the PyCaret library using cross validation.

Choose a classifier and assess performance

OpenHAC can train and evaluate any of the models in PyCaret’s library. Plots are used to analyze and interpret the model based on holdout data.

Predict and save

OpenHAC trains the model on the entire dataset and then assigns prediction labels and scores (probability of predicted class). Our transformation pipeline and trained model object are saved as a pickle file for later use.


OpenHAC can be a great tool for people interested in human activity classification. It is still in development and therefore could benefit from those willing to help it grow. If you are interested in contributing to the OpenHAC project, please reach out to me at [email protected]. Thank you for your support!!

OpenHAC’s code is available on GitHub: https://github.com/chags1313/OpenHAC

OpenHAC’s wiki can help you get started: https://github.com/chags1313/OpenHAC/wiki


Related Articles