PODCAST
The emerging world of ML sensors
Matthew Stewart on a new paradigm for edge device sensing
Editor’s note: The TDS Podcast is hosted by Jeremie Harris, who is the co-founder of Gladstone AI. Every week, Jeremie chats with researchers and business leaders at the forefront of the field to unpack the most pressing questions around data science, machine learning, and AI.
Today, we live in the era of AI scaling. It seems like everywhere you look people are pushing to make large language models larger, or more multi-modal and leveraging ungodly amounts of processing power to do it.
But although that’s one of the defining trends of the modern AI era, it’s not the only one. At the far opposite extreme from the world of hyperscale transformers and giant dense nets is the fast-evolving world of TinyML, where the goal is to pack AI systems onto small edge devices.
My guest today is Matthew Stewart, a deep learning and TinyML researcher at Harvard University, where he collaborates with the world’s leading IoT and TinyML experts on projects aimed at getting small devices to do big things with AI. Recently, along with his colleagues, Matt co-authored a paper that introduced a new way of thinking about sensing.
The idea is to tightly integrate machine learning and sensing on one device. For example, today we might have a sensor like a camera embedded on an edge device, and that camera would have to send data about all the pixels in its field of view back to a central server that might take that data and use it to perform a task like facial recognition. But that’s not great because it involves sending potentially sensitive data — in this case, images of people’s faces — from an edge device to a server, introducing security risks.
So instead, what if the camera’s output was processed on the edge device itself, so that all that had to be sent to the server was much less sensitive information, like whether or not a given face was detected? These systems — where edge devices harness onboard AI, and share only processed outputs with the rest of the world — are what Matt and his colleagues call ML sensors.
ML sensors really do seem like they’ll be part of the future, and they introduce a host of challenging ethical, privacy, and operational questions that I discussed with Matt on this episode of the TDS podcast.
Here were some of my favourite take-homes from the conversation:
- Most modern smart devices aren’t actually smart themselves. Although they collect various kinds of sensor data, they don’t generally process that data themselves. Instead, they typically send it to a centralized server that applies whatever algorithms are needed. This approach has a significant downside: it forces developers to send sensor data from sensors to servers over the cloud — a process that can lead to interception or data corruption. But it’s currently the default simply because it’s historically been so hard to pack vision, robotic control, or language models onto edge devices with limited memory and processing power.
- Amazon Echo and Google Home actually have functionalities that match the ML sensor concept. These devices typically have onboard an algorithm that allows them to detect when a call-to-attention command is given (e.g. “Ok Google”), and once that command is heard, they proceed to forward the user’s query to a central server that can do the heavy lifting involved in responding to the user’s query. In this case, the “ML sensor” would be the combination of Google Home’s microphone and its “Ok Google” recognition software.
- Matt uses the example of facial recognition to illustrate the ML sensor paradigm. Currently, we think of sensors as “things that feed raw data to our powerful processors” — and things like cameras fit that description nicely. But imagine that instead of buying a camera, you could buy a “face detector” — an integrated device that combines a camera with an onboard facial recognition model, and whose overall output isn’t an image, but rather a conclusion (“Jeremie’s face was detected!” Or “Jeremie’s face wasn’t detected.”)
- ML sensors are probably going to be harder to characterize than traditional sensors. That’s because they can contain any combination of hardware and software — any sensor, plus any model applied to that sensor’s output data. Any change to the sensor’s onboard software could have a significant impact on its performance characteristics.
Chapters:
- 0:00 Intro
- 3:20 Special challenges with TinyML
- 9:00 Most challenging aspects of Matt’s work
- 12:30 ML sensors
- 21:30 Customizing the technology
- 24:45 Data sheets and ML sensors
- 31:30 Customers with their own custom software
- 36:00 Access to the algorithm
- 40:30 Wrap-up
