Deep Learning-based Inertial Navigation: A Hybrid Navigation Filter
The inertial navigation domain is considered a very classical one. It aims to provide us with navigation solutions (position, velocity, and orientation) using (low-graded) inertial sensors and accurate (low frequency) sensors, like the GPS receiver. A question has arisen about how powerful Deep Learning tools can boost this classical domain.
Personal Motivation Intro: Why Did I Start It?
Three years ago, I started asking myself if there is a value in combining Machine Learning, generally, and Deep Learning (DL), particularly, in the classical inertial navigation system (INS). After 3 years in the industry, I started my Ph.D. studies at the University of Haifa, including working for Qualcomm and Autotalks. In these stations, I have revolved around many fascinating solutions that the DL accelerated. For example, I worked on Qualcomm’s fingerprint sensors, which currently exist in many smartphone devices.
Being a part of this revolution made me think a lot about my previous education: I graduated from the Technion- The Israeli Institute of Technology with 3 degrees: M.Sc. and B.Sc. in Aerospace Engineering and B.A. in Economics and Management (Cum Laude). I used to focus mainly on optimal control and estimation for real-time navigation and tracking purposes. It seems natural to stay in this field and integrate DL as a part of my Ph.D., titled “Machine Learning-based Inertial Navigation”.
Background
The classical INS combines accelerometer and gyroscope measurements into kinematic equations. Eventually, the navigation solution is achieved. But, these two sensors are very noisy, making the solution bad and suffering from fast divergence. Hence, it is most likely to fuse the classical INS with an accurate sensor. For example, the global positioning system (GPS) receiver. These kinds of fusion schemes are known as INS/GPS. The big question is, “How to integrate all sensors together to achieve the most accurate navigation solution?” The answer to that involves the famous signal processing algorithm, the Kalman filter (KF). The KF is the best-known algorithm for inertial navigation sensor fusion tasks. Most of the time, the extended KF (EKF) version is used as the navigation equations involve nonlinearity.
I presented a problem and a solution -so where is the DL combined? The answer is simple. The navigation filter, or generally any EKF, must have its own parameters to work optimally.
Is The Kalman Filter Really An Optimal Filter?
In most of the estimation books, you will probably find that the KF is an optimal filter in the sense of minimizing the square root error (MSE) between the state and the prediction of these states (the state is a vector that could be built of 15 states: position and velocity in 3 dimensions plus the body orientation angles (3 more). There are additional six states for the accelerometer and gyro biases. But we want to minimize the error between the true state vector and the prediction state vector. In real life, we don’t have an access to this true state unless using a very accurate device (i.e. Real Time Kinematic Device).
As the filter designer responsible for the dynamic model for the entire navigation scenario, he might be wrong with the state and its equations (transition matrix). Remember, this is just a kinematic model and not the true kinematic itself! To be optimal, the designer must design an accurate dynamic model (later, we will also discuss the measurement model). As no one knows the future, the great KF allows us to add some weight to our uncertainty in the modeling phase. This uncertainty is considered during the system model noise covariance matrix. For each state, we can put some values, representing white Gaussian noise variances, to help the KF handle some dynamics it will meet during the real-time navigation.
So, KF is optimal in the sense of MSE between the model state and the estimated state. The KF is not optimal in the sense of MSE between the true state and the estimated state…we need to continue tuning its parameters in a real-time setting, depending on the environment we navigate.
KF is optimal in the sense of MSE between the model state and the estimated state. The KF is not optimal in the sense of MSE between the true state and the estimated state…we need to keep tuning its parameters in a real-time setting.
The Hybrid Approach
So, here is the part where the DL gets involved. Let’s look at the diagram below: the EKF-based navigation filter is provided by the GPS and INS inputs (green and purple, respectively) and eventually provides an accurate navigation solution. The DL is used to learn the tuning parameters of the EKF from the inertial measurements (accelerometer and gyroscope). These measurements are provided in a real-time setting, so the DL model also tunes these parameters in a real time manner. For example, there is a level of uncertainty in the dynamic model of the INS. This uncertainty is due to the noise, disturbances, and additional factors in the accelerometer and gyroscope. These factors are changed during the vehicle’s navigation and lead to degraded navigation performance. To fix it, there is a need to tune the uncertainty of the dynamic model, accordingly.
One way to do that is by tuning the noise covariance matrix of the dynamic model (Q). The matrix values influence the magnitude of the white Gaussian noise we inject into the model. Hence, as the value increases, the uncertainty of the designer increases and vice versa. There are many classical adaptive approaches, considering entities of the EKF (such as the innovation property) to tune it. In this suggested Hybrid Approach, we rely on a trained model from a simulative dataset with many examples of noisy signals and their related variances (labels).
Once designing such a hybrid navigation filer model, one needs to consider many technical details regarding the DL model. A partial list includes the length of the signal (example), deciding to include some feature engineering or not, frequency of updating the tuning parameters. In our most recent paper, “A Hybrid Model and Learning-Based Adaptive Navigation Filter” written by Barak Or and Itzik Klein and published in IEEE Transactions on Instrumentation and Measurement (a preprint is available here), we thought about these details.
Summary
This work is the first of its kind in the hybrid navigation model, where the classical Kalman Filter is considered, aside from a deep learning model — used to tune the filter parameters. The combination yields great improvement in some of the case studies we tested. One included quadcopter navigation, where our Hybrid approach yielded a 27% improvement in position accuracy.
References
If you want to read the original paper, it can be found on the IEEE website:
[1] Or, Barak, and Itzik Klein. “A Hybrid Model and Learning-Based Adaptive Navigation Filter.” IEEE Transactions on Instrumentation and Measurement 71 (2022): 1–11.
If you want to keep reading about the problem of identifying the noise covariance parameters you may refer to this work on the IEEE website:
[2] Zhang, Lingyi, David Sidoti, Adam Bienkowski, Krishna R. Pattipati, Yaakov Bar-Shalom, and David L. Kleinman. “On the identification of noise covariances and adaptive Kalman filtering: A new look at a 50-year-old problem.” IEEE Access 8 (2020): 59362–59388.
if you want to keep reading about deep learning for inertial sensors, I recommend my previous post here on Medium:
[3] Barak Or. “Deep Learning for Inertial Navigation: A short review of cutting edge deep learning-based solutions for inertial navigation.” 2020.
About the Author
Dr. Barak Or is a professional in the field of artificial intelligence and sensor fusion. He is a researcher, lecturer, and entrepreneur who has published numerous patents and articles in professional journals. Dr. Or leads the MetaOr Artificial Intelligence firm. He founded ALMA Tech. LTD holds patents in the field of AI and navigation. He has worked with Qualcomm as DSP and machine learning algorithms expert. He completed his Ph.D. in machine learning for sensor fusion at the University of Haifa, Israel. He holds M.Sc. (2018) and B.Sc. (2016) degrees in Aerospace Engineering and B.A. in Economics and Management (2016, Cum Laude) from the Technion, Israel Institute of Technology. He has received several prizes and research grants from the Israel Innovation Authority, the Israeli Ministry of Defence, and the Israeli Ministry of Economic and Industrial. In 2021, he was nominated by the Technion for “graduate achievements” in the field of High-tech.
Website www.metaor.ai Linkedin www.linkedin.com/in/barakor/ YouTube www.youtube.com/channel/UCYDidZ8GUzUy_tYtxvVjRiQ
