Inside AI Maturity Model

Five steps to transform with data-centric AI engineering

The past decade of AI/ML research and development has been historically significant. From cognitive learning to intelligent analytics, the evolution of deep learning (DL) has kicked off an unprecedented new era of AI. The latest artificial general intelligence (AGI) and self-supervised learning (SSL) are also compelling and promising. Gartner estimates the worldwide AI software market to reach $62 Billion in 2022, an increase of 21.3% from 2021. IDC predicts that investments in AI research and applications are set to hit $500 billion by 2024. And PwC forecasts AI will contribute $15.7 trillion to the global economy by 2030.

Mount Rainier framework (by author)

AI has become an essential technology for enterprise digital transformation and technology innovation. It can significantly improve performance and scalability for organizations and individuals. AI adoption turned into a crazy scramble amid the Covid-19 crisis. However, harnessing and applying AI faces multiple challenges. It is imperative to employ an AI maturity model to guide and accelerate AI adoption in business and beyond.

What is AI Maturity Model?

Software ate the world. The software maturity model (aka Capability Maturity Model, CMM) became a necessary framework for enterprises to standardize and improve software development processes. Now, AI is eating the world. AI maturity model is also essential for AI adoption and refinement.

Along with AlphaGo’s stunning victory in 2016, Gartner, Element AI, Microsoft, IBM, and others started building AI maturity models and frameworks. They have defined different maturity models and frameworks according to their perspectives, but they all curve from low foundational to high transformational levels. Some have segmented the levels into multiple dimensions, forming a 2D model grid. For instance, Element AI has defined five dimensions (Strategy, Data, Technology, People, and Governance) for each stage (i.e., level), from exploring to experimenting, formalizing, optimizing, to transforming.

Gartner and Microsoft AI Maturity Models explained (by author)

Gartner measures an organization against the AI maturity model on five levels, from awareness to active, operational, systemic, and transformational. And Microsoft has defined four maturity levels, from foundational to approaching, aspirational, and mature. Please see detailed descriptions in the above diagram. Companies in the transformational stage are already using AI to achieve significant business value. e.g., Google and Meta use ML extensively to rank pages/posts and advertisements. It’s old news that ML-recommended relevant products or movies have become preferred choices for customers on Amazon and Netflix.

It seems to be an effective process. We can achieve it gradually if we execute it. However, there are two categories of challenges while harnessing an AI maturity model: unique characteristics of AI transformation and challenges of enterprise AI adoption.

Characteristics of AI Transformation

Software and AI share some similarities in terms of maturity model and impact. e.g., DevOps vs. MLOps. But AI/ML is more sophisticated and harder to scale and operate. MLOps can be interpreted as ML plus DevOps. AI technology and transformation present five unique characteristics.

1. While AI is not new in history and has made giant leaps over the past decade, AI itself is still in the early stage. Current AI applications are still limited to specific problems, though there will be full potential to expand AI applications in depth and breadth.
2. AI and ML are learning-based models that rely on data, features, hyperparameters, and even labels (for supervised learning). Thus, data volume and quality can significantly affect AI effectiveness. Big Data was coined as a decade buzzword for data-driven decision-making (BI-related) and then data-intensive deep learning (AI).
3. AI is evolving rapidly in algorithms, modeling, infrastructure, and framework. e.g., generative AI is being upgraded from BERT to RoBERTa, GPT-2, T5, TuringNLG, GPT-3, to DALL·E and DALL·E 2 almost every half a year. ML computing demand is growing nearly 17.5 times faster than Moore’s Law. The increase in demand occurs in both processing and memory.
4. The process of AI development and transformation is complicated. It often involves multiple people working together: business owners, product managers, data engineers, data scientists, ML scientists, ML engineers, etc. In addition to their close collaborations, it may need ML experts to model and explain.
5. Operational AI is far behind its rapid and diverse innovations and applications. Besides the lack of tools, it also needs to be generalized. e.g., MLflow and Kubeflow are open-source ML platforms that require hosting and operation. And Amazon SageMaker is dedicated to AWS cloud.

Challenges of Enterprise AI Adoption

On the other hand, more and more enterprises are excited about AI but face multiple challenges in adopting and transforming with AI. We can wrap it up into five areas.

1. Organizational consensus: AI is new to many organizations and complicated to understand and use. This can lead to inconsistent viewpoints, and agreeing on adoption and development may not be easy. Some may be overly optimistic, believing that AI is now an Omni-use technology, but others may be too pessimistic, thinking that AI is unpractical or too risky. A top-down approach to an agreement can be efficient, but it usually lacks innovation.
2. Unclear use cases: AI works well on specific use cases or domain problems, such as demand forecasting and face recognition. Identifying a suitable scenario is a priority for complex enterprise processes and operations.
3. Insufficient digital intensity: AI/ML is a learning-based mechanism. Digitization and data collection should be the first step after deciding on a use case. Also, the data once used for BI analytics may not be enough for AI.
4. Team readiness: Depending on different use cases, it requires diverse teams to onboard, from business to product to engineering and operation, to achieve AI transformation.
5. Cumbersome process: Today, any good technology will be discounted without moving fast. The process working well for manufacturing and other industrial sectors may not be suitable for AI applications.

Five Effective Steps to Gear Up

So how to address these? Are AI maturity models still practical? The good news is that some companies already use AI extensively to transform their businesses. e.g., Google and Meta have changed the advertising game with AI, as advertising is their primary source of revenue. There are five practical steps to gear up through learning from AI-advanced companies like Google, Meta, Amazon, Microsoft, and Apple.

Five steps to gear up (by author)

1. Organizational AI mindset: Company-wide AI-priority mindset can motivate teams and accelerate experimentation and adoption. It is a cultural shift that targets AI transformation, embracing data-driven and AI-enabled products and services. e.g., Since 2015, Microsoft has been educating and inspiring its employees about AI/ML company-wide. AI/ML is no longer the prerogative of its MSR (Microsoft Research) and some dedicated teams.
2. Measurable AI value: It can be compelling and sustainable if AI value can be measured for a business (or future business value or/and meaningful impact for non-profits). Don’t be too broad. These are good areas to initiate: forecast (e.g., demand forecasting and sales forecasting), classification (e.g., sales team performance classification), anomaly detection (e.g., user transaction outliers), CV (e.g., employee face ID), and ASR/TTS (e.g., automatic voice customer service). Furthermore, it may not be necessary to apply AI/ML if analytics or statistics can work better or more efficiently. e.g., many business decisions can be made through BI analysis without needing ML-based classification or prediction.
3. Data-centric principle: Data is the core of AI/ML. The data-centric principle can improve data quality, availability, and observability. Furthermore, it can enhance model quality and serving accuracy. It is a turning point while facing model-centric challenges.
4. AI engineering: It is a development and practice field aiming to advance, operate, and scale AI/ML in production. It can simplify operations and automate the pipeline from digitization and data collection to feature engineering to modeling, training, and serving. MLOps is a good beginning for operational AI, but AI engineering is more systematic for data engineering and modeling beyond operational automation.
5. Double-speed iteration: As described above, AI/ML is rapid-evolving, learning-based, and process-complex. These require double-speed iteration for fast iterative experiments, developments, and operations. e.g., Amazon has reinvented a lot, from AWS services to fulfillment networks to devices (Kindle, Echo, etc.). But each success relies on its rapid iterative invention and development.

These five steps are related to people, culture, technology, and process. They work together for an organization to gear up for AI transformation. The data-centric principle and AI engineering are tightly associated with technology.

Accelerate with Data-centric AI Engineering

Gartner has identified AI engineering as one of the top strategic technology trends for 2022. AI engineering is a bridge between artificial intelligence research and application. It goes beyond MLOps for ML operational automation from data collection and feature engineering to modeling, training, verifying, serving, and monitoring. It can systematically address data quality, model optimization, user effectiveness, and data and model governance with the engineering discipline.

Data-centric AI engineering is an integrated framework with data as the core and engineering as the discipline. It’s a good practice to develop and operate data-centric AI in an organization for the AI maturity model.

Fulfilling AI Maturity Model explained (by author)

In a Nutshell

The AI maturity model is a framework for adopting AI and transforming with AI. AI transformation is its highest stage and the goal of the AI maturity model. Facing the challenges and characteristics, data-centric AI engineering, which is part of five transformational steps, is the key to fulfilling the efforts. It can accelerate the integration of AI with applications and continue to add value to AI solutions once released.

Both cloud and AI have become foundational technologies in digital transformation. Now cloud is a solution to modernize, and AI is a strategy to innovate. Data-centric AI engineering is an accelerator to reinvent AI applications and reshape the future.

References

1. What is data-centric AI engineering?: https://towardsdatascience.com/what-is-data-centric-ai-engineering-d29b3468062e
2. Digital Transformation 2.0: https://medium.com/@luhuihu/digital-transformation-2-0-bbc97cc8b285
3. Gartner AI Maturity Model: https://www.gartner.com/smarterwithgartner/the-cios-guide-to-artificial-intelligence
4. AI Maturity and Organizations by Microsoft: https://query.prod.cms.rt.microsoft.com/cms/api/am/binary/RE4DIvg
5. AI maturity framework for enterprise applications by IBM: https://www.ibm.com/downloads/cas/OB8M18WR
6. The AI Maturity Framework by Element AI: https://s3.amazonaws.com/element-ai-website-bucket/AI-Maturity-Framework_White-Paper_EN.pdf