Developing a Successful AI Strategy with the aiSTROM Framework
How to navigate implementing a strategy for AI technologies
A large number of AI projects fail. Rackspace Technology’s survey estimates the number at a whopping 34% [2]. Much of this failure is due to management not understanding the risks and intricacies of AI technologies and, vice versa, developers not knowing how to scale the technologies or the business needs. Many managers seem to think that AI projects will be just like typical software project, however, there are specific challenges involved. These challenges may relate to the skills required in the team, legal issues requiring model transparency, big data governance, cultural challenges for adoption, and many more.
In my recent paper on aiSTROM–A Roadmap for Developing a Successful AI Strategy in IEEE Access [1], I present a detailed roadmap on how to navigate these challenges. The acronym aiSTROM stands for AI-focused STRategic ROadMap. For an in-depth discussion, please refer to the full paper [1]. In what follows, we will provide a brief overview of the steps of the aiSTROM roadmap.
1. Identify opportunities
The opportunities provided by AI technologies might be overwhelming. aiSTROM recommends holding brainstorming workshops with both domain-level experts as well as technical experts to come up with a list of top-n projects, e.g. top 5 projects. Some questions to ask when identifying these may include: what are our competitors doing? can we automate processes? Can we leverage AI to do things in a new way and offer innovative services?
For a full list and extensive discussion about what to keep in mind when selecting projects, please refer to [1]. In what follows, considerations for each of these projects will be examined based on the next steps in the frameworks, with the first one being `data’.
2. Data
The availability of ‘big data’ is one of the catalysts of the current deep learning era, together with new technologies such as CNNs as well as the widespread availability of GPU hardware. We can trace the first mention of big data back to Cox and Ellsworth in 1997 [3]:
“… provides an interesting challenge for computer systems: data sets are generally quite large, taxing the capacities of main memory, local disk, and even remote disk. We call this the problem of big data. When data sets do not fit in main memory (in core), or when they do not fit even on local disk, the most common solution is to acquire more resources…”
Big data is typically characterised by 3V’s: volume (lots of data), variety (messy data), and velocity (it grows quickly). These original V’s are sometimes expanded with Value (business value) and veracity (biases). In machine learning, big data is needed to train our models. This means that we will need to set up the necessary database or warehouse infrastructure to deal with the capacity needed for our project. Some special considerations to keep in mind are discussed below.
Data sources. Is the data already available in the organization? If not, it may need to be collected, or will you consider buying the data? If you consider collecting the data yourself, keep in mind that it takes a lot of time, and ethical board approval is typically needed, as well as preprocessing and quality checks. Yet, starting to collect sooner rather than later seems to be the message, even if you are not really sure you need the data yet. Looking at crowdsourcing methods such as Amazon MTurk may offer a faster alternative to self-collecting, but it is often worse in terms of quality. Finding the right, high-quality data source is an important decision. More data is generally good, but keep in mind the `curse of dimensionality` by Bellman [5], as well as the fact that it is expensive to maintain large pools of data.
Legal issues. There are extensive privacy and security considerations to keep in mind when storing/gathering data. Do you follow the local privacy laws, who in the organization has access to the data, etc. are important questions to consider. It will be necessary to find a good balance between allowing large-scale access of all the data in your company to AI employees (rapid development ideas) versus privacy and security.
Storing data. Will the company be organizing internal storage, which involves large costs and administrative manpower, or using an external data centre? When deciding, consider that the data should be stored close to where the customers are located due to slower transmission speeds. In addition, will this data be stored in its raw form, e.g in a data lake, or be preprocessed into a structured form? These choices should take into consideration data management principles such as the CAP theorem and PACELC theorem [4].
3. The AI team
Developing AI models is not just a technologically challenging task, it involves cutting-edge models and creativity, as well as research-level ideation and problem solving and domain knowledge. A more comprehensive list of skills is provided by Herremans in [1]. Companies often hire PhD-level developers, and/or collaborate with academic institutions. Another way to quickly hire a large pool of talent is through an acquihire: when a company acquires another company, but only to reuse the personnel. An example of this is Google’s hiring of Android, back in 2005 before any product was developed.
4. AI in the company
How will the AI development actually happen?
Team positioning. How will the AI team be positioned in the organization? Some organizations opt for a decentralized approach in which each department is responsible for its own AI projects. In a more centralized approach, a CAI might be hired that can work across departments. This approach can result in the formation of a centre of excellence or research labs. Finally, some organizations may prefer a hybrid approach. The best setup will depend on the organization’s needs and readiness.
Portfolio approach. AI projects and financial assets have some aspects in common: both will have risk levels. A company could implement a portfolio of AI projects, which will level out risks, and provide a buffer when some of them go wrong.
AIaas — AI as a service. Some services already exist, so why not just use an API? This can work but is not recommended if the service is critical to the core business.
Developing in-house or not? Outsourcing may be a good option to get development underway quickly. This decision may depend on the AI readiness level of the company. Another option, other than developing in-house would be to acquire a company that has already developed the technology.
Agility. Like any IT project, agile development approaches are typically recommended. Another best practice is MLOps, a fusion of DevOps with machine learning.
5. Technologies
It’s not our objective here to give an overview of AI technologies. But it is interesting to emphasize a few points related to technological choices:
Accuracy versus black box. AI systems are stochastic systems. A manager can order a developer to predict a customer’s behaviour, but the accuracy of the model is not guaranteed. In fact, it will often depend on whether it is a black-box model (more accurate, but unable to explain) or not. In some sectors, e.g. credit scoring, companies are required by law to provide an explanation of why a decision was made. In this case, simple, less accurate systems such as rule-based systems need to be used, versus (often) more accurate deep learning models that are in essence black box models.
Human-in-the-loop. Some strategies use datasets that are labelled in advance, e.g. someone went through the data and tagged all cars in photos. In reinforcement learning strategies, however, the system will continuously take in feedback and corrections from the users.
Replacing or augmenting humans. AI systems are often seen as a threat to human jobs. In reality, they offer opportunities to empower users to do their jobs better, more creative, easier, and with better control.
Cloud versus in-house hosting. This decision will be based on the cost (human capital + equipment) of hosting servers in-house versus the subscription fees that are often easier scalable than renting servers.
6. KPIs
Like any project, having clear value-based success metrics from the onset is essential. These should be linked to the organization’s strategic goals. Note the word ‘value-based’ above, which indicates that we should look at much more than just financial goals, but instead also how this AI technology may create value for the customers or employees. In addition, given that AI models are stochastic, and accuracy is not guaranteed, this is also something that should be evaluated!
7. Setting a risk level
Some of the risks of AI systems that need to be monitored during development include:
- Their stochastic nature. The models may not work accurately!
- Biases and Ethics. Racial/gender/… learned-biases must be avoided in any model that is developed
- Security. The model should be resistant to adversarial attacks as well as spoofing attacks.
- Other strategic decisions. Risks can originate from any of the previous decisions: in-house development hiring now working out, hosting failing, data insecurity, etc.
When considering the risks, we also weigh the benefits. Ideally, these are captured by the above-mentioned value-based KPIs.
A SWOT analysis can put both of these together and provide a nice overview of management decisions.
8. Enabling a cultural shift
Personnel directly involved in the projects, be it managers or developers, must be knowledgeable of AI technologies. Even other employees, as far removed as they may seem from the project, contribute to the overall culture of the organization. AI-educated employees may contribute ideas and insights based on their specific knowledge of the company.
By setting up a centre of excellence, the company may raise awareness and educate employees, which will lead to the embracing of AI technologies throughout the organization. Any fears of being replaced by AI technologies are typically mitigated by increased AI literacy, hence this will greatly help foster a culture of AI adoption.
Conclusion
I hope to have offered a nice overview of some of the strategic management decisions that come along with implementing a long-term AI strategy. aiSTROM offers a roadmap that guides the manager through different domains and areas that need strategic decisions when implementing AI strategies.
If you are interested in reading more details, do check out the original aiSTROM paper, or get in touch to discuss how I can help you! Do you need an interface or template to generate aiSTROM reports, or do you have any other needs, let me know!
About the author
Dorien Herremans (Senior Member, IEEE) received her Ph.D. degree in applied economics from the University of Antwerp. She was awarded a Marie-Curie Fellowship to work at the Centre for Digital Music, Queen Mary University of London. Before that, she graduated as a Business Engineer in Management Information Systems at the University of Antwerp in 2005, after which she worked as a Drupal Consultant and as an IT Lecturer at the world’s leading hospitality business school, Les Roches, Bluche, Switzerland. She is currently an Assistant Professor at the Singapore University of Technology and Design (SUTD). At SUTD, she is also the Director of the SUTD Game Lab and heads both the Audio, Music, and AI (AMAAI) Lab as well as the AI for Finance (AIFi) Group. Her passions include strategic thinking as well as novel applications of AI technologies. She serves on numerous committees and boards and has given invited talks around the world. She is on the Singapore 100 Women in Tech 2021 list, which recognizes and celebrates inspiring women based in Singapore who have made significant contributions to the tech industry.
References
[1] D. Herremans, “aiSTROM–A Roadmap for Developing a Successful AI Strategy,” in IEEE Access, vol. 9, pp. 155826–155838, 2021, doi: 10.1109/ACCESS.2021.3127548.
[2] Global Report: Are Organizations Succeeding at AI and Machine Learning?, 2021, [online] Available: https://www.rackspace.com/solve/succeeding-ai-ml/ty.
[3] M. Cox and D. Ellsworth, “Application-controlled demand paging for out-of-core visualization”, Proc. 8th Conf. Vis. (VIS), pp. 235, 1997.
[4] M. Chen, S. Mao and Y. Liu, “Big data: A survey”, Mobile Netw. Appl., vol. 19, no. 2, pp. 171–209, 2014.
[5] R. Bellman, Adaptive Control Processes: A Guided Tour, Princeton, NJ, USA:Princeton Univ. Press, vol. 3, pp. 2, 1961.
