Towards Responsible AI (Part 2)
In the first part of this series, we looked at AI risks from five dimensions. We talked about the dark side of AI, without really going into how we would manage and mitigate these risks. In this and subsequent articles, we will look at how to exploit the benefits of AI, while at the same time guarding against the risks.
A quick plot of search trends shows that the words "AI Ethics", "Ethical AI", "Beneficial AI", "Trustworthy AI", and "Responsible AI" started becoming extremely popular over the past five years. In my (first author’s) early exploits of AI in the ’80s and ’90s talking about ethics was relegated to a small fringe of academics and definitely not a topic of conversation in the business world – at least with respect to AI ethics. It is not surprising that these terms are trending – given both the adoption of AI and the substantial risks of AI that we examined earlier in the series.
But what do all these terms really mean? Who is coming up with all these terms? What does it really mean for a company – especially if you are not a technology company using and promoting AI.
What is in a Name?
What’s in a name. That which we call a rose by any other name would smell as sweet.
This famous quote is by William Shakespeare in his play "Romeo and Juliet". Does it really matter what we call a set of Principles for AI – isn’t "Responsible AI" as good a term as "Trustworthy AI" or "Ethical AI" or "Beneficial AI"? Unfortunately, the answer is an emphatic NO. What we call some of these principles really matter.
A more apt quote for us might be this one from Charles Babbage, the eighteenth century English mathematician and inventor who conceived the first digital computer.
What is there in a name? It is merely an empty basket, until you put something into it.
While ten years back these terms would have been "empty baskets" they do mean something very specific today. Let’s look at each of these baskets and what they mean today.
Trustworthy AI: The European Commission’s High-Level Expert Group on AI (HLEG-AI) defines trustworthy AI "as lawful (complies with all applicable laws and regulations), ethical (ensures adherence to ethical principles and values), and robust AI (does not cause unintentional harm)".
Ethical AI: From EU’s perspective, Ethical AI is a part of trustworthy AI. The ethical principles are derived from five fundamental rights – respect for human dignity, freedom of the individual, respect for democracy, justice and the rule of law, equality, non-discrimination and solidarity, and citizens’ rights. Flowing from these five fundamental rights the HLEG-AI proposes four ethical principles:
Principle of respect for human autonomy: This principle means that "AI systems should not unjustifiably subordinate, coerce, deceive, manipulate, condition or herd humans. Instead, they should be designed to augment, complement and empower human cognitive, social and cultural skills."
Principle of prevention of harm: This principle requires that "AI systems should neither cause nor exacerbate harm or otherwise adversely affect human beings".
Principle of fairness: This principle ensures that "the development, deployment and use of AI systems must be fair."
Principle of explicability: This principle "means that processes need to be transparent, the capabilities and purpose of AI systems openly communicated, and decisions – to the extent possible – explainable to those directly and indirectly affected."
The HLEG-AI also recognizes that there are tensions between these principles and the trade-offs between them should be identified, evaluated, and then acted upon.
Beneficial AI: The HLEG-AI does not define or delve into this concept. The notion of beneficial AI came from one of the Asilomar Principles. The Asilomar Conference in 2017 defined the research goal of AI as "to create not undirected intelligence, but beneficial intelligence". The word "beneficial" requires more scrutiny – beneficial to whom – all of humanity; some of them; to company shareholders funding the AI? When is it beneficial – now, ten years from now? Who decides it is beneficial? and so on. Stuart Russell introduces the concept of provably beneficial AI. According to him "machines are beneficial to the extent that their actions can be expected to achieve our objectives". The focus here is on aligning the values of the AI with human values and having the AI learn our values. The notion of ‘provable’ is more aspirational on what we should be aiming for – not what we have today.
It is clear from the definitions that Ethical AI is a subset of Trustworthy AI. However, the relationship between Beneficial AI and Ethical AI is not clear. Can we have AI that is ethical as defined by the four principles, yet not be beneficial? Can we have AI that is beneficial, but not ethical based on the four principles?
The answers to both these questions can be affirmative based on how we interpret the definitions and the trade-offs inherent in them. An ethical AI, respecting the four principles can still be created in a manner that doesn’t achieve the objectives of humanity (e.g., an AI initiative to explore deep space may respect human autonomy, prevention of harm etc., and yet not be beneficial to the large majority of humanity – except for a few deep space enthusiasts!!). Similarly, one could have beneficial AI without necessarily respecting all of the fundamental rights and ethical principles (e.g., an AI that imposes severe restriction on human mobility during the pandemic may help save the lives of large numbers of people and yet curtail the freedom of the individuals!!).
Realization of Trustworthy AI
The HLEG-AI document not only lays out the four ethical principles of AI, but it also lays out seven key requirements for the realization of trustworthy AI. These requirements address different stakeholders involved in different stages of the AI model development lifecycle (see Model lifecycle: From ideas to value), including end-users, developers, companies, and society at large. The seven requirements from HLEG-AI are:
(a) human agency and oversight; (b) technical robustness and safety; (c ) privacy and data governance; (d) transparency; (e) diversity, non-discrimination and fairness; (f) societal and environmental wellbeing; and (g) accountability.
These requirements capture the key fundamental rights and the ethical principles. No system is by default trusted; in order to earn that trust, systems must demonstrably reflect these seven principles.
In summary, we do have two distinct concepts of trustworthy AI (that includes ethical AI) and beneficial AI. Now let’s examine Responsible Ai.
Responsible AI
Now let’s examine the word "responsible AI". According to Lima and Cha, The word "responsible" has three different meanings that are relevant:
Responsibility as Blameworthiness: This notion proposes that agent i should be held responsible if it is appropriate to attribute blame to i for a specific action or omission. The necessary conditions for such blameworthiness are 1) moral agency, 2) causality, 3) knowledge, 4) freedom, and 5) wrongdoing.
Responsibility as Accountability: An agent i is considered responsible-as-accountable for a specific action had i been assigned the role to bring about or to prevent it. The necessary conditions for such accountability are 1) the agent’s capacity to act responsibly and 2) a causal connection between i and the action.
Responsibility as Liability: The duty of liability to agent i implies that i should remedy or compensate certain parties for its action or omission. Here, the focus is on the attribution of liability regardless of moral agency, as legal systems often do through strict liability assignment.
Trustworthy AI captures the notion of blameworthiness by its requirement of human agency and oversight. It proposes oversight through governance mechanisms such as human-in-the-loop, human-on-the-loop, and human-in-command approach. Accountability is an explicit requirement in trustworthy AI that includes auditability, minimization and reporting of negative impact, trade-offs, and redress. The notion of liability is captured in trustworthy AI, to the extent that there are existing laws and regulations to compensate parties for action or omission. However, it does not explicitly include such compensation as a requirement.
Another definition of Responsible AI comes from Virgina Dignum:
Responsible Artificial Intelligence is then an approach that aims to consider the ethical, moral, legal, cultural, and socio-economic consequences during the development and deployment of AI systems.
Responsible AI is about human responsibility for the development of intelligent systems along fundamental human principles and values, to ensure human flourishing and well-being in a sustainable world.
This view provides a more holistic view of AI (e.g., not just ethical or moral, but legal, cultural, and socio-economic) and also emphasizes the development and deployment of AI systems. By placing the emphasis on human systems, processes, and governance, as opposed to just the technology of AI, we can assess and determine the beneficial nature of the AI system.
#
Now we are ready to present the ten principles of Responsible AI. These principles are derived from the definitions of trustworthy AI, ethical AI, beneficial AI, and responsible AI. Also, the focus for us here is on one of the specific stakeholders i.e., corporates who are building, deploying, buying, or using AI systems.
- Align on AI principles and practices: Given the specific ability that the AI system is automating, assisting, augmenting or autonomously performing, and the business use cases and impact on its customers and society the company has to align amongst its different business units and functional areas the set of AI principles, policies, practices that it wants to adopt.
- Confirm adequate top-down and end-to-end governance: This principle is a variant of the human agency and oversight, but applied in the corporate setting. The human oversight needs to be applied from an end-to-end AI system lifecycle perspective, as well as from the end-user and regulators to the senior executives and the Board of a company.
- Design for robustness & safety: The principle of robustness and safety of AI system needs to be designed based on the risks associated with the societal impact of the AI system. Risk tiering is a common way of achieving this that we will evaluate in future articles. Furthermore, systems need to perform reliably as expected; instability in systems can violate users’ trust and may result in disparate impact across AI system decision making
- Exercise control and value alignment: The principle of control and value alignment draws its inspiration from the requirements of beneficial AI. The value alignment here is limited to the corporate values and does not address the broad issue of human values. Similarly, the principle of control is focused on ensuring identifying when an AI system may start deviating from its performance and getting it under control.
- Respect privacy: This principle is an extension of the privacy and data governance requirement of trustworthy AI. However, the privacy is not just on the original data, but also on the insights, decisions, actions, and outcomes of the AI system – to the extent that such privacy is required. There is an inevitable trade-off between the privacy and transparency principle. Many regulations focus on this principle (CCPA, GDPR) however organizations should consider more broadly what they "should" vs "can" do with data.
- Be transparent: This principle embodies the traceability, explainability and communication of information, decisions, and actions of the AI system as well as the data that feeds the AI system, and visibility into how (and which) broader systems leverage AI.
- Embed security: This principle is based on embedding security within the design and deployment of the AI system. The principle of security is designed to protect users against both unintentional harm, as well as malicious harm, which could include adversarial attacks on AI decision making. The HLEG-AI includes this principle within the robustness requirement.
- Enable diversity, non-discrimination and fairness: This principle focuses on the avoidance or minimization of unfair and unexpected bias, accessibility and universal design and diversity in the end-to-end model or AI system lifecycle. It facilitates the broader societal benefits of the AI system.
- Clarify accountability: This principle embodies the requirements of auditability, minimization and reporting of negative impact, trade-offs and redress. They address the key elements of responsibility as blameworthiness, accountability, and liability that we discussed earlier.
- Foster societal and environmental well-being: This principle, especially as it applies to companies, is relatively more recent with the desire of companies to focus on Environmental, Social, and Governance (ESG) factors. This principle focuses on the social and environmental factors while Principle #2 addresses the broader governance aspects. This principle has also been sometimes referred to as AI4Good or AI for social good. Some recent efforts have explored adding Well Being measures to ESG, and largely considering how to quantify ESG (and beyond) as measures to include as acceptance criteria in the development and use of AI.
Many of these principles have been evolving over the past couple of years as a number of organizations, not-for-profits and consortiums, professional associations, governments, and regulatory bodies have tried to synthesize and agree upon a common set of principles. Our earlier set of principles can be found in a number of publications including the AI Index, the practical guide to responsible AI, and our responsible AI toolkit and survey.
In the subsequent articles we will focus on how we operationalize some of these principles and examine some leading practices of responsible AI.
Authors: Anand S. Rao and Ilana Golbin
