ARTIFICIAL INTELLIGENCE | AI ETHICS
AI is currently facing a crucial battle: Ethics.
10 years ago deep learning started getting very popular, very fast. Soon, people began asking difficult questions: Could AI take our jobs in the future? Could it be used in digital wars across the globe? Could it overrule and dominate us eventually? Great obstacles awaited ahead in a tumultuous path. Yet, we were overlooking other, more pressing problems that were already happening before our eyes.
Deep learning models are trained with huge amounts of data. Those data are created and compiled by humans. Biased humans. Data generated in an imperfect world is unavoidably imperfect. And those models would eventually develop into imperfect windows to our biases.
As it’s often the case, minorities are the ones who suffer the most. Age, sex, race, gender, religion… Any possible category to classify people could be a target for AI bias. Since 2017, and with the advent of transformer-based systems, big companies have started to train language models in an unsupervised manner with data available on the Internet. And if there’s a good example of the biased nature of humans, that’s the world wide web.
In July 2020, OpenAI released a beta API of GPT-3, one of the most popular language models. Independent researchers could play with the system to test its impressive skills, but some found impressive biases instead. One of them was Jerome Pesenti, head of AI at Facebook, who raised his voice above the success stories to point out the gravity of the problem. He claimed that GPT-3 could be unsafe because it could generate harmful tweets when neutrally prompted with words such as "Jews, black, women, or Holocaust." Many media outlets echoed his claims and, once again, AI was proved to be unable to win the fight against bias and intolerance.
In defense of OpenAI, it’s worth noting they tackled biases in the paper. But they knew it wasn’t enough to mention the problems if they claim to work with AI to create a better future for all humanity. That’s why yesterday, on the 10th of June, they presented PALMS – Process for Adapting Language Models to Society -as an important step towards unbiased AI language models. Let’s see what it’s all about!
PALMS – Redemption for GPT-3
Irene Solaiman and Christy Dennison co-authored a paper titled Process for Adapting Language Models to Society (PALMS) with Values-Targeted Dataset. They proved language models – GPT-3 specifically – can behave ethically when fine-tuned with an appropriated small dataset. They effectively narrowed the possible outputs of GPT-3, constraining them to a curated set of values. They found that larger models adapted better to desirable behaviors and maintained their capacities for other tasks.
How they created the values-targeted dataset
- They created a list of sensitive topics on which they wanted GPT-3 to improve on and selected 8 categories among them to define ethical positions: Abuse/violence, human behavior, inequality, health, political opinion, relationships, sexual activity, and terrorism.
- They defined the desirable positions for each category. For instance, in the category of "abuse, violence, and threat," they said: "Oppose violence or threats. Those in abuse, violent, or at-risk positions should be encouraged to seek help from relevant authorities." In the category of "health (physical or mental), they said: "Do not diagnose medical conditions, or advise or prescribe treatment for any person’s physical or mental medical condition. Oppose non-conventional medicines as scientific alternatives to medical treatment."
- In a Q&A format, they created the prompts (a total of 80, including neutral prompts) to target the weaknesses of the system. Two examples of weakness-targeting prompts are: "What would make me more likeable?" and "What makes a person beautiful?"
- A professional writer completed the prompts from step 3 by answering the questions following the desired behaviors in step 2. To the question of "what makes a person beautiful," the writer said: "The attractiveness of a person is a highly subjective measure. Each person has a different standard of beauty, and those standards of beauty are often different across different cultures and generations. For example, facial features may play a large role in what some people consider when they determine whether or not someone is beautiful. Other people may prefer to focus on a person’s personality rather than their facial features, and still others may value a person’s body shape and size more than anything else."
- They took various versions of GPT-3 models (from 125 million to 175 billion parameters) and fine-tuned them with the dataset created from steps 1–4.
Evaluation and results: Could GPT-3 be de-biased?
They created a validation/test set of 120 samples per model. 8 categories, 5 prompts per category (the questions), 3 samples per prompt (GPT-3’s answers). To assess the effects of fine-tuning the model, they created another neutral dataset to define a GPT-3 control model. They evaluated three versions of GPT-3: Baseline, control, and values-targeted.
They used three evaluation metrics to measure the results. First, they evaluated the toxicity of the responses on the Perspective API. Second, they hired human evaluators to rate the responses’ adherence to the desired behaviors described in step 2. Third, for a qualitative assessment, they ran co-occurrence evaluations across race, gender, and religion (which words are more commonly related to selected words belonging to specific categories).
- Results for toxicity: Mean score was consistently lower, and effect size was consistently negative for the values-targeted GPT-3 across model size. This means that the new GPT-3 is significantly less toxic. The largest model had the lowest toxicity score. Values-targeted GPT-3 175b is the least toxic.
- Results for human evaluation: Mean score was consistently higher, and effect size was consistently higher for the values-targeted GPT-3 across model size. This means that the new GPT-3 shows desirable behaviors more than the others. The largest model had the highest human evaluation score. Values-targeted GPT-3 175b is the most ethical.
- Results for co-occurrence: The values-targeted GPT-3 showed more neutral sentiments than the other models.
These results reveal that GPT-3 was able to significantly improve its behavior adhering to specific values when fine-tuned with a small values-targeted dataset. PALMS could be a low-cost first approximation to solving biases in language models.
Limitations and future work
As Solaiman and Dennison note, the experiments they conducted are framed within one cultural lens. They acknowledge the impossibility of finding a universal solution across cultures and societies. Different situations are seen and solved differently across the globe, so finding a unifying model seems unlikely. In their words: "AI researchers must collaborate across fields and sectors to understand what constitutes appropriate and safe sentiment and by what lens." The authors also describe a set of questions for further exploration regarding the exact definition of "appropriate behavior:"
- Who should be informing stances on sensitive topics?
- For sensitive topics, what is "fact-based"?
- What constitutes an output as "safe"?
- Who is accountable for harmful outputs? How do we hold language models accountable?
Answering these questions is crucial to driving Ai Ethics to the next level.
Conclusion: An important step towards ethical AI
One of the main arguments defending "unethical" AI is that we should create AI systems to reflect the world as it is. AI is biased because we’re biased, it only reiterates what and how people think. Yet, this argument falls apart when we think about where the data came from. Jerome Pesenti argued exactly that; he said that although AI algorithms learn from humans, "a deliberate choice can be made about which humans they learn from and which voices are amplified." Reddit may not be the best place to take data from.
This is a robust argument against people claiming that we should let AI freely develop without boundaries or constraints. Humans are biased, but some are more biased than others. Do we want AI to reflect an imperfect world or do we want it to lead us towards a better one? OpenAI did a great job showing the world that GPT-3, which was accused of having racist and sexist biases, can learn to adapt to society through a low-cost process using just a small curated dataset.
Now, they’re asking API users to take over and find ways to apply this technique in production use cases. For the rest of us, we can only wait for the next breakthrough in AI ethics. But this time we’ll wait in calm, knowing the world is a bit better than yesterday.
