Towards Ethical Machine Learning
An exploratory analysis of data sceintists’ moral responsibilities
I quit my job to enter an intensive data science bootcamp. I understand the value behind the vast amount of data available that enables us to create predictive machine learning algorithms. In addition to recognizing its value on a professional level, I benefit from these technologies as a consumer. Whenever I find myself in a musical rut, I rely on Spotify’s Discover Weekly. I’m often amazed by how Spotify’s algorithms and other machine learning models so accurately predict my behavior. In fact, when I first sat down to write this post, I took a break to watch one Youtube video. Twenty minutes later, I realized just how well Youtube’s recommendation algorithm works. Although I so clearly see the benefits of machine learning, it is also essential to recognize and mitigate its potential dangers.
While Spotify or Youtube recommending a song or video I dislike would not produce any devastating consequences, that’s not the case for all machine learning algorithms. I first realized potential consequences of machine learning when I came across a collaboration between The Marshal Project and FiveThirtyEight, in which they discussed the use of machine learning in the criminal justice system. I assumed machine learning would be nothing but a benefit to the very broken criminal justice system. After all, the United States accounts for 5 percent of the world’s total population, yet 25 percent of the world’s incarcerated population. Beyond the disproportionate incarcerated population, the United States justice system is built on racist policing and prosecuting, skyrocketing recidivism rates, and excessively harsh punishments for nonviolent drug offenders. Wouldn’t the introduction of machine learning do nothing but help a system so deep in crisis?
While at first glance it seems that machine learning could do nothing but help the United States’ broken criminal justice system, after reading more about some efforts being implemented, I realized that it is essential to proceed with caution. Determining a person’s fate after arrest is one way in which machine learning is being introduced to the criminal justice system, and it exemplifies the risks of relying on machine learning to make a potentially life altering decision.
Traditionally, shortly after an individual is arrested, a judge is tasked with the difficult decision to determine whether they must await their court date in jail, which could be for months or years, or if they can await their court date at home, with or without restrictions. In order to make this decision, a judge must analyze the defendant and essentially must predict whether they deem the individual to be a flight risk — one who would fail to appear at their trial or at risk of committing additional crimes if released to their home.
The fact that a single person is granted the power to make a potentially life altering decision is unfair and makes room for biased decisions. With advances in machine learning, technologies have been introduced in attempt to remove judges’ biases and make decisions based on data, rather than on an individual’s sole judgement. These technologies are often referred to as risk assessment tools and have been developed by both non-profit organizations and for-profit companies. Amongst these risk assessment tools is the Public Safety Assessment (PSA) developed by the Laura and John Arnold Foundation (LJAF). The PSA uses historic data to flag those deemed high risk based on a score associated with their risk of failing to appear at court and their risk of committing another crime. It seems that utilizing an algorithm based on historic data would lead to reduced biases in deciding an individual’s fate, rather than giving the full decision power to a single judge, but many are finding that these risk assessments are exacerbating the discriminatory practices they are trying to eliminate.
These algorithms assign points to model features, such as age, gender, income, drug use, and previous convictions. Despite the PSA being used in 29 cities and states, as well as many other risk assessment tools being used nationwide, the comprehensive list of variables and weights used in these models remain proprietary, and are often referred to as a “black box”. According to Christopher Slobogin, director Vanderbilt Law School’s criminal justice program , “Risk assessments should be impermissible unless both parties get to see all the data that go into them. It should be an open, full-court adversarial proceeding.” If we want to ensure that these algorithms make efforts to reduce biases, it is important to understand the data on which they are based.
Transparency in these risk assessments is invaluable so the general population, as well as the judges who rely on them to make decisions, can fully understand the pathway that leads to their recommendations, including the raw data from which these recommendations extend. After all, as noted on IBM Research, “AI systems are only as good as the data we put into them. Bad data can contain implicit racial, gender, or ideological biases. Many AI systems will continue to be trained using bad data, making this an ongoing problem. But we believe that bias can be tamed and that the AI systems that will tackle bias will be the most successful.”
As data scientists, we must recognize our moral responsibility to train machine learning models with unbiased data. One of the starting points is to allow for transparency and a system of check and balances to ensure that the data on which a model is based is fair and unbiased. In regards to risk assessment tools, it is essential to understand the data used, especially given the historic problems, such as racism and overly harsh treatment of nonviolent drug offenses, that have plagued the United States criminal justice system. We must understand this data in order to digress from these biases, rather than perpetuating them.
One step to reduce biases in machine learning algorithms is through Algorithmic Impact Assessments (AIAs), as proposed by New York University’s AI Now Institute. AIAs extend from the idea that the “black box” methodology leads to a vicious cycle, continuously moving further away from understanding these algorithms and diminishing the ability to address any issues that may arise from them. AI Now suggests the use of AIAs to handle the use of machine learning in the public realm, creating a set of standard requirements . Through AIAs, AI Now aims to provide clarity to the public by publicly listing and explaining algorithm systems used while allowing the public to dispute these systems, developing an audit and assessment processes, and increasing public agencies’ internal capabilities to understand the systems they use.
Similar to the use of AIAs to promote transparency in machine learning, the Defense Advanced Research Projects Agency (DARPA) suggests Explainable Artificial Intelligence (XAI) as a part of the solution. As expressed in the below graphic, XAI strives to produce more explainable models that users can understand and trust.
Though there do not yet appear to be any clear, concise descriptions of XAI on DARPA’s website, they say XAI prototypes are constantly tested, having had the goal for Phase 1 system evaluations to be completed this past November. On the website, they also state, “At the end of the program, the final delivery will be a toolkit library consisting of machine learning and human-computer interface software modules that could be used to develop future explainable AI systems”
AIAs and XAI are only two examples in which organizations are working towards more ethical, transparent machine learning models. As machine learning continues to grow at an exploding rate, there will certainly be more ideas introduced to ensure such regulation. Regardless of the specifics behind these ideas, it is important to maintain a system of transparency and comprehension surrounding machine learning models, during which the data is scrutinized throughout all stages of the machine learning process to ensure fair practices that do not perpetuate biases.
Sources:
