How to Remove Bias in Machine Learning Training Data
Understanding & Reducing Bias In Training Data For AI/ML Projects
Much has changed in the AI/ML world but the concept of ‘garbage in; garbage out’ remains stoic. Any algorithm is only as good as its training data. And, no training data is without bias, not even the ones generated through automation.
In the past, many machine learning algorithms have been unfair to certain religions, races, genders, ethnicities, and economical statuses, among others. The Watson supercomputer from IBM that gave suggestions to doctors using a dataset of medical research papers was found to favor reputable studies only. Amazon’s recruiting algorithm was found to favor men over women. COMPAS, a risk assessment algorithm used by many state judges in the USA to assist sentence determination, was disputed for assigning a higher reoffense risk factor in violent crimes to Black people. In 2016, Microsoft’s one-day experiment with a Twitter chatbot (that learned from other tweets) ended badly when the bot released 95000 tweets in 16 hours, most filled with misogyny, racism, and anti-semitic ideas.
But, here is the thing — Algorithms aren’t unethical, racist, or morally flawed. However, the data they are trained on is another story.
Bias In Data Is Complicated
A bias can corrupt data at multiple junctions. It can seep into an otherwise harmless dataset during collection, data aggregation, model selection, or even end-user interpretation. In fact, it is almost alright to assume that no data exists without one or other kind of bias. It may be caused by external prejudice from the human trainer. Or, it may appear as a result of oversimplification or unsatisfactory representation of any group in the dataset.
The root cause isn’t always obvious. But, the impact is!
Impact Of Data Bias On Machine Learning
A biased dataset is unbalanced. It fails in representing the original motive of the machine learning model. Its results are inaccurate, its decisions are prejudiced, and its precision level can vary under different contexts, thus defeating the purpose of the model.
COMPAS (Correctional Offender Management Profiling for Alternative Sanctions) is a very popular example of the ramifications of a self-sufficient algorithm trained on biased data. It uses a questionnaire to evaluate offenders on various parameters and predicts the likelihood of them committing a crime again. This prediction is used by judges across many US states and jurisdictions to determine sentences.
After a few questionable decisions, various questions about its integrity, and several lawsuits, ProPublica examined the algorithm’s risk assessment factors and observed two major absurdities.
Prediction accuracy below 20%
2X instances of wrong labeling in the case of Black offenders
COMPAS is just one example to demonstrate the very real consequences that a human may have to suffer due to a prejudiced ML model. The repercussions of any AI/Ml model’s implementational scope are directly impacted by the nature of training an algorithm gets, and hence, on the data annotation service that creates said dataset.
Types Of Data Bias
Sample Bias:
Sample (or Selection) bias is introduced in a system when a subgroup of the data is excluded during its study, accidentally or knowingly. Such a sample will not genuinely reflect the intended observable environment and lead to inaccurate results.
For example, consider a dataset that studies data on the population of a locality for 15 years to help predict new tenant tenure. A bias may be introduced if we falsely interpret the 15-year term as an absolute and exclude any tenant that lived in the locality for less than that duration from our study. Eliminating such data renders the model ineffective because it will not be trained on every possible case.
Algorithm Bias:
Algorithmic bias is a systematic error that leads to poor computation. It could be a pre-existing issue or emerge because of program limitations or design constraints. Algorithmic bias can also show up if a particular algorithm is used in an environment it isn’t trained for.
For example, consider any automated plagiarism checker that compares over three-word-long strings to a set of content and returns a similarity rate. If the target content is spun and words are changed with synonyms, the algorithm’s rate of accuracy will drop.
Prejudicial Bias:
Prejudice has a massive range. It goes from region-based-terminology differences to deeper discrimination around race, religion, gender, sexuality, etc. Prejudicial bias in a dataset arises because of the perspective of data trainers.
For example, a data labeling team in the UK goes through pictures of women’s wallets and labels them as a purse. For the data’s receptors in the USA, that model will create ineffective results because a ‘purse’ in the States refers to a woman’s handbag, not a wallet.
Measurement Bias:
Measurement bias occurs due to an issue in data measurement or collection. Such deformation can be a result of misclassification, different tools used for data capture, or basic annotation mistakes.
For example, consider an image annotation service provider working on a project to label vehicles for an automated driving project. If the dataset only has images of vehicles on a road but no pedestrians or stray animals, the trainer will have nothing else to label. The final labeled data and the training it imparts to an ML Model will be very ineffective.
Exclusion Bias:
During cleaning, several features of the dataset are weighed and the unimportant ones are often removed. However, if we assign low importance to any feature that is otherwise significant, we may end up creating a dataset with a few underrepresented sections, leading to a prejudiced model.
For example, consider a collection of pre-menopausal symptoms from women- 90% White and 10% Hispanic. It is likely that the 10% are ignored because their symptoms aren’t recognized as mainstream. The resultant algorithm will be less likely to diagnose Hispanic women in need of assistance.
Recall Bias:
Recall bias is also included in data during collection. If the data contributor offers an estimated value instead of an exact one, it will shake the overall accuracy of the set. So, we can consider it a type of measurement bias.
But, an example of direct recall bias in data annotation can be observed in cases where the trainer's label data based on an approximation and the inconsistency is repeated enough times to render the data unreliable for a certain feature.
Observer Bias:
It is possible for a trainer to project their views about a particular data feature into the dataset during labeling, leading to observer bias.
Two people labeling the same set of images can come up with different results. For example, despite the instructions to label the image as boats or not boats, a trainer may label one boat as a yacht and another as a ship, throwing off the consistency in the resultant dataset.
Observer bias is also a very common occurrence with text annotation services. The documents we present to a vendor may fall into the task list of different people across regions. They may perceive handwriting differently and thus create a striking inconsistency across the dataset.
Representation Bias:
If we take two similar situations and predict the outcome of one based on that of another owing to nothing but the similarity, that would lead to a representation bias. Representation bias is personified in the popular phrase-” correlation does not imply causation.”
For example, if an algorithm assesses students’ grades and predicts that those with similar grades will get into a particular college because a few of them did, that prediction will have a low accuracy rate.
At the same time, if we solely train an algorithm on a dataset that does not have equal representation for all the groups in the algorithm’s intended environment, it will also cause a bias. For example, if we train an algorithm on employees' faces with a dataset of white males only, it will have trouble detecting darker skin color or women with shorter hair, etc.
Identifying & Handling Bias In Training Datasets For Machine Learning
These examples prove that biases are diverse in nature and can enter a sample set at any point in its lifetime. Many biases are not evident. Many are so intermingled with other processes that identifying them seems an unmanageable challenge.
There are several proposed techniques to build fairness-aware machine learning models. Many approaches, each following different contexts, have been developed and researched to reduce bias in training data for machine learning. Most of them boil down to one thing- being careful and curious at every step is critical.
1. Question The Preconceptions
A machine learning model learns from historical decisions and their intent, where the intent is known. Therefore, if a decision-maker in the past showed any prejudice in their recorded decisions, the ML model will be at risk of reflecting them.
If during data annotation, a trainer labels pictures of Siberian Huskies and Swedish Vallhund as wolves, the model will learn to do the same. If the last 20 recruitments in a company were for 19 men and 1 woman, the ML model learning from this decision-data is likely to discard more resumes from women than men.
Therefore, at every stage of training data preparation, it is important to question where the data is coming from, whose perceptions affected earlier decisions, and what changes need to be made in the data accordingly to clean it for training purposes.
2. Weed Out The Bias
This idea is challenging; its implementation is more so.
Weeding out biases requires a nearly exhaustive understanding of preconceptions that may have polluted the data. But, depending on the objective of a machine learning model, rooting out potential bias-causing perceptions can range from alright to very tricky.
You can note issues in organizational processes and fix them as well. But, doing so when training a model on multiple dynamic parameters will pose many new problems. If, for example, your ML model uses social media posts from a hundred people to determine their employability score, it might be unfair to prospects from countries whose native language isn’t English, or who are from underdeveloped countries.
Therefore, despite our best attempts at identifying potential bias and rooting it out, it is preferable to test the conclusion and verify its effectiveness.
3. Do Not Let Go Of Oversight
An algorithm that works for one set of data will not likely work with an extended version of the same data. It might if we keep testing the system with challenger models and verify its predictive accuracy, transparency, and improvement rate.
Despite all that, assuming self-sufficiency for any machine learning model is a mistake. As the data, its intended environment, and target objective change, so will its accuracy rate.
For Now, Bias In Real-World Based Machine Learning Models Will Remain An AI-Hard Problem
Despite several reasonable fairness approaches, machine learning models will remain unsatisfactory in one area or another. It is impossible to expect one model to satisfy multiple dynamic constraints and maintain its prediction accuracy simultaneously. We will need to make a choice based on the context.
However, at present, creating decently fair machine learning training data mostly hinges on data annotation and processing. Hopefully, we will have more concrete ways to create fair datasets in near future.
Also Read:
Why is it Better to Outsource Data Annotation?
Why Outsourcing Video Annotation Services Is Profitable For Businesses?
