My Machine Learning Model Performs Well, but I Want It to Fail
Exploring data scientists’ ethical responsibility
I am a recent graduate of Metis’ data science bootcamp and in that program one of our projects was focused on building a classification model. We can pick any dataset of our choice to work with and in my case, I chose to use the Center for Disease Control’s National Health and Nutrition Examination Survey (NHANES) results for the years 2007–2016 and train a model to predict whether or not an individual has high blood pressure (binary classification: yes or no). I tested a range of 28 variables in my machine learning model from demographics, eating habits, alcohol intake, activity level, occupation, and others. After testing various models, I selected a logistic regression model using 5 variables, oversampling, and setting a threshold of 0.34 to bring my target metric of recall to 0.899. I built a nice Tableau graphic to represent my findings and with this, I had technically achieved my objective: classification model complete. Check.
To those in the data science community, this likely isn’t a ground-breaking exercise. To those outside the data science community, you likely are thinking (if I haven’t already lost you), “I don’t know what a logistic regression model or any of the things you said after that are.” Well, regardless of what audience you’re in, I get it. But the technicalities of my model isn’t what I want to talk about.
I want to talk about why I want my model to fail.
Why?
Well, four of the five variables in my model were somewhat expected. First was age, which is unfortunate but inevitable. The higher your age, the higher your chances of having high blood pressure. Then increase in weight, alcohol intake, and cigarette smoking all also increased chances of predicting high blood pressure, which can be hard but can be controlled or changed by an individual.
But it’s the fifth variable of the model that threw me. The variable was a particular minority race — if an individual was of this race they would have higher blood pressure. I don’t feel the need to say the race here, as I don’t want this piece to be about me (white woman) talking about another race. That’s not my place and not a platform I deserve. If you really want to know, it’s not hard to find out with a quick google, as I found out after I saw my model’s results and searched for what put people at risk of high blood pressure to check my variables against (and hoped would show me why my race variable was wrong).
What I do want to talk about is how to feel about that race variable in my model and what responsibility a data scientist has in a situation like this. There is a possibility that there is a genetic connection between the race and high blood pressure — which is sad in itself — but there’s also a possibility of it being representative for many other societal things that this race faces that could lead to high blood pressure. And that’s….really sad. Even if not the case for this particular model, this same line of thinking could be applied to any model where race appears as a variable due to historical oppressions of that group.
What does a data scientist do in a situation when they see results of their model that represent greater societal issues?
Some might believe it’s not the responsibility of the data scientist — that it’s the responsibility of others to address and the data scientist just builds the thing. I don’t quite agree with that and feel an obligation to do more, but I also don’t know what that looks like. So I wanted to raise the question here because what I do believe this should warrant a larger conversation. For me, it’s hard to just hand this model off as “done” and move on. I was proud of my technical work on the modeling, but can’t say I was proud to present the findings. I was sad, mad, and uncomfortable.
Maybe my machine learning model was simple, but the moral question it raised is complex.
I encourage and hope for comments on this article. As mentioned before, I’m not here with answers; I’m here to start a conversation and hope you join in so we can all work towards solutions.
Full code for this model can be found here on github.
