The world’s leading publication for data science, AI, and ML professionals.

Police, Traffic Stops, Data, and Race

Data for Change

Exploring the difference in inequity between two Californian metropolises through police traffic stoppage data.

Credit to Andrey Pristinsky for help researching this project.

Photo by Oliver Hale on Unsplash
Photo by Oliver Hale on Unsplash

At 1:30 pm, April 20th, 2021, ex-police officer and now convicted murderer Derek Chauvin was found guilty of second-degree unintentional murder, third-degree murder, and second-degree manslaughter, following an incident that led to the death of the victim, George Floyd. This death continues to spark widespread public backlash and has been the precursor to international movements, and calls for Police Reform. This conviction, a hallmark case for many reasons, underpins a zeitgeist shared by many citizens regarding the methods and actions of American policing institutions.

There is significant research into the sociological roots of bias in policing, however, for the purposes of this article, I will focus on the data and explore inequality in traffic stoppages between two major Californian cities, "America’s Finest City" San Diego and "The Golden City" San Francisco.

As a result of my research and examination of said statistics, my findings highlight a seemingly significant racial disparity. Hispanic Americans are 2.86% and 4.60% more likely to be searched than their White counterparts in San Diego and San Francisco. For Black drivers, the percentage increases to 6.38% and 13.10% in the respective cities. This discrepancy in stoppage searches is despite no increased chance of finding contraband or arresting the driver, with both Black and Hispanic drivers exhibiting lower rates of contraband possession or arrest following search.


Experimental Setup

In conducting this statistical examination, it is important to ensure that the data collected is accurate and comprehensive, so as to not misrepresent an issue as delicate and contentious as racial bias in the US policing system. Fortunately, Stanford University’s Open Policing Project takes much of the burden of data collection out of our hands. Compiling over 200 million records of local and state police departments, the project aims to provide an accurate and comprehensive record of traffic stoppages in the United States. Included in this dataset are the two key cities in which we are focusing our study:

San Diego with **** 383,027 stoppages from Dec 2013 to Mar 2017.

San Francisco with **** 905,070 stoppages from Dec 2006 to Jun 2016.

Using the intersection of these two data sets we get roughly 300,000 stoppages for San Diego and 200,000 for San Francisco at a combined count of 525,581 for January 2014 through July 2016. This data includes 21 parameters that range from vehicle types to subject demographics and searches to arrests issued.

Parity

In order to thoroughly explore racial inequality between these two Californian metropolises, a new measure for evaluating the notion of fairness applied in decision-making systems must be introduced. This parity measure, as it is defined, represents a simple observational criterion and helps explore how decision systems such as police traffic stoppages may be biased. Specifically, this article will focus on the Demographic Parity defined in the formula below.

Demographic Parity as a formula where: C is classification, A is salient group (Race), and a/b represents groups within the salient group. Credit: Aaron Fraenkel. Reposted with Author's permission.
Demographic Parity as a formula where: C is classification, A is salient group (Race), and a/b represents groups within the salient group. Credit: Aaron Fraenkel. Reposted with Author’s permission.

In the context of police stoppages, satisfying this parity measure doesn’t mean the decision-making systems, police officers, are fair, but rather a parity measure represents a lens to view the results though that can help illuminate the biases within the observed system. More specifically demographic parity represents an equal proportion of searches conducted, following a traffic stoppage, when adjusted for race. This measure, coupled with traditional metrics for evaluating the performance of a classifier, helps quantify the successes or failures of two similar cities with vastly different police institutions.


What Did We Learn?

Based on the data sourced, there are a number of interesting results to explore, discuss and understand. It is worth noting that whilst there may be bias in police stoppage rates, this article analyses the bias that occurs after the car has been stopped because, whilst there may be bias in who the police stop in the first place, in many cases the model, shape, age and other factors of the car make it impossible for the police to know with certainty the race or ethnicity of the driver they are stopping. Once their car is stopped and the police officer comes face to face with the driver, then conscious or unconscious biases can manifest with a sufficient level of certainty for analysis. With this in mind, figure 1 highlights the percentage breakdown of police stoppages by city and by race/ethnicity.

Figure 1: Stoppage breakdown for San Diego and San Francisco by race/ethnicity. Image by author.
Figure 1: Stoppage breakdown for San Diego and San Francisco by race/ethnicity. Image by author.

As depicted in figure 1, San Francisco has a more diverse driving population when compared to San Diego, which has a majority of its drivers being either White or Hispanic. For the context of this analysis, figure 1 serves to represent the racial or ethnic breakdown for the on-road drivers in each city because it is hard to quantify bias in the driving-to-stoppage phase. What is of stringent scientific note is what happens after the traffic stoppage. Does the police officer choose to search the car or allow the citizen to continue with their business? Figure 2 highlights the percentage breakdown of searches conducted following the stoppage in relation to race/ethnicity.

Figure 2: Search breakdown following stoppages for San Diego and San Francisco by race/ethnicity. Image by author.
Figure 2: Search breakdown following stoppages for San Diego and San Francisco by race/ethnicity. Image by author.

As evidenced in figure 2, the proportion of Black drivers searched is significantly higher than their racial or ethnic counterparts. It is evident that there is a disparity in search rates across both the San Diego and San Francisco police departments. This disparity is most exaggerated in Black drivers, however, similar trends are also present for Hispanic drivers.

Given the stark contrast in search rates amongst both cities, one could argue that Black or Hispanic drivers might be more likely to be searched for a reason, which could range from suspicion of possessing contraband, to committing an active crime such as grand theft auto. This argument falls short when considering the arrest rate, or "hit rate", following a stoppage as seen in figure 3.

Figure 3: Arrests following stoppage for San Diego and San Francisco by race/ethnicity. Image by author.
Figure 3: Arrests following stoppage for San Diego and San Francisco by race/ethnicity. Image by author.

Black drivers, in both San Diego and San Francisco, were less likely to be arrested following a search at just 20.58% and 15.15% respectively. This is in stark contrast to White drivers which had a 42.00% and 33.38% chance to be arrested depending on the city. As an aggregate arrest rate, San Diego county outperforms San Francisco, arresting 30.10% of searched drivers compared to San Francisco at 21.31%. This trend can also be observed when considering contraband hit rates following a vehicle stoppage and search as seen in figure 4.

Figure 4: Contraband found following stoppage for San Diego and San Francisco by race/ethnicity. Image by author.
Figure 4: Contraband found following stoppage for San Diego and San Francisco by race/ethnicity. Image by author.

Both Black and Hispanic drivers again represent the two demographics with the lowest hit rate for contraband possession, with White and Asian/Pacific Islander drivers being marginally more likely to offend in San Diego county and significantly more likely to offend in San Francisco in relation to possessing contraband. Furthermore, the hit rate for contraband possession in San Francisco is double that of San Diego at 18.21% versus 9.16%.

The disparity between Black or Hispanic drivers and White or Asian/Pacific Islander drivers raises an interesting question; are White or Asian/Pacific Islander drivers more likely to possess contraband or commit a crime than their Black or Hispanic counterparts? The obvious answer is no. The discrepancy, rather, underpins an inequity that permeates both the San Diego Police Department and the San Francisco Police Department.

Demographic Parity

The demographic parity can be evaluated from the calculated prevalence, which is the decision to search by race/ethnicity over total population by race/ethnicity. This is comparable to figure 5, where a truly fair classifier would equally distribute searches across race/ethnicity according to population hit rates. In figure 5, a red reference for San Diego and a Blue reference for San Francisco have been included to exemplify what a totally equal search rate would look like.

Figure 5: Arrests following stoppage for San Diego and San Francisco by race/ethnicity including an equity threshold for each city. Image by author.
Figure 5: Arrests following stoppage for San Diego and San Francisco by race/ethnicity including an equity threshold for each city. Image by author.

This disparity in search rates suggests that when choosing to search White or Asian/Pacific Islander drivers, officers rely more on overwhelming suspicions or significant evidence, hence, the higher arrest and contraband rates seen in those demographics. This contrasts to the Hispanic and Black demographics which are searched more often, yet arrested less, suggesting the existence of an additional confounding factor influencing the police officers, such as a conscious or unconscious bias towards searching said demographics.

This finding mirrors the findings of similar studies conducted on San Diego and San Francisco individually, such as the study "Traffic Enforcement Through the Lens of Race," an SDSU study on SD police, which found that:

post-stop disparities evident in our analysis suggest that implicit bias is present in officers’ decision-making

San Diego and San Francisco

Summer in "America's Finest City", San Diego. Photo by Ameer Basheer on Unsplash
Summer in "America’s Finest City", San Diego. Photo by Ameer Basheer on Unsplash

It is clear that for both San Diego and San Francisco there are levels of police bias in traffic stoppage searches, likely underpinning larger biases across the police departments. Part of the reason why these county biases were not explored in isolation and instead in tandem is that by exploring how these contrast to each other, possible solutions to this inequality can be inferred. When compared to San Francisco, San Diego does a better job of searching the "right" car, being cars that contain a reason for arrest. San Diego performs these searches at a hit rate that is almost 10% higher than San Francisco, despite doing proportionately fewer traffic stoppage searches at 4.27% compared to 4.56%. Furthermore, whilst not optimal, there is significantly less bias in how San Diego performs these searches, discriminating less by race than San Francisco County.

Limitations of Study

Whilst many efforts were made to explore the underlying bias in police stoppage searches, there are several limitations to this research worth mentioning. Such limitations include, but are not limited to:

  • Data reporting: Did the officer correctly infer race and record the encounter correctly?
  • Parole as a factor: Officers are encouraged to search vehicles regardless of suspicion if the driver is currently on parole.
  • Reason for search: Reasonable suspicion is required for an officer to search the car, ideally this could have been adjusted for in the results however many officers chose not to record a reason for why they engaged in a search.
  • Old data: The data used is archaic (2014–2016) and many of these trends may already be understood and on the path towards improvement.

Final Thoughts

The Golden City, San Francisco. Photo by Chris Leipelt on Unsplash
The Golden City, San Francisco. Photo by Chris Leipelt on Unsplash

As evidenced in both data collected from San Diego County police stoppages and San Francisco County police stoppages, an underlying confounding variable relating to the disproportionate number of Black and Hispanic drivers searched, is present. This variable, as shown, isn’t related to the rates of contraband discovery or arrests, but rather likely related to conscious or unconscious biases in the police officers themselves. Through looking at these two cities in tandem, we can see how San Diego, whilst still exhibiting signs of bias, performs significantly better at picking the "right" cars to search whilst also exhibiting lower levels of racial bias. This exploration isn’t without limitations, however, as much of the data is inconsistent or incomplete. Furthermore, as we have tried to explore through the article, police stoppages and racial biases are immensely complicated issues with a myriad of caveats, nuances, and confounding factors that are sometimes not referenced in the data. This exploration of police stoppage searches pertaining to driver race represents an analytical dive into the data, not necessarily the situation on the ground. With that said I hope that in exploring this topic I can raise awareness for these underlying biases and also encourage the more robust collection of data by US county police departments in an effort to better explore, analyze and amend the disproportionate racial prejudices in policing institutions that are currently creating shockwaves through communities.


Sources

Chanin, J., Welsh, M., & Nurge, D. (2018). Traffic enforcement through the lens of Race: A sequential analysis of Post-Stop outcomes in San Diego, California. Criminal Justice Policy Review, 29(6–7), 561–583. doi:10.1177/0887403417740188

Fairness & algorithmic decision making. (n.d.). Retrieved May 10, 2021, from https://afraenkel.github.io/fairness-book/content/03-harms.html

Fairness & algorithmic decision making. (n.d.). Retrieved May 10, 2021, from https://afraenkel.github.io/fairness-book/content/05-parity-measures.html

Levenson, Eric. Derek Chauvin Found Guilty of All Three Charges for Killing George Floyd. 21 Apr. 2021, www.cnn.com/2021/04/20/us/derek-chauvin-trial-george-floyd-deliberations/index.html.

The Stanford Open policing project. (n.d.). Retrieved May 10, 2021, from https://openpolicing.stanford.edu/data/

Racial disparities in california law enforcement stops. (2020, December 03). Retrieved May 10, 2021, from https://www.ppic.org/blog/racial-disparities-in-california-law-enforcement-stops/

All images used are either created by myself or used with the explicit permission of the authors. Links to the author’s material are included under each image.


Related Articles