Investigating Bias in Predictive Policing and Criminal Justice
COMPAS (Correctional Offender Management Profiling for Alternative Sanctions) is an algorithmic risk assessment tool used by courts across the United States. Developed by the company Equivant (formerly Northpointe), COMPAS is designed to predict the likelihood that a person charged with a crime will reoffend in the future.
Judges and parole boards use COMPAS scores to make critical decisions about:
The algorithm analyzes responses to a 137-question survey that includes information about:
Based on this data, COMPAS generates risk scores from 1 to 10, with higher scores indicating higher predicted risk of reoffending.
COMPAS and similar risk assessment tools are used in courts across multiple states affecting thousands of defendants each year. The use of algorithmic tools in criminal justice has grown significantly in recent years as courts seek data-driven approaches to sentencing and bail decisions.
In 2016, the investigative journalism organization ProPublica conducted a landmark analysis of COMPAS risk scores in Broward County, Florida. Their investigation revealed significant racial disparities in how the algorithm assessed Black and white defendants.
Black defendants who did NOT reoffend were nearly twice as likely to be incorrectly labeled as "high risk" compared to white defendants:
Impact: Black defendants were more likely to receive harsher sentences or be denied bail, even though they would not actually reoffend.
White defendants who DID reoffend were nearly twice as likely to be incorrectly labeled as "low risk" compared to Black defendants:
Impact: White defendants who would reoffend were more likely to receive lenient treatment from the court.
When controlling for prior crimes, future recidivism, age, and gender, Black defendants consistently received higher risk scores than white defendants.
COMPAS is a proprietary algorithm, meaning:
COMPAS was trained on historical criminal justice data that reflects decades of racial disparities in policing and sentencing. If Black communities have historically been policed more heavily, arrest data will show higher rates in those communities—not necessarily higher rates of actual criminal behavior.
Example: Two people might use marijuana at the same rate, but if one neighborhood is policed more heavily, arrests will be higher there. The algorithm learns this as "higher risk" rather than "higher policing."
While COMPAS doesn't directly use race as an input, it uses many factors that correlate with race:
These "proxy variables" allow racial bias to influence predictions without explicitly considering race.
The system creates a self-reinforcing cycle:
Some questions in the COMPAS survey may disadvantage certain groups:
The bias in COMPAS has had devastating consequences for thousands of Black defendants and their families:
Black defendants incorrectly labeled as "high risk" received longer prison sentences than they would have otherwise, spending years away from their families and communities.
Higher risk scores led to higher bail amounts or bail denial, meaning innocent people (remember: not yet convicted) remained in jail simply because they couldn't afford bail while awaiting trial.
People who had served their time and were ready for release were kept in prison longer based on algorithmic predictions that were often wrong.
Higher risk scores led to stricter probation and parole conditions, making it harder for people to rebuild their lives and increasing chances of technical violations that send them back to prison.
Criminal records (often the result of biased enforcement and sentencing) create barriers to employment, housing, education, and civic participation, affecting not just individuals but entire families and communities.
AI trained on biased historical data will perpetuate and potentially worsen those biases. Past discrimination becomes "prediction" for the future.
Mathematical predictions can appear scientific and unbiased while actually reinforcing inequality. The seeming objectivity of algorithms can make them more dangerous than human decision-making because biases are harder to recognize and challenge.
Proprietary "black box" algorithms used for high-stakes decisions about people's liberty are fundamentally incompatible with justice. People have a right to understand and challenge decisions that affect their lives.
Even without directly using protected characteristics like race, algorithms can discriminate through correlated factors. Careful analysis is needed to identify and remove these proxies.
The criminal justice system has a long history of racial discrimination. Introducing algorithms into this context without addressing underlying biases will perpetuate injustice.
Algorithmic systems can create self-fulfilling prophecies where predictions influence outcomes, which then feed back into the data, reinforcing initial biases.
Not every problem can or should be solved with AI. Decisions about human liberty, dignity, and fundamental rights may require human judgment, context, and accountability that algorithms cannot provide.
Why do you think the COMPAS algorithm produced different results for Black and white defendants? Identify at least two specific sources of bias.
Hint: Think about the data the algorithm learned from and the questions it asked.
Choose one of the real-world impacts described (longer sentences, denied bail, etc.). How might this affect not just the individual, but their family and community?
Hint: Consider economic impacts, children and families, employment, and long-term consequences.
The COMPAS developers argue their tool is "fair" because it's equally accurate for Black and white defendants overall. ProPublica argues it's unfair because error rates differ by race. Who do you think is right? Why?
Hint: There's no single answer! This question explores how different definitions of fairness can lead to different conclusions.
Should courts use algorithms like COMPAS at all? What are the alternatives? What would be the advantages and disadvantages of each approach?
Hint: Consider human judgment alone, using algorithms as one factor among many, or completely different approaches to criminal justice.
If you were on trial, would you want an algorithm like COMPAS to help determine your sentence? Why or why not? Would your answer be different if you were from a different demographic group?
If you were tasked with redesigning COMPAS to be fair, what changes would you make? List at least three specific improvements.