The Model Room
Where this idea comes from
When “fair” means two different things
In 2016 the newsroom ProPublica studied COMPAS, a score used in some US courts to estimate whether a defendant would re-offend. It found Black defendants who did not re-offend were wrongly flagged high-risk about twice as often as white defendants. The company that made the score answered that it was fair in another sense: among people given the same score, re-offending rates were about the same for both groups.
One thing to hold: A short mathematical result. In 2016 and 2017, Jon Kleinberg, Sendhil Mullainathan and Manish Raghavan, and separately Alexandra Chouldechova, showed that when two groups have different base rates, a score cannot be calibrated for both groups and also have equal false-positive and false-negative rates for both, except in special cases.
Why it matters: Both sides were measuring something real. A fairness definition is a choice about which mistake matters most, which is why this lab asks you to pick a threshold and say what it costs.
What we don’t know: Which definition should a court use is not a maths question, and people still disagree. The studies also argue about the data itself: an arrest record is not the same as a crime.
Dates:
- 2016 ProPublica publishes its COMPAS study; the maker replies.
- 2017 Two papers prove the fairness definitions cannot all hold at once.