Algorithmic Fairness Tradeoffs, explained.
Algorithmic fairness concerns criteria for evaluating how a decision system treats people or groups. Equal selection rates, equal error rates, and equal predictive value are different criteria and can conflict.
Why it happens
A selected set contains true positives and false positives. Its positive predictive value depends both on detection performance and on the outcome frequency in the group. Similar sensitivity does not therefore imply equal predictive value. Choosing a threshold also changes who is selected and who is missed.
Algorithmic fairness involves multiple criteria that can conflict. Equal error rates, equal selection rates, and equal predictive value ask different questions, especially when outcome frequencies differ.
Read the result
Move each synthetic group's threshold and read the exact confusion counts before comparing percentages. The groups have different positive frequencies, not real demographic identities. Undefined predictive value means nobody was selected; it is not a measured zero. The illustration explores threshold tradeoffs and does not prove a calibration impossibility theorem.
A worked example
Screening two invented applicant groups
Group A has 20 actual positives among 100 cases; group B has 60. Both use the stated positive and negative score ranges.
Apply the same threshold and compare the selected true positives and false positives. The identical rule need not produce identical predictive values.
Change a threshold to improve one measure, then check every other measure and the people represented by the error counts before calling the outcome better.
OPTIONAL DEEPER DETAILGo deeper: inside the model
Inside this model
Each group has 100 synthetic cases, with 20 positives in A and 60 in B. Positive scores are evenly spaced from 35 to 95; negative scores from 5 to 65. Scores at or above the group's threshold are selected. Exact confusion counts and derived metrics are displayed, with undefined ratios labelled rather than set to zero.
Where this idea is useful
A practical use
A screening system can have similar sensitivity across groups while selected cases have different success frequencies. Evaluate the consequences and definitions before calling it fair.
A common misconception
“One equal percentage proves that a system is fair.”
Equality in a particular metric does not resolve other consequences or establish a legal or ethical judgment. The metric's denominator, outcome labels, and context all matter.
What this explanation leaves out
- These are invented groups and scores, not demographic data. This threshold illustration does not prove the original calibration impossibility theorem, define legal fairness, or decide which criterion a real deployment should use.
Which fairness measure should I choose?
The board cannot decide that. Consider the decision's purpose, costs of different errors, affected people, and applicable obligations. Make the chosen criterion and its tradeoffs explicit.
When you call a decision rule fair, which outcome, error, and denominator do you mean?
Associated thinkers
Associations marked provisional are awaiting source review.
Further reading
Explore the original research or the teaching reference behind this experiment.