The current intense focus on machine learning’s biases is crucial because the issue is not only real, it’s rooted in machine learning’s essence: ML constructs models based on the data we feed it, and unless great care is taken, that data will reflect existing biases. But, focusing on fairness is not enough when addressing the full range of ethical issues ML raises.
So, both kids get the same number of cookies unless there’s a good reason for one of them to get more. Likewise, gender, race, etc. are irrelevant to people’s qualifications for almost all jobs. But, while unfairness is often easy to spot, deciding on what is fair can remain contentious for generations. Just look at the history of affirmative action in the United States. It turns out that it’s easier to get people to agree that a situation is unfair than to agree on what would be a fair remedy.
There is, in fact, an asymmetry between unfairness and fairness. We shout “That’s unfair!” to rally a cohort to fight an unambiguously intolerable situation. But that situation’s resolution is likely to require tolerating trade-offs, and not only because that’s how people with irresolvable opinions manage to live together. More fundamentally, it’s because of the finitude of resources and of human certainty. ML, to its credit, is forcing us to confront these mortal limitations.
Machine learning works by performing statistical analyses on data to achieve goals its human operators have defined. This requires its human operators to get meticulously specific about what they want out of the system. For example, imagine that only 30% of the applicants approved for a mortgage loan from Acme Mortgage Company are women. On the face of it, that seems unfair. But now imagine only 30% of the total applicants were women. Would a 30% – 70% gender split in approvals be unfair? It depends…
The rise of ML has furthered the recognition among social and computer science researchers that there are many different types of fairness. For example, one obvious model of fairness says that the percentage of women approved for a loan should match the percentage of female applicants. Another says that so long as loans are granted without regard to gender, the results are fair, at least with regard to gender. Another says the approved pool should reflect the social demographics. Another thinks that the process is fair if both genders are chosen based on the same probability that they will not default on their loan. And so on into increasing levels of complexity.
Machine learning forces us to consider these models and others because ML does not come into the world with any idea of what’s right. For that it depends on us. But ML requires us to think in its terms: false positives, false negatives, confidence levels, risk. These turn out to be extraordinarily useful concepts for the contentious discussions that occur after we all agree that, say, it’s unfair to discriminate on the basis of gender.
Maybe most interestingly, once we’ve identified a case of unfairness, the concept of fairness stops being of much help. We are instead thrown into familiar discussions of values. In the case of mortgage loans, we can all agree that we need to do everything we can to keep ML’s outcomes from being affected by gender biases buried deep in its data and algorithms. But after that, we’re going to have to argue about our goals and values. If we improve the gender balance of the approved pool by lowering the confidence threshold for women — e.g., the system has to be 80% confident that a man will pay back the loan but only 70% confident that a woman will — how much risk are we willing to put on the lender? How willing are we to grant loans to people at higher risk of defaulting since that can be personally catastrophic? Does home ownership affect women’s economic and social opportunities more than men’s? How urgent a social goal is it for more women to be able to buy their own homes? There are a plethora of questions to take on.
The specificity of ML requires us to have difficult conversations in which unfairnesses are resolved by arguments not about fairness but about values and trade-offs — ultimately about the sort of world we want to live in. If ML brings us face to face with questions like these, good.