We live in an age where the call for transparent or “Explainable AI” (XAI) has never been louder. Businesses agree, with 85% believing transparency is critical to winning consumer trust. [1] Given this consensus, it seems reasonable to assume that when an explanation for a high-stakes AI decision is available, people will naturally seek it out to improve their results, ensure compliance, or simply satisfy their curiosity. Yet, in the new paper “Preference for Explanations: Case of Explainable AI”, Alex Chan, Assistant Professor of Business Administration at Harvard Business School and Associate at the Digital Data Design Institute at Harvard (D^3), shows that we’re happy to lean on AI’s predictive power, but much less eager to confront what those predictions might reveal about bias, fairness, or our own choices. His study, centered on loan allocation decisions, reveals an uncomfortable truth: when financial incentives clash with fairness concerns, people don’t just make questionable decisions, they actively avoid information that would force them to confront those choices.
Key Insight: Seeking Predictions While Avoiding Explanations
“People want to know how AI makes decisions—until knowing means they can no longer look away.” [2]
In the main experiment, participants acted as loan officers for a private U.S. lender deciding how to allocate a real, interest-free $10,000 loan between two unemployed borrowers. An AI classified one borrower as low risk and the other as high. Participants could see the AI’s prediction and, in many conditions, they could choose whether to see an explanation of how the model reached its risk assessment.
Roughly 80% of participants opted to see the risk scores, but only about 45% chose to see explanations when given the chance. When their bonus was aligned with the lender (they earned more if loans were repaid), participants were more likely than others to seek the prediction, but significantly more likely to avoid explanations, especially when they were told those explanations could involve race and gender. In one condition that made fairness auditing salient, lender-aligned participants were about 10 percentage points more likely to skip explanations than neutrally paid participants.
Crucially, this avoidance wasn’t about disliking extra information in general. When demographic information was removed and replaced with arbitrary details, the gap in explanation-avoidance between incentive conditions almost vanished. People weren’t shunning explanations as such, they were avoiding what the explanation might force them to confront about discrimination and their own profit-maximizing behavior.
Key Insight: Systematic Underevaluation
“[E]xplanations are systematically under-demanded because individuals fail to anticipate their complementarity with private information.” [3]
To separate moral self-image from pure decision quality, a second experiment removed fairness trade-offs and focused on prediction accuracy. Participants evaluated a loan labeled “high risk” by an AI, potentially due to a two-year employment gap. They first stated their willingness to pay (WTP) for an explanation revealing whether the gap was the driver of the high risk label. Crucially, participants then received free private information explaining that the gap actually resulted from pursuing a full-time professional certificate (benign towards risk), and not a termination (increasing risk) as would commonly be assumed. This private information made the purchased explanation more valuable, a concept the paper calls “complementarity,” because if participants knew that the high-risk AI label resulted from the employment gap, then the addition of the private information told them that the AI label was not to be trusted. In other words, the participants should integrate the private information with the explanation to form a more accurate assessment.
Yet, when WTP was elicited a second time, after participants received this related private information, valuations dropped 25.6%. Valuations only increased (by 23.7%) when participants were explicitly guided through the complementarity logic. This represents a novel behavioral bias: people systematically fail to recognize when explanations would help them integrate their own knowledge with algorithmic outputs.
Why This Matters
For business professionals and executives, this research is a warning that deployment of AI is not purely a technical challenge, it’s also a behavioral one. In high-stakes decisions like credit, hiring, pricing, healthcare, and safety, your employees could eagerly consume AI predictions while quietly avoiding the explanations that would expose uncomfortable trade-offs or discriminatory patterns. That avoidance can skew outcomes, undermine fairness, and create hidden risk. At the same time, teams may systematically under-invest in explanations even when they would improve forecasting by helping experts combine their own domain knowledge with AI outputs. The bottom line: investing in transparent AI systems is insufficient. You must also architect the decision environment and incentive structures that ensure transparency gets used rather than ignored.
Bonus
If you’re interested in how explanation avoidance fits into a broader pattern of human and AI collaboration challenges, Persuasion Bombing: Why Validating AI Gets Harder the More You Question It shows that when professionals do try to validate model outputs, AI can respond by pushing back and working to persuade users to accept its answers. Or if you’re thinking about the governance implications of explainable AI, Evidence at the Core: How Policy Can Shape AI’s Future argues that regulators should insist on robust evidence and transparency, from pre-release evaluations to post-deployment monitoring, so that organizations can’t simply offer explainability features on paper while leaving them unused in practice.
References
[1] Chan, Alex, “Preference for Explanations: Case of Explainable AI,” Harvard Business School Working Paper No. 26-028 (December 5, 2025): 2. https://www.hbs.edu/faculty/Pages/item.aspx?num=68104
[2] Chan, “Preference for Explanations,” 2.
[3] Chan, “Preference for Explanations,” 7.
Meet the Author
Alex Chan is Assistant Professor of Business Administration at Harvard Business School and D^3 Associate. He is an economist interested in how market failures occur, how such failures lead to divergence in economic outcomes, and how to design incentives and engineer markets to remedy these market failures.