A new study shows how AI can influence the kind of innovation you end up funding.
Many of us have started using AI as an “answer machine” to brainstorm ideas, analyze data, and pressure-test assumptions. You might even have a set of preferred prompts saved for just these purposes. But how often do you switch the order of the steps you give the AI, and what kind of influence does it have on the output? In “The Mean-Variance Innovation Tradeoff in AI-Augmented Evaluations,” a team including D^3’s Jacqueline Ng Lane shows that when organizations integrate AI into multi-stage innovation evaluation processes, the sequence creates a powerful but largely invisible tradeoff. By understanding how to structure this dynamic, we can gain a crucial advantage when identifying ideas that could bring value to our portfolios and organizations.
Key Insight: Navigating Between Novelty and Feasibility
“Our focus on sequencing is grounded in the observation that evaluators naturally rely on criteria-sequencing, a heuristic involving the prioritization of alternative criteria at different evaluation stages.” [1]
Evaluating innovation is a high-stakes balancing act between two competing forces: novelty and feasibility. You want solutions that depart from established approaches (novelty) and that can realistically be built and implemented (feasibility). But as the authors note, evaluators can’t weigh everything simultaneously, so they prioritize one over the other at different stages of the process (criteria-sequencing): either novelty-then-feasibility or feasibility-then-novelty. These sequences lead to different results because the order acts as the initial filter. If a solution is eliminated in the first stage based on one criterion (e.g. feasibility), it is never evaluated on the second (e.g. novelty). Since evaluators apply these criteria in personalized ways, the order they use can lead to inconsistent decisions.
Key Insight: An AI Innovation Experiment
“AI recommendations operate much like ‘spotlights on a stage’: they illuminate certain aspects of a solution while leaving others in the dark, subtly structuring the order and weighting of the cues evaluators consider.” [2]
To see how AI could structure these heuristics, the researchers partnered with the crowdsourcing platform Hackster.io for a field experiment involving 353 evaluators and 132 open-source solutions. They utilized two distinct types of AI: Predictive AI and Generative AI. Predictive AI, which excels at identifying patterns from past data, was used to provide feasibility recommendations based on technical benchmarks. Generative AI, capable of recombining knowledge in unconventional ways, provided novelty-focused recommendations. Both systems provided “Pass” or “Fail” recommendations with explanatory content, with half of the evaluators receiving feasibility-then-novelty sequencing, and the other half receiving novelty-then-feasibility. The researchers predicted that the criteria-sequencing would create what they call a mean-variance innovation tradeoff: feasibility-then-novelty would allow evaluators to take greater risks with fewer options, resulting in higher mean innovation, while novelty-then-feasibility would cast a wider initial net, surfacing atypical solutions and producing higher variance.
Key Insight: Tradeoffs in the Pursuit of Breakthroughs
“Overall, our experimental results provide compelling evidence of a mean-variance innovation tradeoff.” [3]
The results supported the researchers’ predictions, meaning the order of evaluation dictates the type of innovation an organization is likely to champion. The researchers also found in post hoc analysis that the AI’s format played a role: compared to a static summary, an interactive chatbot increased innovation variance, but led to a lower mean innovation rating. It appears that without a fixed, standardized summary to guide them, evaluators spent more time exploring diverse questions and ultimately relied more on their own judgment. As a result, evaluations became more complex, average quality declined, and the set of selected options became more diverse.This suggests that dynamic “thought partners” encourage more exploration and reliance on human judgment, while static AI recommendations act more as rigid guides.
Why This Matters
For business leaders and executives encouraging their employees to use AI-augmented workflows, this research fundamentally reframes the integration question. It’s not just about whether to use AI, or even which tasks to automate versus augment, it’s about recognizing that AI recommendations create structure that shapes human judgment in path-dependent ways. The sequence you choose could determine whether your organization builds a portfolio optimized for steady performance or one that swings for breakthrough innovation. The question isn’t whether AI will influence your decisions, it’s whether you’ll deliberately design that influence, or let it emerge accidentally from your initial prompt.
Bonus
For another look at how AI can shape outcomes by steering what kind of ideas people generate and select, check out “The Creative Edge: How Human-AI Collaboration is Reshaping Problem-Solving.”
References
[1] Grumbach, Cyrille, Jacqueline N. Lane, and Georg von Krogh, “The Mean-Variance Innovation Tradeoff in AI-Augmented Evaluations,” Harvard Business School Technology & Operations Mgt. Unit Working Paper No. 26-038 (2025): 1. https://ssrn.com/abstract=5933495
[2] Grumbach et al., “The Mean-Variance Innovation Tradeoff in AI-Augmented Evaluations,” 2.
[3] Grumbach et al., “The Mean-Variance Innovation Tradeoff in AI-Augmented Evaluations,” 33.
Meet the Authors
Cyrille Grumbach is a PhD Candidate and Research Associate at the Chair of Strategic Management and Innovation at ETH Zurich.
Jacqueline Ng Lane is Assistant Professor of Business Administration at HBS and co-Principal Investigator of the Laboratory for Innovation Science at Harvard (LISH) at the Digital Data Design Institute at Harvard (D^3).
Georg von Krogh is a Professor at ETH Zurich and holds the Chair of Strategic Management and Innovation.