On May 7, the Digital Data Design Institute at Harvard hosted Leading with AI: Exploring Business and Technology Frontiers. The conference featured breakout sessions where HBS staff and industry experts discussed specific and specialized topics. The breakout group on The Role of GenAI in the Design Innovation Process featured Firdosh Tangri, CEO of Fantasy; Roger Rohatgi, VP, Global Head of Design at BP; and Doug Ng, Head of Design at the HBS Digital Data Design (D^3) Institute and session moderator; Professor Jackie Lane, HBS Assistant Professor in the Technology and Operations Management unit; and Professor Michael Parzen, Senior Lecturer in the HBS Technology and Operations Management unit.
In his opening remarks, Ng emphasized the importance of keeping humans at the center of design efforts, especially related to technology-driven research and transformations.
Definitions and Questions
To start the session, Ng asked the question: How do you use AI to augment the design innovation process? For context, he defined “design” as a way of thinking and approaching business problems, and “augmentation” as the intersection of what machines are good at and what humans are good at. He then posed another question: Are empathy and creativity human skills or can machines do them?
In some studies, AI was able to develop better ideas in industry, better advice in personal situations, and better accuracy and bedside manner in healthcare settings. How do these findings apply to design innovation?
Key Insight: Understanding Customers
“…we need to accelerate the way that we’re doing design here at [BP]…[All our companies] need better experiences and they all need to accelerate how they reach their customers or how we reach the world…”
Roger Rohatgi
Rohatgi talked about bringing human-centered design concepts and AI to BP, which was making a big digital transformation and an enormous energy transition to achieve net zero. BP worked with Ng and Tangri from Fantasy to consider how to get closer to customers. Using research combined with AI, they used human mimicry by building digital twins of their customers, or what they called “synthetic humans.” Synthetic humans were used to understand and test customer situations and experiences.
Insight: Using AI for Deeper Insights
“…human feedback is really critical to take the concepts to the next level.”
Firdosh Tangri
Tangri spoke about Fantasy’s work with BP on an Innovation Sprint to identify BP’s “North Star vision” to transform the company using a human-centered ideation process augmented by AI. Tangri helped BP combine the Innovation Sprint with AI to “harness the power of AI to craft a potential net-zero vision for the future of sustainable smart cities and on-demand convenience and mobility.”
The Innovation Sprint process normally takes 10 to 12 weeks, but with AI, it took 12 days. The process consists of:
- Research review to understand the company
- Empathy opportunity identification, or future world analysis
- Ideation and concepting
- Validation and refinement
The research and world analysis steps involve a comprehensive review of documents (research, articles, filings, annual reports, patents, etc.). The team used large language models (LLMs) to derive insights and ideas from this huge number of documents. With added human context, the team fed the data back to the LLMs to expand on the revised ideas. Tangri called it a “true AI-human collaboration.”
Key Insight: Augmenting User Testing
“We developed a platform called synthetic humans. I know it sounds scary, but this is just a visual aspect of it. We’re not building robots or anything of the sort…we were trying to figure out how can we not replace traditional user testing with real humans, but how can you augment it so that we’re doing it at least more often?”
Firdosh Tangri
Fantasy helped BP test concepts by augmenting real humans with synthetic humans (versions of human testers) to run the same studies. Synthetic humans have over 100 psychographics—personality, speech, temperament—plus additional attributes, such as interests and hobbies, that enable them to respond in human ways. This creates empathy. Fantasy uses synthetic humans for:
- User testing
- Qualitative studies
This approach helps with divergent creative thinking by sparking new ideas and adding different perspectives.
Key Insight: Solving Problems with Generative AI
“[Can AI] be creative on open-ended, messy challenges?”
Jackie Lane
In a similar but unrelated study, Professor Lane aimed to test AI’s problem-solving abilities by using a crowdsourcing challenge on the circular economy, or an economic system that emphasizes reuse and regeneration of materials in an environmentally sustainable way. This problem was a good experiment because it is messy, doesn’t have clear-cut solutions, and will require technological innovation to solve.
The challenge gathered ideas from humans through Freelancer, and compared them to prompt engineering AI results from ChatGPT. Expert human evaluators judged thousands of ideas on creativity—novelty (originality) and value (potential financial impact). The result was that AI-generated ideas matched the creativity of human ideas. The most novel idea came from a human (construction innovation bricks). The most valuable idea came from AI (biogas from food waste).
AI did the work in 5.5 hours, at a direct cost of $27.01. Humans did the work over 2,520 hours at a cost of $2,555.
Lane concluded that “What AI is enabling us to do is to basically recalibrate human screeners or human works because we’re not used to seeing such high-quality ideas. It’s essentially making us more stringent in our decisions because it’s coming up with recommendations that are more stringent than what we would have necessarily thought in this pre-ChatGPT, pre-generative AI world.”
Applications
- Prompt engineering is important. Using AI, constraints can produce generic ideas, but blue-sky prompts can produce out-of-category ideas.
- If you get obvious answers, tweak your questions. Or if ideas are too generic, prompt AI to avoid considering certain solutions and add context to push AI to come up with more and different ideas.
- You need the human element to filter ideas.
- Interdisciplinary skills are transferable—concepts can drive preliminary designs and tests and different team members can influence how they can change and work together.
- Fine-tune human user testing with synthetic humans.
- Show preliminary visual concepts to synthetic humans and ask for feedback.
- When using AI for idea generation, have humans develop novel ideas, and use AI to refine the ideas for value. But be aware of two problems: (1) The number of ideas increases, maybe beyond human ability to process them; and (2) We are limited in resources, so we can only fund, mentor, and train the best ideas.
Professor Mike Parzen closed the discussion with a quote found on the blackboard of Richard Feynman, the American theoretical physicist, after his death, which said: “Know how to solve every problem that has been solved.” He described this as the precursor to ChatGPT. Why spend your time answering old questions? Start from the answers and build up. But while that’s easy to do with general physics knowledge, it’s very hard to do with creativity.
The session ended with questions and conversation around bias in humans and LLMs, “aha” moments while using AI as part of the business process, and the definitions of experience and memory. Ng ended the session by asking: “What does empathetic and creative AI mean for us at the end of the day?”
Meet the Speakers
Jacqueline (Jackie) Lane is an Assistant Professor in the Technology and Operations Management Unit at Harvard Business School and a co-Principal Investigator of the Laboratory for Innovation Science at Harvard (LISH) at the Digital Data Design Institute (D^3) at Harvard. Jackie earned a bachelor’s degree in operations research and financial engineering from Princeton University, an MBA from Columbia Business School, and a PhD from Northwestern University. Most recently, she was a postdoctoral fellow at Harvard Business School and the Laboratory for Innovation Science at Harvard.
Doug Ng is currently the Head of Design at the Digital Data Design (D^3) Institute at Harvard Business School. He is a graduate student in the Instructional Technology & Media program at Columbia University and earned a Bachelor of Business Administration in Management as well as Bachelor of Arts in Journalism (Telecommunication Arts-Broadcast Management) from the University of Georgia.
Michael Parzen is a Senior Lecturer in the Technology and Operations Management unit at Harvard Business School and was Co-Director of Undergraduate Studies and Director of the Master’s Program in Statistics for Harvard University’s Department of Statistics. Professor Parzen holds ScB and ScM degrees in Mathematics from Carnegie Mellon University, a ScM degree in Applied Mathematics from Brown University, and a ScD in Biostatistics from the Harvard School of Public Health, and a PhD in Biostatistics from Harvard University.
Roger Rohatgi is a VP, Global Head of Design at BP, where he is working on an initiative called #NetZeroUX, looking to decarbonize designs and design for decarbonization. He received his BA in Media Policy from the University of Houston.
Firdosh Tangri is the CEO of Fantasy, an experience design company. He earned his bachelor’s degree in Computer Science and Economics from Purdue University and completed executive education courses in digital strategies, accounting and finance, blockchain, and design thinking from Columbia Business School, the Saïd Business School at University of Oxford, and the MIT Sloan School of Management.
Additional Resources
Jackie Lane:
- Teams in the Digital Workplace: Technology’s Role for Communication, Collaboration, and Performance (article)
- Setting Gendered Expectations? Recruiter Outreach Bias in Online Tech Training Programs (article)
Doug Ng:
- An Art & A Science: How to Apply Design Thinking to Data Science Challenges (article)
- Fizzy Fusion: When Data-Driven Decision Making Failed (article)
Michael Parzen:
- Coursera’s Foray into GenAI (HBR case study)
- AI Product Development Lifecycle (HBR case study)
Roger Rohatgi:
- BP
- X: @rogerrohatgi
- Humanizing innovation: putting humans at the heart of bp through digital design (LinkedIn article)
Firdosh Tangri:
- Fantasy
- X: @FirdoshTangri
- These ‘synthetic humans’ offer a bold new approach to user research (FastCompany article)