Disney – “A Whole New World” of Machine Learning
Let us explore how the masters of imagination are using machine learning to alter the movie production and animation processes
Situation
Disney is an entertainment powerhouse – it captured 61% of movie industry earnings in 2016 [1]. In 2017, the studio entertainment division earned 2.3 billion dollars [2]. This is a small part of the company’s earnings, but it is the lifeblood as the characters and stories create the basis for merchandise, theme parks and interactive experiences [3].
However, Disney is not immune to changing consumer preferences pressuring the profitability of the industry (more demand for streaming, lower willingness to buy tickets etc.) [1]. To remain competitive, it is crucial to pick the right content and produce it quickly, at low cost without sacrificing quality. Thus, Disney is leveraging the power of machine learning to stay ahead of consumer preferences and automate the animation process.
Current solutions
In the short term, Disney is working on further automating the animation process using machine learning. Getting the lighting of animations right is a difficult process, which when sped up produces visual imperfections known as “noise” [4]. In collaboration with Pixar and the University of California Santa Barbara, Disney has improved the previously time-intensive and partly manual “denoising” by using a “Convolutional Neural Network” [4]. This deep-learning model relies on previous Disney movies as guides to improve the image quality of new productions [5]. Thus, Disney can save time and money in producing its future movies and leverage past productions.
In the medium term, Disney is moving towards leveraging machine learning to guide content choices. Disney has been capturing the second-by-second facial reactions of its audiences [6]. Using an algorithm, factorized variational autoencoders (FVAEs), millions of data-points can be interpreted to signal which moments in a movie elicited which audience response [7]. This technique could be used in assessing different versions of movies before releases and gives Disney a vast knowledge base to how content decisions affect viewers [3]. While using test-audiences is not new, this innovation allows for massively more data to be analyzed and does not rely on subjective customer feedback gathered through surveys or interviews [3]. This FVAE technology also has a dual application – its large calculation power can be used to analyze how scenes behave in nature and translate that into automation [6]. For instance, it can capture how different trees on a hill react to wind and automate the translation of this image into animation [6].
The most ambitious use of machine learning is Disney’s attempt to judge potential content through neural networks [8]. This would mean not only making small adaptations based on audience preferences, but deciding to produce stories identified by AI. For now, the research has focused on predicting the popularity of “Quora” posts as an indication of the attractiveness of a narrative, but Disney has high hopes [9]. The end outcome might be to have scripts or movie ideas evaluated and altered by machine learning to speed up the production process.
Further considerations
Disney is clearly pioneering using machine learning to facilitate the animation process. Complex tasks are being rendered less labor intensive and thus cheaper [10]. Disney has purposefully founded The Walk Disney Studios LAB, together with HP Enterprises, Cisco and Accenture Interactive, to explore how technological innovations can improve story-telling [11]. It is unclear if these changes will result in enough cost and time savings to allow Disney to better adapt to industry pressures (e.g. producing more content for streaming services).
The bigger challenge will be instituting the right guardrails for using machine learning to dictate content decisions. It will be crucial for Disney to monitor biases and errors machine learning-based content choices can create:
- FVAEs are already being questioned. For instance, they might misread the natural human tendency to mirror an expression seen on-screen (e.g. smiling because we see a smiling baby on screen) as a genuine positive reaction to the image as opposed to an automatic response [12].
- By not ensuring a diverse enough audience in the FVAE screenings, certain customer segment preferences could be excluded [13]. This is especially relevant for a global company as cultural differences will affect reactions to content.
- Supplying the neural networks with historically-successful narratives might not produce the forward-thinking Disney has tried to become known for, like diverse characters such as in Moana or The Princess and the Frog [14].
Along with improving the content-generating technology, Disney should ensure content-choices based on machine learning will be evaluated critically and adjusted by human judgement as necessary [13].
Questions
- Disney is investing heavily into novel technologies – do they intend to use them only in the entertainment space or is there room for them to branch into other applications? (e.g. using FVAEs to improve care for patients unable to speak [6])
- What is the long-term relationship between human imagination and machine learning in determining content to produce?
(798 words)
[1] Lang, Brent. “The Reckoning: Why the Movie Business Is in Big Trouble.” Variety, 27 Mar. 2017. Accessed 12 Nov. 2018
[2] The Walt Disney Company. Fiscal Year 2017 Annual Financial Report. 2017. Accessed 12 Nov. 2018
[3] McKenna, Beth. “Disney and NVIDIA Team Up on Artificial Intelligence for Making Better Movies.” The Motley Fool, 27 Aug. 2017. Accessed 12 Nov. 2018
[4] “Disney Research, Pixar Animation Studios and UCSB Accelerate Rendering with AI.” Disney Research, Accessed 12 Nov. 2018
[5] Bako, Steve, et al. “Kernel-Predicting Convolutional Networks for Denoising Monte Carlo Renderings.” ACM Transactions on Graphics, vol. 36, no. 4, 2017, Accessed 12 Nov. 2018 via Disney Research Website
[6] Perkins, Robert. “Neural Networks Model Audience Reactions to Movies | Caltech.” The California Institute of Technology, 21 July 2017, Accessed 12 Nov. 2018
[7] Johnson, Madeleine. “Here’s How Disney Is Implementing Artificial Intelligence.” Nasdaq, 27 July 2017, Accessed 12 Nov. 2018
[8] Nield, David. Disney Is Developing an AI That Can Judge What Makes For a Truly Great Story. ScienceAlert, 23 Aug. 2017, Accessed 12 Nov. 2018
[9] LeFebvre, Rob. “Disney Research Taught AI How to Judge Short Stories.” Engadget, 21 Aug. 2017, Accessed 12 Nov. 2018
[10] Editorial team. “Trends in Entertainment Industry Push Enterprises Toward AI.” InsideBIGDATA, 19 Sept. 2018, Accessed 12 Nov. 2018
[11] “The Walt Disney Studios StudioLAB Advances the Art of Storytelling Through Next-Generation Technology.” The Walt Disney Company, 16 July 2018, Accessed 12 Nov. 2018
[12] Breakey, Julia. “Disney Uses AI to Predict Viewers’ Expressions, but It Isn’t Foolproof.” Memeburn, 28 July 2017. Accessed 12 Nov. 2018
[13] Bloomberg, Jason. Bias Is AI’s Achilles Heel. Here’s How To Fix It. Forbes Magazine, 20 Aug. 2018. Accessed 12 Nov. 2018
[14] Desta, Yohana. The Year Disney Started to Take Diversity Seriously. Vanity Fair, 23 Nov. 2016, Accessed 12 Nov. 2018
Extremely interesting, especially the other potential applications of FVAE. I like the idea of using FVAE to do consumer research, but I am less supporting of the idea of using machine learning to predict what kind of content should be produced. Call me a romantic, but I think art should be a creative process rather than an optimized, data-driven process to maximize sales. It takes the joy and wonder out of creative content if I know it was just made from algorithms. Also, the future can only be predicted if it looks a lot like the past, so using machine learning to help generate creative content means that we’ll probably end up with things that look a lot like what is already out there, which means there could be a stall in creativity.
Our thoughts are in complete sync Satomi. Reading this, I begin to wonder if this will reduce creativity. I am in the school of thought that sometimes people do not know what they want until you show it to them or until they try it out. AI to me seems more as a reactionary tool and not necessarily proactive. I believe the movie industry, in general, runs a risk of creating the same content because their algorithm said it is what people like. Lastly, we have these algorithms to thank for all the movie sequels we have.
For example, why does every spiderman movie have the same plot? why must every Batman movie be about his fight with the Joker? When can we expect to see another chapter in their lives? I would love to see when Batman finally gains the trust of the public or die.
I believe if the movie industry is not careful with AI and Machine learning, my grandchildren will watch the same storyline about how Captain America was found in the ocean, hundreds of years after he drowned.
Thanks for sharing this perspective. This article has me thinking even bigger about the potential for an algorithm like FVAE, which fundamentally is trying to derive a person’s thoughts based on his/her facial expressions. I wonder about the potential outside of content creation (or monetization) like using it to assess the truthfulness of interviewees, public figures, or even loved ones. Looking forward to seeing how this shakes out.
This is an excellent article, providing perspective between the tension of art and science. Since Disney has always been a leader in technology, I think they are onto something here. Although some signals may be misinterpreted, I think the advantage is the democratization of movie scripts. Because of the shear amount of movie scripts, there are many talented writers that go unnoticed. The best writers are ones that can captivate a person who has some authority to produce their script, which only a select few may successfully accomplish. The key for Disney is to be willing to take risky bets to stress test their ML platform. If they only use the platform as a justification and approval platform for blockbuster movies, their movies may become stale and homogenous. This can be seen already with some of their Star Wars spin-offs.