Crowdsourcing at Amazon: Democratization of TV / Film Content
Amazon Studios recently shutdown its crowdsourcing initiatives. This article explores their approach: What went wrong? How could they have better utilized open innovation?
Crowdsourcing at Amazon: Democratization of TV / Film Content
“Hopefully we can avoid big bombs. Our notion for what the world needs may be a roller-skating movie or a battleship film, but that could be wrong. We can do tests and find out that, actually, no one cares about this project or that one. If you do that before you spend $200 million on it, that would be good. Good for customers and good for the business.”
– Roy Price, Former Head of Amazon Studios(1)
Introduction to Amazon: A Pioneer in Content Crowdsourcing:
Amazon launched Amazon Studios in 2010 to begin producing original content for their Amazon Prime streaming platform(9). Streaming platforms, unlike traditional linear media formats, allow for more programming “shelf space”, given the lack of primetime TV slots or prime movie opening days. While TV producers often seek out formal customer feedback before greenlighting a show, Amazon Studios spearheaded a fully crowdsourced model of content creation and feedback. This crowdsourcing approach aims to increase the success rate of new creative content(8), compared to the traditional Hollywood practice of sourcing and filtering early-stage ideas through a board of media executives.
Amazon Studio’s Crowdsourcing Across The Production Funnel:
Amazon Studios has attempted to introduce crowdsourcing into multiple aspects of their content production funnel:
Example 1 – Open-Submission of Script and Concept Videos:
Amazon Studios has an open-submission policy for scripts and concept videos, meaning it will accept concepts from anyone, regardless of their level of experience. This approach dramatically increases the size their production funnel, which may be a significant advantage in the streaming video economy (as described above). Through open-submission Amazon has produced a number of successful products, such as “Zombies versus Gladiators” and the children’s TV series “Magic Monkey Billionaire.” (1)
Example 2 – Customer Feedback on Concepts / Footage via Amazon Preview:
Amazon Studios has also implemented Amazon Preview, an invite-only community of top customers who are encouraged to provide feedback on various concepts, storyboards, and test footage of submitted content. These customers provide valuable feedback, ratings, commentary, and hashtags that serve to categorize and analyze content(4). This system allows Amazon to collect information on new content earlier than other studios and at a much lower cost. (3)
Example 3 – Public Voting on Pilots via Pilot Season:
Amazon implemented “Pilot Season” in 2012. This program encouraged customers to watch pilots and vote on which should have a full season commissioned. Former Director of Amazon Studios, Roy Price, described the value of this process by saying, “we’re trying to develop or find shows that will be popular with Amazon customers, so who better to ask than Amazon customers themselves.” Greenlight decisions are made not only based on voting data, but on actual customer usage behavior(1). Two of Amazon’s biggest hits, “Transparent” and “Mozart in the Jungle”, were selected through the Pilot Season program. (2)
Amazon Studio’s Impediments and Avenues for Improvement:
Amazon is the only player to utilize a crowdsourcing model. At the end of June 2018 Amazon closed their open-submission portal on June 30th, 2018(6). Creators will still have access to the companion Storybuilder and Storywriter tools (Amazon-provided digital tools aimed at helping under-resourced creatives) (6). In addition, Amazon announced in July 2018 that it would end its Pilot Season program as well. As of November 2018, Amazon still offers Amazon Preview, which it views as value additive to both users and to the company’s market research efforts. One reason for the closure of these programs, according to Amazon’s co-head of TV, Albert Chang, was that “you end up taking way too long to get the actual season done.” (3)
Impetus for Change:
The company hasn’t explained much more regarding the reasoning behind the shutdown besides “looking for ways to become even more efficient.” (6) In part, this transition was likely due to a change in Amazon’s Prime Video strategy. As the streaming video category has matured, Amazon is increasingly focused on blockbuster productions with very large title-level budgets (Amazon spent an estimated $4.5B on content in 2017 — twice HBO’s budget(5)). As the required investment of each individual title increases, studios are increasingly incentivized to mitigate the risk of new projects through the utilization of existing IP (e.g., licensing known entities such as comic books) or through partnerships with established talents (e.g., writers, directors, actors) (6).
Suggestions for Management:
It is difficult to maintain crowdsourcing as a company scales and matures, especially in markets where competition for new projects is competitive and requires a high level of expertise. Amazon’s crowdsourcing model failed because A) it created a delay between pilot and production that was unacceptable to the most in-demand content creators(3), and B) it sometimes created data biased toward a non-representative demographic of consumers (those who view and vote on new pilots). However, there are ways for Amazon to address these concerns and re-implement crowdsourcing. I suggest that the company explore the use of computer assisted tools to sort and filter user-submitted scripts, which could and minimize labor requirements and increase speed-to-market. Additionally, Amazon could better integrate voting mechanisms into their core Prime Video Platform, utilizing “painted door tests” or showing short previews of potential projects before shows, which would widen the sample of consumers to a more representative sample. This would enable Amazon to collect a greater and more representative scale of data much faster than their initial model allowed(7). Crowdsourcing was an innovative attempt at gaining a competitive edge in content creation, and Amazon should continue investing in related models.
- Can we utilize machine learning or machine-automated selection to improve crowdsourcing efficiency?
- What part of the funnel should Amazon reconsider utilizing user input – idea generation or selection?
(Word Count: 799)
 Reuters. (2018). Crowdsourcing goes to Hollywood as Amazon makes movies. [online] Available at: https://www.reuters.com/article/us-amazon-hollywood-crowd/crowdsourcing-goes-to-hollywood-as-amazon-makes-movies-idUSBRE8990JH20121010 [Accessed 13 Nov. 2018].
 Business-standard.com. (2018). Amazon stops putting ‘pilot episodes’ online for viewer feedback. [online] Available at: https://www.business-standard.com/article/news-ians/amazon-stops-putting-pilot-episodes-online-for-viewer-feedback-118073000713_1.html [Accessed 13 Nov. 2018].
 Fortune. (2018). Amazon’s ‘Pilot Season’ Practice Officially Ends. [online] Available at: http://fortune.com/2018/07/29/amazon-pilot-season-viewer-feedback/ [Accessed 13 Nov. 2018].
 Page, V. (2018). Amazon Studios vs. Hollywood: Is Netflix the Better Pick? (AMZN, NFLX). [online] Investopedia. Available at: https://www.investopedia.com/articles/personal-finance/111815/amazon-studios-vs-hollywood-netflix-better-pick.asp [Accessed 13 Nov. 2018].
 CB Insights Research. (2018). Amazon Strategy Teardown. [online] Available at: https://www.cbinsights.com/research/report/amazon-strategy-teardown/ [Accessed 13 Nov. 2018].
 Engadget. (2018). Amazon Studios stops taking amateur scripts on June 30th. [online] Available at: https://www.engadget.com/2018/04/14/amazon-studios-ends-open-call-for-scripts/ [Accessed 13 Nov. 2018].
 Tucker, R. (2018). How Crowdsourcing Can Enhance Innovation Performance. [online] Forbes. Available at: https://www.forbes.com/sites/robertbtucker/2017/06/16/how-crowdsourcing-can-enhance-innovation-performance/#75695dae54df [Accessed 13 Nov. 2018].
 K. Boudreau and K. Lakhani. Using the crowd as an innovation partner. Harvard Business Review 91, no. 4 (April 2013): 61–69.
 Wallenstein, A. and Wallenstein, A. (2018). What the TV Biz Could Learn from Amazon Studios. [online] Variety. Available at: http://variety.com/2013/biz/news/amazon-viewer-data-for-development-process-1200429921/ [Accessed 13 Nov. 2018].
Student comments on Crowdsourcing at Amazon: Democratization of TV / Film Content
Really interesting article about a bold experiment on open innovation in the entertainment space!
My question is: how well does completely open sourcing really work in entertainment? TV/Film writing is notorious for having extremely low talent density (few very talented individuals among a huge number of individuals trying to participate in the space). Should Amazon be seeking contributions / submissions from only qualified individuals? Or should they continue accepting submissions from literally anyone?
I think this a really unique perspective of how Amazon is using some of these innovation trends unsuccessfully. I would argue that “example 3” is not that far from what already exists today; I was under the impression that pilots are shown to potential audiences and production companies move forward the ones that have best reactions. If the main difference was the ability to have many people in a room give quick feedback versus waiting for a video product to be watched on others’ own time around the world, it feels like there could be different changes to make that turnaround time shorter.
Just a general thought around “example 1”; this is a real shame as I think open-submission has the potential to find some really incredible and unique stories. I hate the Marvel-esque lack of original content for blockbusters and the industry trend of sequels & remakes. I wonder why Amazon didn’t decide to leave the open submission open, just so that they have an option to look at them? And then they would have to bid to produce one that was really good, similar to how it works outside of the crowd-sourced platform. I guess we would need to know how many submissions they were getting vs. the number of quality ones that are worth looking into further… Maybe that’s a future machine learning challenge.
Very interesting article – I had not realized Amazon was running their studio this way! I wonder if another reason for the shutdown of this project was the inherent tension of “crowd-sourced” ideas and creativity. This is just my perception, but I would like to see a study done on the sorts of ideas that crowds would come back with in creative spaces, such as TV. I think that creativity is by definition the antithesis of the “popular opinion”, or the widely held opinion, so crowd-sourcing creative content could just end up with a very bland content portfolio. This challenge would primarily relate to examples 2 and 3 above – example 1 is more interesting from my perspective, as they are trying to expand their funnel and bring more creativity into their process.
To your second question, I do see the benefit in user input for the selection setting, but wonder how sustainable this could be for idea generation (which it seems like was a driver for the shutdown due to inefficiency). At its core, for a creative product output, it feels as though just taking ideas from others will not lead Amazon to be the leading thinker in the space at the forefront of new trends. I also think this leaves Amazon with the risk of having content that is the most “generic”, however I think to develop a blockbuster, content needs to be new and fresh. Because of this, it is counterintuitive for Amazon Studios to rely on innovation for its core competency, but it does make sense to use it to refine and improve on its core.
Great piece! I appreciate hearing the challenges in addition to the successes as companies work to integrate new techniques. So often we only hear about how these types of things are completely transforming industries and the struggles aren’t shared.
In this specific example, similar to Boreal, the thing I can’t get over is why they weren’t able to dictate the turnaround time to make it more efficient. This seems like it should have been completely within their control.
I think that this is a really interesting article. The thing that is tough about crowd sourcing is that words on a page don’t always reflect the finished product that ends up on screen. While it is good idea to try crowdsourcing ideas, the execution of a film is what counts more in consumers and critics’ minds. To your final point, how much of a competitive advantage does crowdsourcing really give Amazon? In an industry where there are millions of bad ideas and a shortage of good ideas, it seems that the good ones tend to float to the top on their own without the crowdsourcing.
I’ve been increasingly impressed with the unique content produced by Amazon Studios. For them to be even relevant in the media and entertainment industry, competing with giants such as HBO, Netflix, and Hulu, is incredible. Certainly, this shows the value of crowd sourcing ideas. A key issue here though is how large the audiences might need to be for a particular idea to get off the ground. Obviously producing content that pleases the masses is incredibly difficult and some of the ideas generated might be extremely attractive but only to small segments of customers. The challenge is then estimating the appeal to broader audiences without sacrificing the quality of the content.
I didn’t know that Amazon did this, it’s an inspiring idea. I liked how the author explains the reasons for this project to be closed down, but I’m a bit skeptical about the opportunities to reopen it. In order for Amazon to benefit from this in terms of new ideas, it would have to believe that their own in-house creative content capabilities (based on a lot of user data (just like Netflix…) could not create what people outside of Amazon can. Not only once, but several times. On the other hand, if Amazon wants to use crowdsourcing as a way of selecting ideas, it would also have to believe that the data they have on consumer preferences, which is substantial, is also less accurate than opinions gathered through crowdsourcing.
What an interesting read! It sounds like crowd sourcing at later stages of the development process is more costly because of the time/delays associated with it. If Amazon were to reignite this program, I would recommend focuses on idea generation. One huge huge trend in television right now is hearing from traditionally under represented voices (Transparent, Fresh Off the Boat, Blackish, etc.). I wonder if there is an opportunity for audiences to play a role by voting on what topics they want to see on screen in terms of race, gender, socio-economic status, international stories, family dynamics, etc. I would encourage Amazon to engage with audiences in this very early stage to help the company identify gaps in their story telling coverage.
Something I’ve grappled with, and I think it’s growing in recent times, is the divergence between box office numbers and critical acclaim (the classic tension between film as a commercial product vs as an art form; of course, it is both in varying degrees). Case in point – a lot of the superhero movies make an absolute bonanza, but typically review poorly (much to the chagrin of loyal fans).
My question – if this technology were adopted by players across the industry, could we use crowd-sourcing to generate the innovation and ‘art’ that we need to ensure the medium’s continued vibrancy? Or do we need the continued out-sized influence of a few very successful creative experts to continue to prop up that side of the industry?
I can’t feel anything but respect for Jeff Bezos after reading this: crowd-sourcing content for AmazonStudios is an attempt to democratize an industry that has been, since its beginning, based on insane power imbalances. Why couldn’t one of us come up with something better than the same, recycled story Hollywood executives share?
Since Amazon has now tweaked its content creation model, and I think what they discovered is that there is not necessarily a business case for their initial approach. Ultimately, it may be more cost-effective to allocate less money on staff who read endless contest submissions, and spend more time and money on producing titles that have a higher guarantee of viewership. They are finding new ways to work with experienced talent, and although an initial concept, or kernel of an idea, from the outside Hollywood might make for an extraordinary movie, I would imagine that fleshing out a script in full, and overseeing a complete film production process, is best left to industry veterans. However, Amazon is still flexing its muscles in Hollywood, and showing that the entrenched, historical victors are not necessarily the best in the game! #2goldenglobes #3oscars #5emmys
Great article. With 20-20 hindsight it’s easy to say that Open Innovation should not be a tool that drives volume. I feel how to introduce cohesion between different elements in an Open Innovation environment is a question practitioners must address before charging ahead with OI initiatives.
I really appreciate this case example of where open innovation might be considered a “failure”, but I wonder if this failure is specific to conditions that are unique to the process of entertainment production. First, when considering the huge volume of individuals (from average joes to professional writers) who have ideas for scripts and shows, I worry whether the much wider production funnel of having to process through suggestions would have been a feasibly surmountable challenge for really any group or organization, even with crowdsourced prioritization. Second, even if the number of suggestions were somehow made to be more manageable, I’d question whether crowdsourced selection is truly what a creative industry like entertainment production should rely on. There’s a reason the entertainment industry is led by creatives and artists: they have an ability to have an artistic vision and see ahead to the “next thing” that will provide artistic value to viewers, and I’d argue there is an artistic intuition required for such an eye for selection that the masses won’t have.