You can pick your friends, and your friends can pack a box, but you can’t pick & pack a box without your friends
Amazon's "Open Innovation" approach to solving warehouse automation
Identifying an object and picking it up are tasks that still elude modern robotic systems. In 2015 Amazon Robotics started its “Picking and Packing Challenge” to increase focus from researchers and startups on this exact problem. Inviting submissions from teams around the world, Amazon hosts the annual challenge and offers cash prizes to teams whose robots can quickly and accurately pick specific items (components of a hypothetical order) from boxes in one area, and place these items in a box-to-be-shipped (for a hypothetical customer). As the cost of labor, the human pick & packers, increases and Amazon ships more products to more places, the benefits of automating the picking and packing process are increasingly lucrative for the company. While its proprietary logistics and order fulfillment technology enabled Amazon’s rise over the last twenty years, Amazon does not presume to have the best way to solve the same problems in the future. The “Picking and Packing Challenge” gives Amazon Robotics access to completely new approaches, bright minds, and real solutions.
The task of identifying and picking up objects may be simple for a small human to complete (the human hand has 27 degrees of freedom), but for a robot actually prove quite challenging. The task is particularly complex because in the case of Amazon order fulfillment, the object of interest can be any size, weight, shape or material. For a robot to excel at the task, it must have the ability to perceive an object in its environment, plan its own movement, attempt physical manipulation, and detect and correct errors in packing the final order. Thus, this problem requires expertise in a number of fields: visual detection (software), robotic control algorithms (software) and gripping techniques (hardware). The choice to crowdsource ideas, knowledge, and implementation from teams around the world is essential for Amazon to stay competitive. Universities, where research teams work in many diverse fields, as well as startups who solve similar challenges, are much more likely to offer compelling solutions than an in-house innovation team, specifically because this task sits at the convergence of multiple fields. These are fields that are still developing, as well, thus Amazon is unlikely to find the best approach among employees it has already hired and removed from the world of research.
Amazon’s short-term solution for solving the future of fulfillment and logistics is to host the open challenge. In the long-term, Amazon either hires talented engineers and scientists who compete in the challenge or acquires entire teams (or even businesses). Hiring and acquisition guarantee that Amazon finds and retains the best talent and ideas solving these exact problems today.
It is unclear, however, if the best future solution requires broadening the problem of “Picking and Packing.” A successful “Picking and Packing” robot would directly replace human pickers & packers in Amazon’s warehouse. In the current setup, a worker stands in one place all day, with boxes passing by on a conveyor belt with items needed to fulfill a specific order, and the human only needs to pick up the appropriate amount of items and place them in a box. The assumption is that the logistics and system external to this singular task remain the same: boxes of similar items arranged in the warehouse, coming to one location. However, the best new approaches to warehouse automation may assume an entirely new construct. For example, Ocado (1), a British supermarket, has solved warehouse automation by constructing a grid over which robots can move seamlessly, centralizing the machine or system intelligence outside of the moving robots, and optimizing vertical space.
Two questions remain. First, is the assumption of Amazon’s current warehouse setup that is inherent in the way the challenge is structured a good one? Stated another way, would Amazon discover more effective warehouse automation solutions by starting with a blank slate? And secondly, is there an asymptote on the graph of “Time to Fulfill an Order” versus “Customer Willingness to Pay ” and is Amazon approaching it?
My assumption in answering the second question is that Amazon would rather solve the problem & find out otherwise, then let competitors in the space advance beyond them.
- The Verge. (2018). Welcome to the automated warehouse of the future. [online] Available at: https://www.theverge.com/2018/5/8/17331250/automated-warehouses-jobs-ocado-andover-amazon [Accessed 14 Nov. 2018].
I didn’t know Amazon was doing this, LLovegood – thanks for sharing! It’s very interesting that Amazon does not host this competition as a blank slate. I wonder what the costs would be of renovating their warehouse designs, and whether that sort of project would be NPV positive for the company (my guess is it would be, but who knows!). Perhaps once they find functioning robots, they can work with the winning engineers to test some scenarios outside of Amazon’s pre-existing warehouses to improve the robots’ functionality and ability to pick & pack quickly.
I wonder why you say, “Amazon is unlikely to find the best approach among employees it has already hired and removed from the world of research.” Though I agree that opening the competition to non-employees is imperative toward ensuring that fresh, out of the box ideas are brought to the table, I disagree that those types of ideas cannot come from Amazon employees, especially junior ones who are not yet set in their ways and are eager to take on new challenges. Is it truly open innovation if the contest is not fully open? It would be interesting if anyone, employee or not, could form a “self managed team” similar to the ones we learned about in LEAD, to determine solutions to this problem!
I think what Amazon is doing here is genius. Why spend their own time and money on this problem, when they can essentially outsource it through a “contest”. One concern with these robots I have is if they can ever be value positive for the company. I think Amazon may need too many of them at too high of a price to actually net a positive return, even in the long run. With these robots, they will need to hire higher paid engineers to constantly maintain or fix them if they break down. This can end up being even more costly than the minimum wage warehouse workers of today.
I’m wondering if this is not necessarily a value play for Amazon, but a public relations play. They have recently gotten some extreme negative press about the working conditions in their warehouse (https://www.businessinsider.com/amazon-warehouse-workers-share-their-horror-stories-2018-4). This included workers peeing in bottles, because they couldn’t take breaks and 55% of workers suffering from depression since starting to work for Amazon. If they can replace these workers, then there may be less pressure from Bernie Sanders about raising the minimum wage and less PR issues surrounding the working conditions (https://www.cnbc.com/2018/09/05/bernie-sanders-introduces-the-bezos-act-slamming-amazon-low-wages.html).
Amazon here seems to be adoping the “hackathon” or “red-team” model that a lot of tech and software companies have employed for a long time. By using a competition to identify vulnerabilities in security of firms, they are able to get a fresh set of critical eyes on how strong their systems are.
In contrast, Amazon uses open competition to innovate their warehouse processes. To your question of whether this is a value play or a public relations play, I think the answer is both! They are able to foster a community that cares about making their processes more efficient so that they may be able to address conditions of overwork in their warehouses. At the same time, the PR around having an open innovation contest can only help their public image.
Great article.
Incredibly well delivered *cough* article!
I found your point on customer willingness to pay really interesting. For the last decade, delivery times have been crashing down, especially in city centres with high densities. But there’s a crucial point – am I really willing to pay for less than 1 hour delivery? The incremental effort required for me to pre-plan by this amount is trivial, and additional investment on the behalf of Amazon seems to be with little consumer benefit. Given the scale of change needed, perhaps there are other ways to short-circuit delivery times, outside of asset investment, e.g. own courier network.
Thanks for sharing! I think the structure of the contest will highlight flaws in Amazon’s current warehouse structure and suggest different means of organization. However, my concern is that Amazon’s warehouse is a game of Tetris, carefully organized to manage the flow of inbound and outbound inventory. In a system this complex with robotic specialization, I question if robotic systems will be able to spot and fix problems created in storing, sorting and optimizing across thousands of products. Since issues in the system can create systematic problems, the lack of a human andon cord or a timely kaizen can prove problematic to the overall warehouse’s long-term success.
Separately, I agree that there is a limit to which customers are willing to pay for quicker shipping. I think Amazon can use open innovation and small scale roll outs to test the customers’ willingness to pay for quicker shipping across products.
Super interesting read! As a frequent Amazon user, I am always amazed how packages are shipped out within hours of submitting an order. I think you bring up a good point that Amazon will need to tap into cutting edge researchers to solve its problems so it can’t fully rely on internal R&D, but I question if this contest is actually bringing in new solutions or is mostly a recruiting tool as you mention. Crowdsourcing is great for novel ideas, but this contest requires building an actual robot so is much more time (and capital!) intensive. Would contest participants who are working on this part-time actually be able to deliver better solutions than a full-time R&D team? Either way, I agree in the long-term hiring and retaining this talent is the real asset.
Very interesting article! I’m intrigued by the nature of the competition rules and whether or not Amazon would own the Intellectual Property and solutions of the contest? I wonder if this competition could be detrimental to some of the companies/people competing as this has the potential to follow the normal Amazon model of utilizing the data of items hosted with them to develop their own products similar at a cheaper price. This could potentially be used as a means for competitors to gain a more detailed understanding of the way Amazon solves these complex problems as well as some of the ways they may move forward in the future. I also really enjoyed your question as to the validity of the current model of their warehouses and speculation of some possible alternatives to the conventional warehouse model, I’d love to hear more about these!
Amazon’s biggest risk is a combination of increased customer expectations (that they’ve created) combined with increased pressure on their logistics operations. While I like the approach they’re using to solve some of their logistical problems, I worry that they’re not ambitious enough. If Amazon wants to continue transforming its business, they need to reimagine how they do warehousing to begin with. Amazon needs to start with a blank slate and see if there is a better way to get a package from one location to another to ultimately a box that gets shipped out to a customer. If they don’t, they risk ceding their lead to Walmart or a number of other well capitalized competitors.
Great article, LLovegood! The picking and packing functions are certainly one of the biggest challenges in Amazon’s fulfillment operation, especially given the scale they have grown to. I was a bit surprised that the company is turning to crowd sourcing for a solution. I guess it speaks to true complexity of the problem. To your question, I do think a possible solution will be easier for the company to come across if they are able to start with a blank slate in regard to warehouse setup. Another aspect that comes to mind with this solution is the management of human capital. Would a solution to this challenge essentially eliminate all of their fulfillment workers?
The concern that I have with crowdsourcing warehouse operations is that they tend to be very capital intensive, especially when they involve robots. Is a contest enough to incentivize people and companies to build they robots? Also, how far off are we from a fully automated world? It seems that we’re still in the beginning stages of it and there’s a long way to go. Thus, Amazon will likely be working on this problem in the long term and it seems hiring in house might help focus their efforts better. Overall, it’s going to be interesting to see where warehouse fulfillment goes in the future and Amazon will certainly be one of the leaders.
Great essay! I didn’t know these details about the Amazon warehouses. I agree with your statement, “The choice to crowd source ideas, knowledge, and implementation from teams around the world is essential for Amazon to stay competitive.” However, I wonder if teams are working in the constraints of Amazon’s current warehouse design & structure. I think the ideas would be more valuable if the teams were asked to start with a blank slate and advise Amazon on changes it needs to make beyond robotic abilities. I also wonder how clear Amazon is to these crowd source participants about their standards for customer service, etc. While automation is extremely beneficial in efficiencies and cost reduction, I worry that some of the potential solutions may lower the customer service standards.
Thanks for sharing! This open innovation project by Amazon seems to be super effective in certain domains of problem solving. Specifically, it works well with project with well-defined scope and the goal of making incremental changes. I would assume that this will be hard if you want to crowdsource the solution for drone delivery, for example. While I do think these competitions are useful, I am reserved about how scalable and sustainable it is for the broader innovation topics that Amazon is working on.
This was interesting; I’m surprised that it is such a challenge to automate warehouse picking and packing. I wonder if the use of RFID coupled with basic product dimensions would help in analyzing size of box to use and determining location of SKUs in the warehouse. I think this is the most high-tech example of crowd-sourcing that I have heard of so far. I am interested to see how the digital grid works out for Ocado.
Open innovation is extremely attractive because of the sheer number of ideas that it can provide. It is smart of Amazon to do this, as you mentioned for 2 reasons: 1) the competitive edge 2) the great talent it attracts. I do disagree that these ideas cannot come from their current employees but I agree that sometimes it is easier from outside sources but that may be something imposed by Amazon itself. Large companies tend to be very structured so employees may not have the time or flexibility to try and solve some of the problems that the company is facing.
I do think that Open Innovation can help push the envelope a little further when it comes to the problem they are facing. I am curious to know the parameters of the Picking and Packing challenge because that can sometimes influence the direction in which contestants are trying to solve the problem. Also what would be the right of Intellectual Property of the participants?
Thanks a lot for sharing this, super interesting!
I disagree with only one thing: I think Amazon still has best talent to solve this pick & pack problem. Amazon is one of the first and largest e-commerce players of the world, and as far as I know automatizing pick & pack is one of their top priorities. They are putting a great effort on recruiting the most talented engineers in the world, and they pay really high salaries for that. I believe that the reason why they are doing open innovation for pick & pack is like a lottery: I think they are doing it for the very very small possibility of “winning the lottery” – a.k.a. finding a great idea from outside the company.
Amazon will stop at nothing to fully automate the fulfillment process and the medium/last mile delivery, because unlike the human labor cost that always increases, the cost of tech/robots always decreases as time goes by. I would presume that the recent pay increase for the hourly workers is not a major issue because Amazon intends to fully replace any human worker in the mid term. Their acquisition of the robotics company Kiva in 2012 for 775 million dollars and the subsequent renaming to Amazon Robotics is an example of their endeavor in this domain. The robotics company no longer serves outside customers. Only people working in the fulfillment center will be the managers working on further optimizations/improvements and the robotics maintenance team.
Thank you for writing this. Great read!
Open innovation platforms allow companies to harness the creativity of thousands of minds to solve hard to crack questions. However, the crowd needs to have sufficient information to solve the problem at hand. My question is whether Amazon is revealing vital information to the masses and as a result revealing industry secrets that could empower competitors to gain an edge over them?
Great article! Having spent time doing a project in a fulfillment center very similar to those Amazon operates this article particularly resonated with me. I think it is only a matter of time until this “picking” task is able to be automated whether by an actual robot or by just condensing shelves and have the items “vended” out onto a conveyer belt. This will be a disruptive force in the lives of the hundreds of thousands of people employed in this type of work. In the one warehouse alone that I worked in there were hundreds of people relying on this job for their full income. Society will very soon need to grapple with how to deal with the employment losses that these innovations will bring about.