DeepMind and the Development of Artificial General Intelligence
First Chess and now Go. What’s the next to go?
DeepMind is a leader in artificial intelligence research most famous for its AlphaGo program, an AI program that defeated the world’s best player at Go[1]. Its sole mission is to “push the boundaries of AI, developing programs that can learn to solve any complex problem without needing to be taught how”[2]. As a result, it aims to be at forefront of AI research, so that it can develop and apply artificial intelligence to a wide range of disciplines. It tackles real-world problems by collaborating with experts on long-term research projects centered on the application of AI to different fields.
Despite recent advancements in AI, researchers at DeepMind recognize that there are constraints and areas of artificial intelligence that have yet to be explored in detail. For example, many of the problems tackled by AI have had limited action states, perfect information between participants, or have had one agent acting at a time. In the real world, most of these constraints don’t exist and actions are more dynamic and varied. As a result, DeepMind has teamed up with Blizzard to use Starcraft II (a real-time strategy game) as a research learning environment, because gameplay in Starcraft II represents a “more difficult class of problems than considered in most prior [AI] work”[3] because it removes many of the constraints mentioned above. In addition, Starcraft II brings to the forefront an issue that AI, and humans, have difficulty dealing with: uncertainty and lying. A key component of Starcraft gameplay is the idea of a “fog of war”. Each player is only able to see parts of the map where they currently have a unit[4]. This lack of visibility increases the amount of uncertainty in a game and creates the opportunity for players to lie and provide false information[5]. As a result, machines cannot simply “iterate” and make the best move but must make a judgment based on the incomplete set of information that it has. This, in comparison to having perfect information, is much closer to reality for many of the problems that AI is trying to tackle.
DeepMind’s goal with Starcraft II is to build “artificial general intelligence” by better understanding what the “learning paradigm” is, so they can build an agent that can learn to “play any game without much prior knowledge”[6]. Unlike previous AIs like Deep Blue (chess) or IBM’s Watson, newest AI by DeepMind is meant to “remember, to strategize, and to learn”[7]. Since DeepMind understands that their overall goal is lofty, they have broken the problem down into “smaller, more manageable chunks”[8], something that makes games a particularly attractive research environment. They’ve partnered with Blizzard to gain access to a database of replays that the AI can learn from and have designed mini-games to break the overall gameplay into smaller problems[9]. For the foreseeable future, DeepMind is trying to tackle the big picture problem by looking at the little pieces with the promise of general application in the future. They have opened the resources to the public to generate interest and gain insight from third parties. I believe DeepMind should also apply their AI in more practical ways in parallel with their work with Starcraft, so they can identify some transitional difficulties early on. They don’t need to address any of the issues before the AI is more developed, but it could be helpful to see how the AI performs outside of the controlled environment if it is meant to have general applications.
While I understand the pursuit of the overall big picture for building artificial general intelligence, does this grand quest make sense in a world with constrained resources? Will we, in the foreseeable future, use AI in a way where constraints will be minimal, if not nonexistent? Does it make more sense to tackle a problem with more constraints if that’s the intermediate step anyway?
Word Count: 800
Photo Credit: DeepMind
[1] Elizabeth Gibney, “What Google’s winning Go algorithm will do next”, Nature (Mar 15, 2016).
[2] DeepMind website “About Us”, https://deepmind.com/about/
[3] Oriol Vinyals et al, “Starcraft II: A New Challenge for Reinforcement Learning” p. 1, Cornell University (Aug 2017).
[4] Hochul Cho, “Investigation of the Effect of ‘Fog of War’ in the Prediction of StarCraft Strategy Using Machine Learning” p. 2, Computers in Entertainment (Jan 2017).
[5] Jonathan Cheng, “Humans Still Rule in This Game – To win ‘StarCraft,’ machines need to learn to lie” p. 2, Wall Street Journal (Apr 23, 2016).
[6] Justin Groot, “Checking in with the DeepMind Starcraft II Team: Interview with Oriol Vinyals”, Blizzard News (Mar 4, 2018).
[7] Christina Beck, “Next AI Challenge: Computers take on StarCraft”, The Christian Science Monitor (Nov 05, 2016).
[8] “Shall we play a game?; Artificial intelligence”, The Economist (May 13, 2017).
[9] Oriol Vinyals et al, “Starcraft II: A New Challenge for Reinforcement Learning” p. 1, Cornell University (Aug 2017).
It is definitely interesting to see the parallels you drew between DeepMind’s work with Starcraft II and projects like Deep Blue and IBM’s Watson, as although Starcraft II is still just a game, it seems to represent a natural progression and the next evolution of AI. With constrained resources, you certainly have a point that maybe this technology could be applied somewhere where social impact could be better felt, but I think that as each iteration of AI progresses from one challenge to the next, we get closer to principles and capabilities that are more generalizable for real world applications. As an added bonus, Starcraft II may be able to draw more data scientists who are attracted to this project and its unique and fun learning environment. Great article!