Defeating Terrorism With Big Data
“This is a new way of war-fighting.” – Mary Beth Long, former Assistant Secretary of Defense
Terrorism
Terrorism is something familiar to us all, especially over the last decade where we have seen many terrorist attacks around the world either directly or through the media. From 2006 to 2013, there have been approximately 90,000 terrorist attacks causing about 130,000 fatalities throughout the world.[1] The rise of terror groups like Al Qaeda and Al-Shabaab over the last several decades has created chaos and fear throughout the international system. The loss of life and the economic impact of terrorist attacks are devastating. How do intelligence communities, the military, and law enforcement combat this threat?
Big Data
What exactly is “big data” and how can it help in the global effort to stop terrorism? Big data is the concept of using an enormous volume of available information, from a variety of sources and networks, to make insights at a rapid velocity to, in turn, gain an advantage against competitors.[2] Big data is a technology that lets users crunch an enormous amount of data, that when used correctly, can identify hard-to-detect patterns of terrorist groups or terrorist attacks, allowing the users to either prevent or respond to terrorism.[3]
Palantir Technologies, a Peter Thiel created Palo-Alto technology company currently valued at $20 billion, utilizes software to find connections between people, places and events in large repositories of electronic data.[4] Palantir’s current customers include the Central Intelligence Agency, the Federal Bureau of Investigations, Special Operations Command, the Department of Homeland Security, the New York and Los Angeles Police Department, and numerous other organizations within and outside the United States.[5] Why is this software so valuable? Intelligence and law enforcement organizations often have their own databases, with their own formats, that do not link to other databases, making it very difficult for analysts to conduct investigations and to obtain all the possible information that exists.[6] By interconnecting the databases, Palantir allows users to link and add new information where gaps previously existed. Palantir also allows users to see other users that are working on a project to promote interagency communication.[7]
Show Me an Example of this Technology!
The following is a condensed version of a hypothetical situation that Palantir created:[8]
“… a foreign national named Mike Fikri purchased a one-way plane ticket from Cairo to Miami, where he rented a condo. Over the previous few weeks, he’d made a number of large withdrawals from a Russian bank account and placed repeated calls to a few people in Syria. More recently, he rented a truck, drove to Orlando, and visited Walt Disney World by himself. He spent his day taking pictures of crowded plazas and gate areas.
The day Fikri drives to Orlando, he gets a speeding ticket, which triggers an alert in the CIA’s Palantir system. An analyst types Fikri’s name into a search box and up pops a wealth of information pulled from every database at the government’s disposal. There’s fingerprint and DNA evidence for Fikri gathered by a CIA operative in Cairo; video of him going to an ATM in Miami; shots of his rental truck’s license plate at a tollbooth; phone records; and a map pinpointing his movements across the globe. All this information is then displayed on a clearly designed graphical interface that looks like something Tom Cruise would use in a Mission: Impossible movie.”
As you can see in the above example, Palantir links all useful information into a single portal for intelligence and law enforcement communities to act upon. With this new link in the chain, information can be linked and communicated to prevent terrorist attacks from occurring or it can be examined retroactively to find terrorist locations.
That’s Cool, Can it Do Anything Else?
JPMorgan Chase is using Palantir’s software to spot fraud and prevent people from hacking into others’ online or ATM accounts.[9] Following the devastating 2010 Haiti earthquake, Palantir developed a database of names and structures affected by the crisis.[10] Palantir Health is a platform that allows authorities to reconstruct the history of patient interactions with health administrators.[11] These are simply the opportunities available for Palantir… what else can be done with big data?
Word Count: 797
Endnotes
[1] Statistics and Facts – Terrorism, https://www.statista.com/topics/2267/terrorism/
[2] Big Data: The Management Evolution, Andrew McAfee and Erik Brynjolfsson, https://hbr.org/2012/10/big-data-the-management-revolution
[3] Counter-terrorism Tools Used to Spot Fraud, Richard Waters, Financial Times, https://www.ft.com/content/796b412a-4513-11e2-838f-00144feabdc0
[4] Palantir Technologies Spots Patterns to Solve Crimes and Track Terrorists, Shane Harris, Wired Magazine, http://www.wired.co.uk/article/joining-the-dots
[5] Ibid.
[6] How Team of Geeks Cracked Spy Trade, Siobhan Gorman, The Wall Street Journal, http://www.wsj.com/articles/SB125200842406984303
[7] Ibid.
[8] Palantir, the War on Terror’s Secret Weapon, Ashlee Vance and Brad Stone, Bloomberg, http://www.cfr.org/world/bloomberg-palantir-war-terrors-secret-weapon/p26601
[9] Counter-terrorism Tools Used to Spot Fraud, Richard Waters, Financial Times, https://www.ft.com/content/796b412a-4513-11e2-838f-00144feabdc0
[10] Palantir Technologies Spots Patterns to Solve Crimes and Track Terrorists, Shane Harris, Wired Magazine, http://www.wired.co.uk/article/joining-the-dots
[11] Ibid.
This software can be a real game changer, but it depends entirely on organizational discipline in entering data over time. Often times, the ground forces (Soldiers, police, FBI field agents) see entering the data into these systems after a mission or patrol as a burden and don’t realize the value of the information for everyone else in the network. No technology will ever be invented to fix this problem, but engaged leaders can ensure that their junior leaders see the value in compiling the information and enter it routinely.
Thanks for the great article. Some of the stories that have come out about the ways in which Palantir has been able to stop crimes, save lives and solve problems are incredible. The obvious challenge here is the moral dilemma between arming intelligence agencies with as much data as possible and individual’s rights to privacy. Great arguments can be made on both sides. One trend that has unfolded over the past few years, as especially since the first round of this debate in the years following 9/11 is that tech companies have been aggressively mining and collecting data about their customers. The very citizens that were outraged by the government collecting data about them without warrants are giving their information away for free to Facebook and Google.
Seperately, I agree with AJ’s point above in that we should never strive to move past human involvement in these investigations. Palantir can hopefully create powerful tools for the intelligence community that allow us to save lives and stop crime – will minimizing invasion of privacy. At the end of the day though, this needs to be a tool that is used by human professionals.
MM, great post!
The recent increase in isolated, lone-wolf terrorist actions like the Orlando night club attack and the Bastille Day attack in France underscore the importance of developing new intelligence tools to identify and prevent terrorists. Palantir’s big data solution seems well suited to detect suspects that more traditional intelligence gathering methods might miss.
I do see some area for concern though. As noted, Palantir is attempting to “promote interagency communication” and “interconnect the databases.” While linking together information may be necessary, it must be done very carefully. I worry as our intelligence data becomes more pooled it also becomes more vulnerable to theft or cyber-warfare/crime. The Edward Snowden case illustrates the vulnerability of having single individuals have such broad access to data. This isn’t to say that Palantir and others shouldn’t proceed along this path. I just think it’s important to be incredibly thoughtful on how we secure and protect these databases and tools as we develop them.
Great post MM!
Building on AJ’s post, these incredible capabilities are only as good as the data they are fed and the way they are built into standard operating procedures/ daily workflows — I have heard from former users of Palantir products that the latter issue can sometimes be even more challenging than the former (e.g., Palantir will come in and build a great predictive tool that will then sit on the shelf without being utilized with the regularity/ in the way it was envisioned to). I could imagine this being a particular issue in deployed combat operations, where there are so many demands on a unit leaders time that it may not be feasible to circle back with the analysts on what the Palantir tool is saying. It sounds like Palantir could create a lot of value by investing more energy into actually building utilization of its tools more closely into workflows and making sure that the end users are trained well enough to use the tools to their full capability and to be able to propose incremental improvements.
This post is fascinating – although we all wish there wasn’t really a need for such a system!
One thing that strikes out to me is the essential value of the data. The importance of sensitive information is dependent on the fact of who owns the information and how it may it be used. Although the transfer of information is key to solving such a close-knit world’s problems, I worry if Palantir will get drunk on its own Kool-Aid, and thus the very problem it is trying to solve, might not remain relevant anymore.