Palantir – Big Brother is Watching You
Highly sophisticated tools for big-data analysis and visualization
Palantir is a private US company, that develops a software for advanced data analysis. Its main customers are US security agencies, investment banks, hedge funds. As of 2016, it is #4 startup in the world in terms of capitalization (after Uber, Airbnb, and Xiaomi) with a valuation of $20 billion.
Palantir creates value through highly sophisticated tools for analysis and visualization of data that help government agencies, banks, hedge funds and other organizations to collect and analyze large amounts of data, maintain information security, combat terrorism and identify fraud.
Palantir program is capable of processing an incredible amount of information (in some cases up to a million gigabytes) from different databases and allows to search for facts necessary to convert data and filter it according to time and place. Very simple interface allows the operators to sift information using understandable terms like “fraud” or “suspicious account”.
Palantir captures value by means of selling its software products and services to clients. Since it was founded six years ago Palantir is rapidly expanding its client base in Washington in government sector and in New York in the financial sector. This software company helped to find information about the suicide bombers in Iraq and to identify abuses in the expenditure of “stimulus” in the United States. JPMorgan Chase became the first customer as the financial institution. Palantir software helps the bank to detect fraud before the crime took place. The bank evaluates the contract with Palantir as their best deal over a decade. In December 2009, the bank signed with Palantir a multimillion-dollar contract for several years.
Palantir does not disclose information about their revenues, saying only that since 2008, every year they have tripled. An extremely reliable source who knows the company firsthand, said that revenues exceed $1.7 billion in 2015. According to Karp, in 2010 cash flow became positive.
Karp has founded Palantir along with several researchers from Stanford and former employees of PayPal. The most famous of the founders was Peter Thiel, the billionaire and Facebook investor. Thiel, the largest investor in Palantir, says that the company is about to become the new Facebook or Google. “Its value in a few years will be measured in tens of billions of dollars,” he says. Karp expects $10 billion in revenue in 5 years. Nominally, the company was created with the direct participation of the CIA and other intelligence agencies of the United States. They also were the first clients of the company. Karp and Thiel say that by means of Palantir they are preventing terrorism and violence while preserving the civil liberties. Involved in their work the information is strictly confidential and users have access only to its individual fragments.
Products:
The basic concept of products of the company — analysis and visualization of big-data from disparate sources, allowing users without technical training find the relationship between objects to detect similarities between the objects and events around them, detect abnormal objects — Data Mining with a focus on interactive visual analysis in the spirit of the concept of intelligence amplification. As sources of software Palantir uses both traditional databases and other structured sources such as texts, audio, video.
Gotham — a product targeted for the state security agencies, the original name of the product — Government — reflect its specifics. The Central storage mechanism in Gotham uses the technique of ontologies, by means of which heterogeneous data from multiple sources is equipped with semantic information and unified for a joint analysis.
Metropolis — originally named Finance for its orientation on the financial sector, in contrast to Gotham, focuses more on the processing of numerical information, identifying anomalies and regularities in sets of quantifiable data. Metropolis provides visual analysis capabilities over the controls in the style of Windows Explorer, spreadsheets, regression analysis and time series analysis.
Palantir is different from its competitors by very complicated and sophisticated software on the one hand, and by very intuitive and user-friendly interface on the other. Another difference is that it’s quazi-market company: it was developed with financial support of CIA fund In’Q’Tel and still has its funding. It’s working in a very closed niche – governmental agencies. I’m 100% sure that Palantir is a future success, but will it be beneficial to society and its citizens – this is so far uncertain.
I think the point you raised about operating in a closed niche is an important one. I imagine that Palantir’s future success rests on its ability to share learnings (e.g., algorithms, organizational knowledge) across government and non-government clients. On the one hand, serving government clients has its advantages. In my experience in the defense sector, big brother tends to have a lot of data and has a more receptive attitude towards machine learning approaches; the use cases for big data/cybersecurity emerged pretty early in the defense sector, so it’s more of a proven concept than in other sectors. On the other hand, Palantir may not be able to reuse technologies or knowledge they developed while working with a government client due to the sensitive nature of the data/information. I often wanted to collaborate with people outside of the defense bubble in my last job, and this required very creative structuring of programs/development processes to separate the sensitive from the non-sensitive information/data. So it can be done, but it’s tricky.