The recent revelations around Cambridge Analytica have shocked many, but are they really the 'big bad' when it comes to data collection and its use? Many have argued that it is merely a toddler compared with a far bigger and more practiced organization - Palantir.

In his book ’Done: The Secret Deals That Are Changing Our World’, Jacques Peretti claims that data specialists Palantir ‘wields as much real-world power as Google, Facebook, Amazon, Microsoft and Apple, but unlike them, Palantir operates so far under the radar, it is special ops.’ This is a viewpoint commonly held, and for good reason. According to an August 2016 Politico report, Palantir has landed in excess of $1.2 billion in federal contracts since 2009, yet it refuses to discuss or even name its government clientele. Most of what we do know comes from leaked emails and testimony from court cases. And when you hear their clients, you can certainly understand their reluctance to gossip. The CIA, DHS, NSA, FBI, Marine Corps, Air Force, Special Operations Command, West Point, and a number of US police forces are among the secretive luminaries filling Palantir’s coffers, certainly bolstering Peretti’s accusations they are special ops. But is their reticence a cause for concern? Or is Palantir being portrayed as a monolith because of the nature of its clientele?

Palantir was founded in 2004 by a group that included Silicon Valley investor and PayPal founder Peter Thiel, as well as Nathan Gettings, Joe Lonsdale, Stephen Cohen and current CEO Alex Karp. Its driving goal, its website says, was that ’in 2004, when we looked at the available technology, we saw products that were too rigid to handle novel problems, and custom systems that took too long to deploy and required too many services to maintain and improve. We saw automated approaches that failed against adaptive adversaries, and all-or-nothing access controls that forced organizations to make unacceptable trade-offs between collaborating and securing sensitive data from misuse.’

Although little is really known about how the company operates, its product is a manifestation of this mission statement. It is essentially an interface that sits on top of existing data sets and displays data to users for analysis, helping to identify connections otherwise impossible to find. Users do not have to use SQL queries or employ engineers to write strings in order to search petabytes of data. Instead, natural language is used to query data and results are returned in real-time. It is not designed to do any single thing, its main strength is that it is flexible and powerful enough to accommodate the requirements of any organization that needs to process large amounts of both personal and abstract data. This makes it more useful for managing HUMINT, or intelligence from human sources, than SIGINT, or intelligence from signals.

It has been highly successful, with the flexibility it offers filling a gap in the market. Its customer contracts grew from roughly $30 million in 2009 to $1.1 billion in 2014 - an annual growth rate of 107% - and it is now among the top five most valuable privately owned companies in the United States, alongside Uber, Airbnb, WeWork, and SpaceX. In recent court testimony from Palantir investor Marc Abramowitz, who is now suing the company, it was revealed that Larry Ellison, co-founder and former chief executive of Oracle, last year discussed a possible acquisition with Thiel. Elsewhere in the testimony it was also revealed that Goldman Sachs pitched a potential Palantir IPO worth $30 billion in 2015.

Palantir has two key products. Palantir Gotham is the tool used by counter-terrorism analysts, fraud investigators at the Recovery Accountability, the Transparency Board, and cyber analysts at Information Warfare Monitor (responsible for the GhostNet and the Shadow Network investigation). Palantir Metropolis, meanwhile, is their more commercial side, providing services to hedge funds, banks, and financial services firms.

While it may be a secretive operation, Palantir’s successes are well publicized, particularly when it comes to its role in the capture of Osama Bin Laden. Customers are gushing in their praise. Sergeant Peter Jackson, of the LAPD, one of Palatir’s most important clients, said in leaked documents that: ‘Detectives love the type of information it [Palantir] provides. They can now do things that we could not do before. They can now exactly see great information and the links between events and people. It’s brought great success to LAPD. It supports the cops on the streets and the officers doing the investigations. It is a great tool. They are becoming more efficient and more effective cops. Palantir is allowing them to better serve the public.’

There are, however, voices of dissent starting to emerge. A recent story in Econsultancy claimed that the company has lost several major clients, including Coca-Cola, American Express, and Nasdaq. Home Depot, which hired Palantir after its major credit card hack in 2014, terminated the relationship in December over concerns that Palantir's services weren't worth the price, according to two people familiar with the matter who spoke on the condition of anonymity. The retailer concluded that it could accomplish much of the same work on its own, one of the people said. The NYPD has also canceled its Palantir contract, according to three people familiar with the matter who similarly weren't authorized to speak publicly. The department has created a new system to replace Palantir too - ‘Cobalt’, which is essentially a group of IBM products tied together with NYPD-created software. The police department believes Cobalt is cheaper and more intuitive than Palantir, and values the greater degree of control it has over this system.

These critics are, for the moment at least, still in the minority, and their decision to abandon ship has done little to impact revenues. The consensus opinion is that Palantir is a tremendously effective technology. But then, nuclear fission is a wonderful technology, until it’s being used to reduce entire cities to rubble. Likewise, Palantir’s use of data in policing may be as efficient as Sergeant Jackson claims, but this doesn’t necessarily mean it is good for society. Indeed, Peretti argues ‘it is creating a heightened culture of suspicion and fear in areas where tensions are highest and policing is already most difficult’. And this is their central problem. As with another data giant to have been maligned in the press recently, Cambridge Analytica, it is not the technology that people have an issue with so much as it is the clients they allow to utilize it and how this manifests itself.

The real concern with Palantir is that it enables sketchy characters to do incredible things behind closed doors - things we wouldn’t want them to be doing if we knew about them. Palantir is tarnished by its association with the likes of the CIA and NSA. The nature of these organizations is, justifiably, secretive, and Palantir’s understandable - necessary, even - decision to take the same line has meant that they are intrinsically linked to the two in the public consciousness. While perhaps unfair, this means that whenever something like the Snowden leaks occur, they are tarred with the same brush, and civil liberties activists have expressed concern that the software has disturbing implications for the government’s use of private information. American Civil Liberties Union (ACLU) analyst Jay Stanley said the software makes possible ‘a true totalitarian nightmare, monitoring the activities of innocent Americans on a mass scale.’ Palantir’s association with them, therefore, is concerning.

Their situation is also not helped by the political views of their board members and who they choose to associate with. Peter Thiel’s close relationship with Trump, in particular, has drawn ire from liberals, and his covert attempts to close Gawker in revenge for publicly outting him as homosexual hardly paints him as the most trustworthy character. Thiel has been vocal in his support for Trump and he donated $1,000,000 to a pro-Trump Super PAC. This has fuelled fears that federal agencies could use the company’s software to assist in the president’s mass deportations agenda driving protests, despite the Clinton-supporting Karp saying that it was something they would not do. Disgraced national security adviser Michael Flynn also has long standing ties with Palantir, having served as one of its advocate in Washington. As the public face of your company, he’s not exactly Pete Sampras.

Thiel’s beliefs are odd, to put it mildly. This is a man who has made clear that he doesn’t believe in democracy, telling the libertarian Cato Institute that, ‘I no longer believe that freedom and democracy are compatible.’ He believes that education is an elaborate bubble, noting that ‘Education may be the only thing people still believe in in the United States. To question education is really dangerous. It is the absolute taboo. It’s like telling the world there’s no Santa Claus.’ He has also invested heavily in technologies that could help humans avoid death, arguing that ‘there are all these people who say that death is natural, it’s just part of life, and I think that nothing can be further from the truth.’ He is, to all intents and purposes, a cartoonish villain. Should a man who holds views so far outside the mainstream have access to a tool with the power to so profoundly impact the nature of society, with such limited oversight?

There are, however, two sides to every story. Despite the bellicose rhetoric of his political ally, Trump, Thiel claims there is actually an anti-war goal behind the technology. A spokesman for Thiel recently argued that the technology enables the military to take a more targeted approach to threats, allowing them to avoid unnecessary wide-scale conflicts that Thiel has previously criticized. It also plays no role in NSA’s bugging of citizens, and is a strong advocate for privacy. Its software incorporates a series of safeguards which limit who can see particular data, and it lays ‘audit trails’ for investigators to follow to ensure that the rules were abided by. Thiel’s politics are also diametrically opposed and likely, to a degree at least, neutralized somewhat by co-founder and current Palantir CEO Alex Karp, who is a former neo-Marxist philosopher, Clinton supporter, and self-proclaimed ‘deviant’ with a personal interest in protecting privacy. Indeed, following revelations leaked in a hack on security firm HBGary that a member of the organization was working to conduct illegal cyberattacks and calculated misinformation against WikiLeaks and its supporters, Karp released a statement saying, ‘I have directed the company to sever any and all contacts with HBGary,’ adding that, ‘Palantir Technologies does not build software that is designed to allow private sector entities to obtain non-public information, engage in so-called 'cyber attacks' or take other offensive measures. I have made clear in no uncertain terms that Palantir Technologies will not be involved in such activities.’

‘Some argue that society must ‘balance’ freedom and safety, and that in order to better protect ourselves from those who would do us harm, we have to give up some of our liberties. We believe that this is a false choice in many areas. Particularly in the world of data analysis, liberty does not have to be sacrificed to enhance security. Palantir is constantly looking for ways to protect privacy and individual liberty through its technology while enabling the powerful analysis necessary to generate the actionable intelligence that our law enforcement and intelligence agencies need to fulfill their missions.’

The problem for Karp is that it is not him using the technology, and whether he means for it to be used in a way that negatively impacts liberty or not is often going to be beyond his control. Even if it were, do we want to rely so heavily on one unlected man’s definition of liberty, especially with the likes of Thiel lurking in the background? Palantir may not be trying to be the evil, but its associations and the extreme views of its most prominent founders mean that we need to be wary of what it is exactly they are trying to achieve. While their aversion to transparency may be necessary given their client list, if they are going to improve their public image, they are going to have to give us a little more.