Thursday, March 19, 2015

Palantir Technologies spots patterns to solve crimes and track terrorists                                  ~ Oops & every fucking 1 of us. ? :o  ...can u say "controller files"    ...folks !

This article was taken from the September 2012 issue of Wired magazine. Be the first to read Wired's articles in print before they're posted online, and get your hands on loads of additional content by 
On the afternoon of February 15, 2011, Jaime Zapata, a 32-year-old special agent with US Immigration and Customs Enforcement, was shot dead by members of a drug cartel as he drove along a four-lane highway in Mexico.
Zapata's partner, Victor Ávila, 38, who survived the attack, later said that as many as 15 gunmen opened fire with automatic rifles, even though Zapata had identified himself as a diplomat and the armoured SUV in which the pair were riding had number plates identifying it as an official vehicle.
Back in Washington DC, senior US administration officials wanted revenge for what they saw as the deliberate killing of a federal agent and the attempted murder of another. Michele Leonhart, head of the US Drug Enforcement Administration (DEA) and a veteran of the drug wars, asked, "What can we do to make an immediate impact against the cartels, to send a message?" recalls Derek Maltz, the special agent in charge of the DEA's Special Operations Division. "We decided to do a law-enforcement strike," Maltz says. 
To conduct what became known as Operation Fallen Hero, investigators turned to a little-known Silicon Valley software company called Palantir Technologies. Palantir's expertise is in finding connections between people, places and events in large repositories of electronic data. Federal agents had a trove of reporting on the drug cartels, their members, their funding mechanisms and smuggling routes. They had dossiers, informants' reports, surveillance images, intercepted electronic communications, footage from drones. But investigators lacked a way to assemble and share all that intelligence with one another, and to quickly find leads buried in mountains of information.
Investigators with Zapata's agency bought Palantir's software, plugged it into their databases and used it to track down members of the cartel. The results were astonishing. Palantir helped to identify connections among key individuals and organisations. Officials reported that this kind of painstaking detective work -- reading reports, piecing together clues, drawing links between people -- would have taken months without technological assistance. With the help of Palantir, large amounts of data from disparate sources were analysed within days.
Law-enforcement officers across the US, Mexico and South America confiscated 467 kilograms of cocaine, 30 kilograms of methamphetamine and 282 weapons, and arrested 676 people -- including the cartel member suspected of killing Zapata.
Officials were so impressed with Palantir's software that seven months later they bought licences for 1,150 investigators and analysts across the country. The price, including training, was £4.8 million a year. The government chose not to seek a bid from some of Palantir's competitors because, officials said, analysts had already tried three products and each "failed to provide the necessary comprehensive solution on missions where our agents risk life and limb".
As far as Washington was concerned, only Palantir would do.
Such an endorsement would be remarkable if it were unique. But over the past three years, Palantir, whose office in Tysons Corner, Virginia, is just ten kilometres from the CIA's headquarters, has become a darling of the US law-enforcement and national-security establishment -- and its business in the UK is growing. US national security agencies now use Palantir for numerous variation on the challenge that bedevilled analysts in Operation Fallen Hero -- how to organise and catalogue intimidating amounts of data and then find meaningful insights that defy humans alone.
In the US, Palantir has sold its software to the CIA, the military's Special Command and the Marine Corps. The FBI, the Defense Intelligence Agency, the National Counterterrorism Center and the Department of Homeland Security are all customers. The director of the National Security Agency has said that Palantir's software could help the agency to "see" into cyberspace to defend against hackers and spies attempting to breach government computer networks. The Los Angeles Police Department uses Palantir. So does the New York Police Department, whose intelligence and counterterrorism unit rivals the sophistication of the FBI and the CIA.
The company also has a small London office in Covent Garden, and it plans to move to a larger, 930m2 space elsewhere in central London this year. Palantir's main UK customers are part of "Five Eyes", an international partnership comprising intelligence and security services from the UK, the US, Canada, Australia and New Zealand. The member countries share intelligence with one another that's particularly focused on cyber security and signals intelligence -- intercepted phone-calls, emails and other electronic messages. The British "Eye" is the Secret Intelligence Service (MI6) and the Government Communications Headquarters (GCHQ), the signals-intelligence agency. 
Palantir is extraordinarily tight-lipped about its UK customers, much more so than about its American ones. (British secrecy laws are much stricter and more readily enforced than in the US.) According to sources, Palantir has few or no customers in domestic law enforcement, such as Scotland Yard. However, because British agencies routinely share foreign and domestic information with one another, there are domestic security organisations in the UK receiving the benefit of Palantir without having actually used the software themselves. 
Asher Sinensky, who's in charge of Palantir's UK business, says the British government will be using Palantir during the Olympics. "There will be half a million more people in London every day during the games," he says. "The transportation infrastructure will be swept in ways that it hasn't before." He says there's also "a lot of concern" among British officials about defending borders and controlling who comes in and out of the country. "These outsiders, we don't have the same record of their interactions with people as we do with [British] citizens and others we've been tracking."
Palantir was launched in 2004 by Alex Karp, a financial adviser with a PhD in neoclassical social theory and no experience running a technology company, and Peter Thiel, a billionaire venture capitalist who'd helped start PayPal and was an early investor in Facebook. Karp, a self-described progressive, knew Thiel, a prominent libertarian, from their days at Stanford Law School. After 9/11, Karp had reconnected with Thiel, who had the idea that Silicon Valley should do something to improve national security and secure civil liberties. Karp, who is tall and thin with a bushy crop of hair, has none of the swagger of so many Silicon Valley entrepreneurs. He's a restless academic, more at home in a seminar than a pitch meeting. During an interview at his office in Tysons Corner, he stands up, paces the room for a few moments, sits down, gets up again and starts sketching out diagrams and graphs on a white board to explain how Palantir works. 
They and three other cofounders wanted to use PayPal's fraud-detection technology as the model for a new counterterrorism software, which would be used by analysts to crunch huge amounts of data. 
The parallels between moving money and fighting al-Qaeda might not be immediately obvious, but the ascendancy of PayPal, which was founded in 1998, was largely because of its ability to prevent criminals from stealing its customers' money. Engineers designed an algorithm that let human fraud experts -- many of whom were former law-enforcement officers -- quickly sift through transaction data, look into the transaction network and map out connections among suspected criminals. That approach made PayPal the world's most trusted system for online payments. In 2002, the company was acquired by eBay for £964 million. "The bread and butter of PayPal was to look at a transaction and to know if it came from a bad IP address," says Bob McGrew, Palantir's director of engineering. That same approach became the heart of Palantir, which Karp says is an "attribution" software, meaning it's used for finding the people behind pieces of data.
Palantir wanted to beat terrorists the way PayPal beat Russian criminal gangs. Thiel, who's something of a policy dilettante, also thought the idea could make a lot of money. Palantir is considered one of the most valuable startups in the US -- VC firm Globespan Capital Partners estimates Palantir's market valuation to be as high as £2.5 billion. A top executive at JPMorgan Chase says it's poised to do for information locked inside organisations what Google did for information on the web.
Before Karp and Thiel ever signed their first contract, they sought counsel from some of the most important national-security experts in Washington. Not long after the company was officially formed, in 2004, the two cofounders met with John Poindexter, a former national-security adviser to Ronald Reagan, at the home of Richard Perle, who was chairman of the Defense Policy Board, a group of influential Pentagon advisers, in 2001. 
"I told them I thought they had an interesting idea," says Poindexter, who, from 2002 to 2003, ran a Defense Department initiative called Total Information Awareness, which bore striking similarities to Palantir's approach to data analysis. The programme was shut down following outcries from privacy activists -- TIA proposed to mine not just government intelligence databases, but privately held records such as credit-card transactions, email and phone records. 
Experts such as Poindexter helped Palantir open doors. In a short time, the company has assembled a legion of advocates from the most influential strata of government. Karp counts former CIA director George Tenet as a friend; he says the same about Tenet's employer, Herb Allen, who runs the enigmatic investment bank Allen & Co, a Palantir investor. And another top adviser, Bryan Cunningham, was a CIA intelligence officer and a senior staffer to former secretary of state Condoleezza Rice. 
Early prospective investors were sceptical of Palantir. Venture-capital firms were looking for the next Facebook or Google -- most presumed it would come in the form of a consumer technology, probably a new social-media site. No one wanted to back an expensive software platform for large organisations. The Palantir founders met several venture-capital firms, and all turned them down. "We believed it would work," Karp says. "No one else did."
Potential investors were also leery of Washington. "The government was unpopular in Silicon Valley," Karp says. At the end of one failed pitch meeting, an investor who'd turned him down said there was a group he should talk to "that does this kind of thing". The group was In-Q-Tel, the venture-capital arm of the CIA that was set up in 1999 to bypass the cumbersome government procurement process and to fund technologies that might be useful to intelligence agencies.
Palantir got a meeting with In-Q-Tel's CEO, Gilman Louie, a former computer-game designer. Stephen Cohen, one of Palantir's founders, was 22 at the time, but had been writing code since he was a teenager. He spent the next eight weeks with another cofounder, hammering out a version of Palantir they could take to the meeting. They worked -- and slept -- in an office Thiel had used when he founded PayPal.
In-Q-Tel invested a relatively insignificant amount of money -- about £1.3 million, a small chunk of the nearly £26 million that Karp says Palantir's investors spent before the company saw its first dollar in revenue. But it led to a meeting with another interested backer, the venture wing of Reed Elsevier, the publishing and information conglomerate. One of its partners saw Karp give a presentation at an In-Q-Tel meeting, and was so impressed that he invested a few million dollars. Crucially, In-Q-Tel put Palantir's founders in the room with frontline US intelligence analysts, the people they hoped would use their product. The analysts gave Palantir the software equivalent of a test drive. "They'd say, 'I love that, I hate that,'" Karp explains.
The founders spent the next three years flying to Washington, taking notes and then returning to Palo Alto to tweak the software. Cohen says he was getting "most of my calories from Red Bull".
Karp estimates that he and Cohen had more than 300 meetings with likely users, people far down the government hierarchy. The Silicon Valley techies found themselves deep in an unfamiliar culture: some people introduced themselves only by their first names and refused to say where they worked in the government.
While shuttling between the coasts, Palantir's founders discovered that intelligence analysts wanted a way to search their own databases and to know what their colleagues in other agencies had available. But just as important, agencies needed to restrict access, so that only those with the proper security clearances could, for instance, look at the video of a drone attack or read a classified interrogation summary. Palantir developed a method for indexing information so the system would match up a particular data point with the user's security clearances. If he didn't have the authority to read it, the information was unavailable. This technique had the added benefit of creating an audit trail of what the Palantir users were reading, whether they'd handled the information properly, and whether they'd modified it in any way.
It's difficult to overstate the importance of this security regime. Without such nuanced controls -- down to the level of a single person or one nugget of intelligence -- the kinds of collaboration necessary to prevent terrorist attacks just won't happen. An audit trail like this also lets analysts check their own prior judgments to see if there was a flaw in their logic.
Palantir also developed a way to organise data that spoke to a great yearning in the spy world: the need to quickly assimilate new information into an unfolding narrative. Once data is put into Palantir, the software uses a model called "dynamic ontology" to show how names, places and events relate to one another. For instance, imagine a suspected terrorist who's being tracked by MI6 makes contact with someone whom the service hasn't seen yet. That person's name goes into Palantir's system, and the entire dossier on the original target changes to account for any previously unseen connections between the two. The network of relationships between the target and anyone the new person knows can be seen as well. Every time an analyst adds a new piece of data, the picture changes automatically. And this new picture can be shared with other analysts using the software.

No comments:

Post a Comment