Palantir Technologies spots patterns to solve crimes and track terrorists ~ Oops & every fucking 1 of us. ? :o ...can u say "controller files" ...folks !
This article was taken from the September 2012 issue of
Wired magazine. Be the first to read Wired's articles in print
before they're posted online, and get your hands on loads of
additional content by
On the afternoon of February
15, 2011, Jaime Zapata, a 32-year-old special agent with US
Immigration and Customs Enforcement, was shot dead by members of a
drug cartel as he drove along a four-lane highway in
Mexico.
Zapata's partner, Victor Ávila, 38,
who survived the attack, later said that as many as 15 gunmen
opened fire with automatic rifles, even though Zapata had
identified himself as a diplomat and the armoured SUV in which the
pair were riding had number plates identifying it as an official
vehicle.
Back in Washington DC, senior US
administration officials wanted revenge for what they saw as the
deliberate killing of a federal agent and the attempted murder of
another. Michele Leonhart, head of the US Drug Enforcement
Administration (DEA) and a veteran of the drug wars, asked, "What
can we do to make an immediate impact against the cartels, to send
a message?" recalls Derek Maltz, the special agent in charge of the
DEA's Special Operations Division. "We decided to do a
law-enforcement strike," Maltz says.
To conduct what became known as
Operation Fallen Hero, investigators turned to a little-known
Silicon Valley software company called Palantir Technologies.
Palantir's expertise is in finding connections between people,
places and events in large repositories of electronic data. Federal
agents had a trove of reporting on the drug cartels, their members,
their funding mechanisms and smuggling routes. They had dossiers,
informants' reports, surveillance images, intercepted electronic
communications, footage from drones. But investigators lacked a way
to assemble and share all that intelligence with one another, and
to quickly find leads buried in mountains of
information.
Investigators with Zapata's agency bought Palantir's
software, plugged it into their databases and used it to track down
members of the cartel. The results were astonishing. Palantir
helped to identify connections among key individuals and
organisations. Officials reported that this kind of painstaking
detective work -- reading reports, piecing together clues, drawing
links between people -- would have taken months without
technological assistance. With the help of Palantir, large amounts
of data from disparate sources were analysed within days.
Law-enforcement officers across the
US, Mexico and South America confiscated 467 kilograms of cocaine,
30 kilograms of methamphetamine and 282 weapons, and arrested 676
people -- including the cartel member suspected of killing
Zapata.
Officials were so impressed with Palantir's software
that seven months later they bought licences for 1,150
investigators and analysts across the country. The price, including
training, was £4.8 million a year. The government chose not to seek
a bid from some of Palantir's competitors because, officials said,
analysts had already tried three products and each "failed to
provide the necessary comprehensive solution on missions where our
agents risk life and limb".
As far as Washington was concerned,
only Palantir would do.
Such an endorsement would be
remarkable if it were unique. But over the
past three years, Palantir, whose office in Tysons Corner,
Virginia, is just ten kilometres from the CIA's headquarters, has
become a darling of the US law-enforcement and national-security
establishment -- and its business in the UK is growing. US national
security agencies now use Palantir for numerous variation on the
challenge that bedevilled analysts in Operation Fallen Hero -- how
to organise and catalogue intimidating amounts of data and then
find meaningful insights that defy humans alone.
In the US, Palantir has sold its
software to the CIA, the military's Special Command and the Marine
Corps. The FBI, the Defense Intelligence Agency, the National
Counterterrorism Center and the Department of Homeland Security are
all customers. The director of the National Security Agency has
said that Palantir's software could help the agency to "see" into
cyberspace to defend against hackers and spies attempting to breach
government computer networks. The Los Angeles Police Department
uses Palantir. So does the New York Police Department, whose
intelligence and counterterrorism unit rivals the sophistication of
the FBI and the CIA.
The company also has a small London
office in Covent Garden, and it plans to move to a larger,
930m2 space elsewhere in central London this year. Palantir's
main UK customers are part of "Five Eyes", an international
partnership comprising intelligence and security services from the
UK, the US, Canada, Australia and New Zealand. The member countries
share intelligence with one another that's particularly focused on
cyber security and signals intelligence -- intercepted phone-calls,
emails and other electronic messages. The British "Eye" is the
Secret Intelligence Service (MI6) and the Government Communications
Headquarters (GCHQ), the signals-intelligence
agency.
Palantir is extraordinarily
tight-lipped about its UK customers, much more so than about its
American ones. (British secrecy laws are
much stricter and more readily enforced than in the US.) According
to sources, Palantir has few or no customers in domestic law
enforcement, such as Scotland Yard. However, because British
agencies routinely share foreign and domestic information with one
another, there are domestic security organisations in the UK
receiving the benefit of Palantir without having actually used the
software themselves.
Asher Sinensky, who's in charge of
Palantir's UK business, says the British government will be using
Palantir during the Olympics. "There will be half a million more
people in London every day during the games," he says. "The
transportation infrastructure will be swept in ways that it hasn't
before." He says there's also "a lot of concern" among British
officials about defending borders and controlling who comes in and
out of the country. "These outsiders, we don't have the same record
of their interactions with people as we do with [British] citizens
and others we've been tracking."
Palantir was launched in 2004 by Alex Karp, a
financial adviser with a PhD in neoclassical social theory and no
experience running a technology company, and Peter Thiel, a
billionaire venture capitalist who'd helped start PayPal and was an
early investor in Facebook. Karp, a self-described progressive,
knew Thiel, a prominent libertarian, from their days at Stanford
Law School. After 9/11, Karp had reconnected with Thiel, who had
the idea that Silicon Valley should do something to improve
national security and secure civil liberties. Karp, who is tall and
thin with a bushy crop of hair, has none of the swagger of so many
Silicon Valley entrepreneurs. He's a restless academic, more at
home in a seminar than a pitch meeting. During
an interview at his office in Tysons Corner, he stands
up, paces the room for a few moments, sits down, gets up again and
starts sketching out diagrams and graphs on a white board to
explain how Palantir works.
They and three other cofounders
wanted to use PayPal's fraud-detection technology as the model for
a new counterterrorism software, which would be used by analysts to
crunch huge amounts of data.
The parallels between moving money
and fighting al-Qaeda might not be immediately obvious, but the
ascendancy of PayPal, which was founded in 1998, was largely
because of its ability to prevent criminals from stealing its
customers' money. Engineers designed an algorithm that let human
fraud experts -- many of whom were former law-enforcement officers
-- quickly sift through transaction data, look into the transaction
network and map out connections among suspected criminals. That
approach made PayPal the world's most trusted system for online
payments. In 2002, the company was acquired by eBay for £964
million. "The bread and butter of PayPal was to look at a
transaction and to know if it came from a bad IP address," says Bob
McGrew, Palantir's director of engineering. That same approach
became the heart of Palantir, which Karp says is an "attribution"
software, meaning it's used for finding the people behind pieces of
data.
Palantir wanted to beat terrorists the way PayPal
beat Russian criminal gangs. Thiel, who's something of a policy
dilettante, also thought the idea could make a lot of money.
Palantir is considered one of the most valuable startups in the US
-- VC firm Globespan
Capital Partners estimates Palantir's market valuation to be as
high as £2.5 billion. A top executive at JPMorgan Chase says it's
poised to do for information locked inside organisations what
Google did for information on the web.
Before Karp and Thiel ever signed their first
contract, they sought counsel from some of the most important
national-security experts in Washington. Not long after the company
was officially formed, in 2004, the two cofounders met with John
Poindexter, a former national-security adviser to Ronald Reagan, at
the home of Richard Perle, who was chairman of the Defense Policy Board, a group of influential Pentagon advisers,
in 2001.
"I told them I thought they had an
interesting idea," says Poindexter, who, from 2002 to 2003, ran a
Defense Department initiative called Total
Information Awareness, which bore striking similarities to
Palantir's approach to data analysis. The programme was shut down
following outcries from privacy activists -- TIA proposed to
mine not just government intelligence databases, but privately held
records such as credit-card transactions, email and phone
records.
Experts such as Poindexter helped Palantir open
doors. In a short time, the company has assembled a legion of
advocates from the most influential strata of government. Karp
counts former CIA director George Tenet as a friend; he says the
same about Tenet's employer, Herb Allen, who runs the enigmatic
investment bank Allen
& Co, a Palantir investor. And another top adviser, Bryan
Cunningham, was a CIA intelligence officer and a senior staffer to
former secretary of state Condoleezza Rice.
Early prospective investors were
sceptical of Palantir. Venture-capital firms were looking
for the next Facebook or Google -- most presumed it would come in
the form of a consumer technology, probably a new social-media
site. No one wanted to back an expensive software platform for
large organisations. The Palantir founders met several
venture-capital firms, and all turned them down. "We believed it
would work," Karp says. "No one else did."
Potential investors were also leery
of Washington. "The government was unpopular in Silicon Valley,"
Karp says. At the end of one failed pitch meeting, an investor
who'd turned him down said there was a group he should talk to
"that does this kind of thing". The group was In-Q-Tel, the venture-capital arm of
the CIA that was set up in 1999 to bypass the cumbersome government
procurement process and to fund technologies that might be useful
to intelligence agencies.
Palantir got a meeting with
In-Q-Tel's CEO, Gilman Louie, a former computer-game designer.
Stephen Cohen, one of Palantir's founders, was 22 at the time, but
had been writing code since he was a teenager. He spent the next
eight weeks with another cofounder, hammering out a version of
Palantir they could take to the meeting. They worked -- and slept
-- in an office Thiel had used when he founded PayPal.
In-Q-Tel invested a relatively
insignificant amount of money -- about £1.3 million, a small chunk
of the nearly £26 million that Karp says Palantir's investors spent
before the company saw its first dollar in revenue. But it led to a
meeting with another interested backer, the venture wing of Reed
Elsevier, the publishing and information conglomerate. One of
its partners saw Karp give a presentation at an In-Q-Tel meeting,
and was so impressed that he invested a few million dollars.
Crucially, In-Q-Tel put Palantir's founders in the room with
frontline US intelligence analysts, the people they hoped would use
their product. The analysts gave Palantir the software equivalent
of a test drive. "They'd say, 'I love that, I hate that,'" Karp
explains.
The founders spent the next three years flying to
Washington, taking notes and then returning to Palo Alto to tweak
the software. Cohen says he was getting "most of my calories from
Red Bull".
Karp estimates that he and Cohen had more than 300
meetings with likely users, people far down the government
hierarchy. The Silicon Valley techies found themselves deep in an
unfamiliar culture: some people introduced themselves only by their
first names and refused to say where they worked in the
government.
While shuttling between the coasts, Palantir's
founders discovered that intelligence analysts wanted a way to
search their own databases and to know what their colleagues in
other agencies had available. But just as important, agencies
needed to restrict access, so that only those with the proper
security clearances could, for instance, look at the video of a
drone attack or read a classified interrogation summary. Palantir
developed a method for indexing information so the system would
match up a particular data point with the user's security
clearances. If he didn't have the authority to read it, the
information was unavailable. This technique had the added benefit
of creating an audit trail of what the Palantir users were reading,
whether they'd handled the information properly, and whether they'd
modified it in any way.
It's difficult to overstate the
importance of this security regime. Without such nuanced controls
-- down to the level of a single person or one nugget of
intelligence -- the kinds of collaboration necessary to prevent
terrorist attacks just won't happen. An audit trail like this also
lets analysts check their own prior judgments to see if there was a
flaw in their logic.
Palantir also developed a way to organise data that
spoke to a great yearning in the spy world: the need to quickly
assimilate new information into an unfolding narrative. Once data
is put into Palantir, the software uses a model called "dynamic
ontology" to show how names, places and events relate to one
another. For instance, imagine a suspected terrorist who's being
tracked by MI6 makes contact with someone whom the service hasn't
seen yet. That person's name goes into Palantir's system, and the
entire dossier on the original target changes to account for any
previously unseen connections between the two. The network of
relationships between the target and anyone the new person knows
can be seen as well. Every time an analyst adds a new piece of
data, the picture changes automatically. And this new picture can
be shared with other analysts using the software.
No comments:
Post a Comment