This algorithm can predict a revolution

Source: The Verge

For students of international conflict, 2013 provided plenty to examine. There was civil war in Syria, ethnic violence in China, and riots to the point of revolution in the Ukraine. For those working at Duke University’s Ward Lab, all specialists in predicting conflict, the year looks like a betting sheet, full of predictions that worked and others that didn’t pan out.
GUERRILLA CAMPAIGNS INTENSIFIED, PROVING OUT THE PREDICTION
When the lab put out their semiannual predictions in July, they gave Paraguay a 97 percent chance of insurgency, largely based on reports of Marxist rebels. The next month, guerrilla campaigns intensified, proving out the prediction. In the case of China’s armed clashes between Uighurs and Hans, the models showed a 33 percent chance of violence, even as the cause of each individual flare-up was concealed by the country’s state-run media. On the other hand, the unrest in the Ukraine didn’t start raising alarms until the action had already started, so the country was left off the report entirely.
According to Ward Lab’s staff, the purpose of the project isn’t to make predictions but to test theories. If a certain theory of geopolitics can predict an uprising in the Ukraine, then maybe that theory is onto something. And even if these specialists could predict every conflict, it would only be half the battle. “It’s a success only if it doesn’t come at the cost of predicting a lot of incidents that don’t occur,” says Michael D. Ward, the lab’s founder and chief investigator, who also runs the blog Predictive Heuristics. “But it suggests that we might be on the right track.”
IF A CERTAIN THEORY OF GEOPOLITICS CAN PREDICT AN UPRISING IN THE UKRAINE, MAYBE THAT THEORY IS ONTO SOMETHING
Forecasting the future of a country wasn’t always done this way. Traditionally, predicting revolution or war has been a secretive project, for the simple reason that any reliable prediction would be too valuable to share. But as predictions lean more on data, they’ve actually become harder to keep secret, ushering in a new generation of open-source prediction models that butt against the siloed status quo.
WILL THIS COUNTRY’S GOVERNMENT FACE AN ACUTE EXISTENTIAL THREAT IN THE NEXT SIX MONTHS?
The story of automated conflict prediction starts at the Defense Advance Research Projects Agency, known as the Pentagon’s R&D wing. In the 1990s, DARPA wanted to try out software-based approaches to anticipating which governments might collapse in the near future. The CIA was already on the case, with section chiefs from every region filing regular forecasts, but DARPA wanted to see if a computerized approach could do better. They looked at a simple question: will this country’s government face an acute existential threat in the next six months? When CIA analysts were put to the test, they averaged roughly 60 percent accuracy, so DARPA’s new system set the bar at 80 percent, looking at 29 different countries in Asia with populations over half a million. It was dubbed ICEWS, the Integrated Conflict Early Warning System, and it succeeded almost immediately, clearing 80 percent with algorithms built on simple regression analysis.
STATISTICS DON’T HAVE TO WORRY ABOUT HURT FEELINGS
Why was it so easy to beat the CIA’s best analysts? To some extent, the answer has more to do with humans than machines. Imagine the agency’s Indonesia expert, for example. He wants to make accurate predictions, but he’s also subject to a range of biases that never show up in the data. He wants his work to be exciting and relevant, earning the attention of his superiors; he wants Indonesia to be important in the world. Predictions are also used to direct resources within the CIA, and he may want to attract more of the resources than the Indonesia bureau would otherwise receive. By the time all the biases are accounted for, he’s doing only slightly better than a coin flip. The statistics, on the other hand, don’t have to worry about internal politics or hurt feelings.
IT’S A MIRROR OF THE SAME OPEN-VS-CLOSED DEBATE IN SOFTWARE
It’s a lesson that conflict prediction has taken to heart, growing more transparent even as the tools become more powerful. ICEWS itself was reclassified and taken back into the secretive corners of the Pentagon, but the most exciting projects right now are fully open endeavors that publish their predictions for anyone to see. On the data side, researchers at Georgetown University are cataloging every significant political event of the past century into a single database called GDELT, and leaving the whole thing open for public research. Already, projects have used it to map the Syrian civil war and diplomatic gestures between Japan and South Korea, looking at dynamics that had never been mapped before. And then, of course, there’s Ward Lab, releasing a new sheet of predictions every six months and tweaking its algorithms with every development. It’s a mirror of the same open-vs.-closed debate in software — only now, instead of fighting over source code and security audits, it’s a fight over who can see the future the best.
Of course, the secretive predictors are still alive and well. Conflict prediction is a lucrative business for national security consultants, even if it’s harder to check their work. And it’s impossible to know what the Pentagon’s working on. They could be beating Ward’s predictions each month — but even if they were, it wouldn’t give them much of an edge over the publicly available information. And just like the CIA’s analysts, they’re handicapped by closed sources and institutional biases. As long as public predictions are getting more outside insight, more revisions, and more scrutiny, it’s easy to like their odds. “Look, it’s a messy and complicated world,” Ward says. “I don’t think we’ll ever get to a place where things are predicted perfectly.” In the meantime, the trick is to keep getting better.