Decentralised Military Intelligence

The attempted attack on a flight from Amsterdam to Detroit on Christmas Day has been described as a "systematic failure" of the US Intelligence Agencies by President Barack Obama who stated:

"Where our government has information on a known extremist and that information is not shared and acted upon as it should have, so that this extremist boarded a plane with dangerous explosives that could have cost nearly 300 lives, a systemic failure has occurred and I consider that totally unacceptable," Mr Obama said.

"We need to learn from this episode and act quickly to fix the flaws in our system because our security is at stake and lives are at stake."

As with many terrorist attacks, retrospectively it is obvious that intelligence agencies knew about the attack before it occurred. The problem is that the information is disaggregated and decentralised and there is no way of combining and aggregating the information to determine the wisdom of the crowd. Decentralisation is important because it allows for specialisation and the exploration of tacit knowledge. However, aggregation is necessary if the useful information uncovered in one part of the system is to become widely available. James Surowiecki covers this in Chapter 4 of The Wisdom of Crowds.

Section V of Chapter 4 begins:

When it comes to the problem of the U.S. intelligence community before September 11, the problem was not decentralizsation. The problem was the kind of decentralization that the intelligence community was practicising. On the face of it, the division of labor between the different agencies makes a good deal of sense. Specialization allows for a more fine-grained appreciation of information and greater expertise in analysis. And everything we know about decision making suggests that the more diverse the available perspectives on a problem, the more likely it is that the final decision will be smart. Acting Defense Intelligence Agency director Lowell Jacoby suggested precisely this in written testimony before Congress, writing, "Information considered irrelevant noise by one set of analysts may provide critical clues or reveal significant relationships when subjected to analytic scrutiny by another."

What was missing in the intelligence community, though, was any real means of aggregating not just information but also judgements. In other words, there was no mechanism to tap into the collective wisdom of National Security Agency nerds, CIA spooks and FBI agents. There was decentralization but no aggregation, and therefore no organization. Richard Shelby's solution to the problem - creating a truly central intelligence agency - would solve the organization problem, and would make it easier for at least one agency to be in charge of all the information. But it would also forego all the benefits - diversity, local knowledge, independence - that decentralization brings. Shelby was right that information needed to be shared. But he assumed thats someone - or a small group of someones - needed to be at the center, sifting through the information, figuring out what was important and what was not. But everything we know about cognition suggests that a small group of people, no matter how intelligent, simply will not be smarter than the larger group. And the best tool for appreciating the collective significance of the information that the intelligence community had gathered was the collective wisdom of the intelligence community. Centralization is not the answer. But aggregation is.