Office of the Director of National Intelligence
Intelligence Advanced Research Projects Activity
Intelligence Value
FOCUS enhanced analysis tradecraft by improving analysts’ ability to learn lessons from the past. FOCUS helped analysts correctly answer the question, “how would history have been different if actor X had instead done action Y on date Z?”
Summary
Counterfactual forecasts -- statements about what would have happened if different circumstances had occurred -- are the foundation for lessons about what to do in the future. For example, analysts performing a postmortem review of an intelligence failure might conclude the Intelligence Community could have avoided the failure if it had employed different methods based on prior experience and best practices. Unfortunately, research across multiple disciplines has concluded that experience-based best practices are generally ineffective. There has been no systematic prior research on improving counterfactual reasoning or lessons learned. The FOCUS program aimed to develop practical, accurate, evidence-based techniques for counterfactual forecasting.
Because we can’t re-run history in the real world, FOCUS methods were evaluated by having hundreds of people make predictions about outcomes in complex simulations.