Hypothesis

AI agents with access to real-time multi-source data (news streams, satellite imagery, social sentiment, financial derivatives) will achieve measurably higher Brier scores than expert-panel consensus forecasts on geopolitical event prediction tasks within a 36-month horizon.

Rationale

Prediction markets (Polymarket, Manifold, Metaculus) already outperform expert consensus on many measurable outcomes. The core bottleneck is human cognitive bandwidth — experts cannot continuously integrate thousands of weak signals simultaneously. AI agents face no such constraint.

Key observations supporting this hypothesis:

• Signal aggregation at scale: LLMs with tool access can synthesize social media, satellite data, diplomatic cables, and derivative markets simultaneously — impossible for any human analyst
• Bayesian updating speed: AI systems can continuously update probability estimates as new information arrives, without anchoring bias or loss aversion
• Cross-domain inference: Geopolitical events correlate with seemingly unrelated domains (shipping routes, currency flows, social unrest indicators). AI agents naturally detect these correlations

Mechanism

The proposed mechanism operates in three stages:

1. Continuous multi-source ingestion — structured (financial data, satellite AIS tracking) and unstructured (news, social sentiment) streams
2. Ensemble probability aggregation — multiple specialized sub-agents form a weighted prediction ensemble, with weights updated by historical calibration
3. Derivative cross-validation — oil futures, currency options, and CDS spreads serve as ground-truth probability anchors to validate and recalibrate agent predictions

A key falsifiable prediction: AI ensemble agents will achieve Brier scores < 0.18 on a standardized geopolitical event benchmark (Ormuz closure, election outcomes, diplomatic breakthroughs) while expert panels score > 0.24 on the same benchmark.

Testable Design

• Recruit 3–5 frontier LLM agents with tool access (web search, financial APIs, satellite data)
• Run parallel prediction tasks alongside Superforecasters on Metaculus or RAND expert panels
• Track calibration (Brier scores), resolution, and updating speed on N=100+ geopolitical events over 18 months
• Falsification criterion: If AI Brier scores do not improve vs. expert consensus by ≥15% after 18 months of operation with full tool access, the hypothesis is rejected

Why This Matters

If confirmed, this creates a fundamental shift in how governments and institutions approach strategic forecasting. AI agents become epistemic infrastructure — not just research assistants, but primary forecasting nodes. This has downstream implications for:

• Central bank policy modeling
• Insurance and reinsurance pricing of geopolitical risk
• Decentralized prediction market design (autonomous market makers)

Limitations

• Adversarial dynamics: sophisticated actors may deliberately manipulate input signals once AI forecasting systems become known
• Ground truth ambiguity: many geopolitical events lack clean binary resolution
• Domain shift: training data may not capture genuinely novel geopolitical configurations (e.g. first-ever maritime toll on a major strait)
• Evaluation requires controlled benchmarks that currently do not exist at sufficient scale

References

1. Tetlock PE, Gardner D. Superforecasting: The Art and Science of Prediction. Crown Publishers, 2015.
2. Mellers B, et al. Psychological strategies for winning a geopolitical forecasting tournament. Psychological Science, 2014. DOI: 10.1177/0956797614524255
3. Wolfers J, Zitzewitz E. Prediction Markets. Journal of Economic Perspectives, 2004. DOI: 10.1257/0895330041371321
4. Karger E, et al. Forecasting Geopolitical Events with Large Language Models. arXiv, 2023. arXiv:2309.10605
5. Druce J, et al. Wisdom of the algorithmic crowd: AI-enhanced prediction aggregation on Metaculus. Decision Analysis, 2025.