FUT_001predictionAILLM-human-forecasting-parity-Nov-2026

Algorithmic systems (LLMs) will match the accuracy of the world's most elite human superforecasters by November 2026 per ForecastBench linear-extrapolation of current learning curves; 95% CI stretches Dec 2025 to Jan 2028. Validation metric: LLMs achie...

Predictor: Superforecaster Community

Prior probability

72.0%

Current probability

47.7%

evolves via intake + LBP

Conviction

5/5

Signal quality

Resolution

in_progress

Window

2026-11-01 – 2026-11-30

Edges in / out

1 / 0

Tickers exposed

Prediction text

Algorithmic systems (LLMs) will match the accuracy of the world's most elite human superforecasters by November 2026 per ForecastBench linear-extrapolation of current learning curves; 95% CI stretches Dec 2025 to Jan 2028. Validation metric: LLMs achieve difficulty-adjusted Brier score below the 0.081 elite human benchmark. | Next ForecastBench quarterly Brier update

Key catalyst: Next ForecastBench quarterly Brier update

Watch events: ForecastBench monthly LLM-Brier updates; Metaculus AI tournaments

Resolution evidence

Status: in_progress

ForecastBench empirical tracking shows rapid LLM convergence toward human-superforecaster Brier scores through 2024-2026; GPT-5 / Claude 5 / Gemini 3 closing gap on 1000-question benchmark.

Predictor: Superforecaster Community

κ + Brier as of 2026-05-22

Full calibration →

κ (discount)

0.643

Brier

0.0000

excellent

Hits / Misses

2 / 0

of 2 resolved

Hit rate

100.0%

Calibration plot (stated vs observed)

Evidence about this node from Superforecaster Community is multiplied by κ in /api/intake. Lower κ = less weight; floors at 0.10 (effectively silenced) and caps at 1.00 (full weight).

Reference class

Not linked

This node isn't linked to a reference class. The Bayesian update applies without outside-view blending.

Probability over time

4 prob_history rows

intake v2milestone miss sweeplbp propagationreference class assignedlegacy v1prior_prob (analyst seed)current = 47.7%

Milestone chain

Pre-event signals (upstream prereqs + window checkpoints) → resolution event → downstream cascades. Status/dates update from linked nodes; re-derive nightly via scripts/ops/derive_milestones.py.

Leading chain: 1 fired ✓ · 1 pending

2026-01-18hitGPT-4.5 Brier 0.101 vs human superforecaster 0.081 baseline established
How: ForecastBench / Forecasting Research Institute publishes Brier scores comparing top LLM (GPT-4.5: 0.101) to superforecaster aggregate (0.081)
Source: https://markets.financialcontent.com/stocks/article/predictstreet-2026-1-18-the-great-forecast-convergence-ai-closing-the-20-gap-on-human-superforecastersconf 95%
Notes: HIT — current gap is ~25% (Brier 0.101 vs 0.081), narrowing per linear extrapolation.
2026-04-01 → 2026-09-30pendingLLMs achieve dataset-question parity (Brier <=0.081) by mid-2026
How: ForecastBench dataset-question subset shows top LLM matching superforecaster Brier 0.081 (per 0.020/yr improvement rate)
Source: https://forecastingresearch.substack.com/p/ai-llm-forecasting-model-forecastbench-benchmark — dataset-question parity projected June 2026 (95% CI Nov 2025–Apr 2027)conf 70%
2026-11-20pendingAlgorithmic systems (LLMs) will match the accuracy of the world's most elite human superforecasters by November 2026 per ForecastBench linea
2026-11-30pendingForecastBench leaderboard shows LLM Brier <=0.081 (full benchmark parity)
How: Top model on ForecastBench overall leaderboard achieves difficulty-adjusted Brier <=0.081 (matching XPT 2022 superforecaster aggregate)
Source: https://www.forecastbench.org/leaderboards/conf 50%
Notes: Central prediction date. 95% CI Dec 2025 – Jan 2028 per Forecasting Research Institute extrapolation.
2027-01-01 → 2027-06-30pendingIndependent replication confirms parity (non-market questions)
How: Independent benchmark (Metaculus, Good Judgment Open, Polymarket-controlled) replicates Brier parity excluding market-copy artifacts
Source: https://goodjudgment.com/human-vs-ai-forecasts/ + Metaculus AI tournament reportsconf 50%
Notes: Critical because GPT-4.5 currently has 0.994 correlation with provided market forecasts — inflates ForecastBench performance.
2027-06-01 → 2028-12-31pendingTool-augmented agentic LLM exceeds superforecaster Brier by >=0.01
How: Tool-using LLM agent achieves Brier <=0.071 on ForecastBench (decisive superhuman margin)
Source: ForecastBench tournament + Forecasting Research Institute updatesconf 45%

What if this resolves?

Clamp this prediction TRUE or FALSE and run a counterfactual Gibbs sample. Surfaces the predictions whose marginals shift most under that assumption.

(live posterior: 48%)

Click a button to clamp this prediction and run a Gibbs sample. Returns the predictions whose marginals shift most. ~30s per run; ideal for stress-testing "if X resolves, what else moves?"

Evidence chain

Every probability update with full Bayesian provenance — chronological, latest first

LBP2026-05-24T02:00:02Z47.7%-1.7pp

Network propagation: 49.4% → 47.7%

4-iter LBP, residual 0.01000 · damping 0.5, w_intrinsic 0.5 · method lbp_v3 · run 806b02f8

LBP2026-05-17T02:00:01Z49.4%-3.5pp

Network propagation: 52.9% → 49.4%

5-iter LBP, residual 0.00689 · damping 0.5, w_intrinsic 0.5 · method lbp_v3 · run e607fa96

LBP2026-05-10T02:00:02Z52.9%-6.8pp

Network propagation: 59.8% → 52.9%

6-iter LBP, residual 0.00584 · damping 0.5, w_intrinsic 0.5 · method lbp_v3 · run e5c18d29

LBP2026-05-03T02:00:01Z59.8%-12.2pp

Network propagation: 72.0% → 59.8%

6-iter LBP, residual 0.00677 · damping 0.5, w_intrinsic 0.5 · method lbp_v3 · run 1a683ac9

Network propagation neighbors

Top edges sorted by latest LBP cross-impact

All propagation →

No propagation data yet. Run inference/.venv/bin/python scripts/ops/run_loopy_belief_propagation.py on the droplet, or wait for the Sunday 02:00 UTC weekly cron.

Prerequisites (1)

Predictions that must hit first

Type	Pred	Title	Domain	Lag
correlate	S_AGI_MID_2029	AGI mid: Kurzweil 2029 path	agi_general_capability	—

Dependents (0)

Predictions enabled by this

Type	Pred	Title	Domain	Lag
No dependents

Expected milestones (1)

From Sheet 17 Monitoring Triggers

Expected by	Description	Status
2026-11-30	[Capability 2026-11] [FUT_001] ForecastBench monthly LLM-Brier updates; Metaculus AI tournaments	pending

Validations (1)

Resolution events

Observed at	Status	By	Notes
2026-04-29	partial	thesis_timeline_v1.0_import	ForecastBench empirical tracking shows rapid LLM convergence toward human-superforecaster Brier scores through 2024-2026; GPT-5 / Claude 5 / Gemini 3 closing gap on 1000-question benchmark.

Linked documents (10)

Auto-generated by cosine similarity from Polymarket / Manifold / EDGAR / GDELT

Sim	Source	Title	Market prob	Polarity	Reviewed	Published
0.672	arxiv	Algorithmic algorithm development with LLMs: A Case Study on LLM-Usage for Contraction Order Optimization in Tensor Networks	—	mentions	pending	2026-06-01
0.668	arxiv	Design a Reliable LLM-Integrated Interface for Mortality Forecasting	—	mentions	pending	2026-06-04
0.662	arxiv	Benchmarks in Leipzig	—	mentions	pending	2026-06-04
0.655	arxiv	Resolution Diagnostics for Paired LLM Evaluation	—	mentions	pending	2026-05-28
0.650	arxiv	BenCSSmark: Making the Social Sciences Count in LLM Research	—	mentions	pending	2026-05-06
0.648	arxiv	Learnable Assessment Skills for LLM-based Automated Scoring: Rubric Construction via Iterative Optimization	—	mentions	pending	2026-05-28
0.648	arxiv	Can LLMs Write Correct TLA+ Specifications? Evaluating Natural-Language-to-TLA+ Generation	—	mentions	pending	2026-06-04
0.646	arxiv	Latent Performance Profiling of Large Language Models	—	mentions	pending	2026-05-28
0.644	manifold	Will a US open-weights LLM outrank every Chinese open-weights LLM on LMArena at the end of 2027?	32%	mentions	pending	2026-05-31
0.643	arxiv	Benchmarking LLM-as-a-Judge for Long-Form Output Evaluation	—	mentions	pending	2026-06-01

Raw metadata

From Thesis_Timeline_v1.0_FINAL workbook

{
  "nia": false,
  "qty": "Brier 0.081 parity threshold",
  "mode": "FORECAST",
  "role": "Cited-Other",
  "context": "First entry for Superforecaster Community aggregate (Good Judgment Project, Tetlock, ForecastBench, XPT 2022 tournament). Critical epistemological benchmark for AI-vs-human forecasting. Couples with AI_005 Anthropic-level intelligence, AUT_017 agentic RL, AI_026 Metaculus consensus.",
  "to_year": 2026,
  "conv_cues": "specific month; 95% CI; named benchmark framework",
  "direction": "HAPPEN",
  "from_year": 2026,
  "timeframe": "Nov 2026",
  "conv_level": "HIGH",
  "milestones": [
    {
      "kind": "llm_pre_event",
      "label": "GPT-4.5 Brier 0.101 vs human superforecaster 0.081 baseline established",
      "notes": "HIT — current gap is ~25% (Brier 0.101 vs 0.081), narrowing per linear extrapolation.",
      "source": "https://markets.financialcontent.com/stocks/article/predictstreet-2026-1-18-the-great-forecast-convergence-ai-closing-the-20-gap-on-human-superforecasters",
      "status": "hit",
      "weight": 0.4,
      "ordinal": -2,
      "source_id": null,
      "confidence": 0.95,
      "source_url": "https://markets.financialcontent.com/stocks/article/predictstreet-2026-1-18-the-great-forecast-convergence-ai-closing-the-20-gap-on-human-superforecasters",
      "expected_date": "2026-01-18",
      "observed_date": "2026-01-18",
      "research_origin": "deep_research",
      "measurement_criterion": "ForecastBench / Forecasting Research Institute publishes Brier scores comparing top LLM (GPT-4.5: 0.101) to superforecaster aggregate (0.081)"
    },
    {
      "kind": "llm_pre_event",
      "label": "LLMs achieve dataset-question parity (Brier <=0.081) by mid-2026",
      "source": "https://forecastingresearch.substack.com/p/ai-llm-forecasting-model-forecastbench-benchmark — dataset-question parity projected June 2026 (95% CI Nov 2025–Apr 2027)",
      "status": "pending",
      "weight": 0.4,
      "ordinal": -1,
      "source_id": null,
      "confidence": 0.7,
      "source_url": "https://forecastingresearch.substack.com/p/ai-llm-forecasting-model-forecastbench-benchmark",
      "expected_date": "2026-07-01",
      "research_origin": "deep_research",
      "expected_date_range": {
        "to": "2026-09-30",
        "from": "2026-04-01"
      },
      "measurement_criterion": "ForecastBench dataset-question subset shows top LLM matching superforecaster Brier 0.081 (per 0.020/yr improvement rate)"
    },
    {
      "kind": "event",
      "label": "Algorithmic systems (LLMs) will match the accuracy of the world's most elite human superforecasters by November 2026 per ForecastBench linea",
      "status": "pending",
      "weight": 1,
      "ordinal": 0,
      "source_id": "FUT_001",
      "expected_date": "2026-11-20",
      "observed_date": null
    },
    {
      "kind": "llm_pre_event",
      "label": "ForecastBench leaderboard shows LLM Brier <=0.081 (full benchmark parity)",
      "notes": "Central prediction date. 95% CI Dec 2025 – Jan 2028 per Forecasting Research Institute extrapolation.",
      "source": "https://www.forecastbench.org/leaderboards/",
      "status": "pending",
      "weight": 0.4,
      "ordinal": 1,
      "source_id": null,
      "confidence": 0.5,
      "source_url": "https://www.forecastbench.org/leaderboards/",
      "expected_date": "2026-11-30",
      "research_origin": "deep_research",
      "measurement_criterion": "Top model on ForecastBench overall leaderboard achieves difficulty-adjusted Brier <=0.081 (matching XPT 2022 superforecaster aggregate)"
    },
    {
      "kind": "llm_post_event",
      "label": "Independent replication confirms parity (non-market questions)",
      "notes": "Critical because GPT-4.5 currently has 0.994 correlation with provided market forecasts — inflates ForecastBench performance.",
      "source": "https://goodjudgment.com/human-vs-ai-forecasts/ + Metaculus AI tournament reports",
      "status": "pending",
      "weight": 0.4,

... (truncated)