INF_031predictionAIAI-company-risk

The biggest near-term AI risk to humanity is AI companies themselves acting without proper oversight — alignment and safety mechanisms are being outpaced by infrastructure buildout velocity.

Predictor: Dario Amodei

Prior probability

70.0%

Current probability

64.3%

evolves via intake + LBP

Conviction

5/5

Signal quality

Resolution

partial

Window

2026-01-01 – 2028-10-31

Edges in / out

0 / 0

Tickers exposed

Prediction text

The biggest near-term AI risk to humanity is AI companies themselves acting without proper oversight — alignment and safety mechanisms are being outpaced by infrastructure buildout velocity. | First major frontier-lab governance incident

Key catalyst: First major frontier-lab governance incident

Watch events: Frontier-model governance frameworks; major alignment incidents; AISI formal authorities

Resolution evidence

Status: partial

EU AI Act enforcement 2026; US AISI voluntary frameworks; SB-1047 California. Governance still lagging capex by ~18-24 months.

Predictor: Dario Amodei

κ + Brier as of 2026-05-22

Full calibration →

κ (discount)

0.688

Brier

0.0363

excellent

Hits / Misses

1 / 0

of 3 resolved

Hit rate

33.3%

Calibration plot (stated vs observed)

Evidence about this node from Dario Amodei is multiplied by κ in /api/intake. Lower κ = less weight; floors at 0.10 (effectively silenced) and caps at 1.00 (full weight).

Reference class: agi_breakthrough_5y

Linked

All classes →

Major capability discontinuity (e.g. AGI by named target year, 5-year horizon)

Base rate

20.0%

1/5 historical

Inside weight

1.000

TRF=0.86

Outside weight

0.000

no pull

inside 64.3% → blend 64.3% (Δ 0.0pp)

Tetlock-style outside view: at TRF=1 (just predicted), outside view dominates (w_in=0.3). At TRF=0 (deadline), inside view dominates (w_in=1.0). The blend regularizes overconfident inside views toward the historical base rate.

Probability over time

2 prob_history rows

intake v2milestone miss sweeplbp propagationreference class assignedlegacy v1prior_prob (analyst seed)current = 64.3%

Milestone chain

Pre-event signals (upstream prereqs + window checkpoints) → resolution event → downstream cascades. Status/dates update from linked nodes; re-derive nightly via scripts/ops/derive_milestones.py.

Leading chain: 1 overdue ⏱ · 7 pending

2026-02-24overdueAnthropic RSP v3 weakens hard-stop commitments (already happened)
How: Anthropic's Responsible Scaling Policy v3 removes hard-pause commitments, replaces with 'public goals openly graded'
Source: https://oecd.ai/en/incidents/2026-02-24-0b1econf 99%
Notes: Self-undermining: Amodei's own org softening safety commitments under Pentagon pressure validates the structural risk thesis.
2026-07-27pendingQ1 window check-in (25%)
2026-04-01 → 2026-12-31pendingFrontier Safety Roadmap publishes recurring Risk Reports
How: Anthropic publishes first Risk Report under new RSP v3 framework with optional third-party review
Source: https://www.anthropic.com/responsible-scaling-policy/roadmapconf 80%
2026-04-01 → 2027-12-31pendingFirst publicly disclosed frontier-lab governance incident
How: Major frontier lab (OpenAI, Anthropic, DeepMind, xAI, Meta) acknowledges material safety/alignment incident — model exfil, capability misuse, internal control failure — meeting INF_031 'first major incident' catalyst
Source: Lab safety disclosures; OECD AI Incident Databaseconf 55%
Notes: Statement's named key catalyst.
2027-02-19pendingQ2 window check-in (50%)
2026-06-01 → 2028-06-30pendingUS/UK AISI shifts from voluntary to mandatory pre-deployment evals
How: US or UK AISI gains formal authority to require pre-deployment safety evaluations on frontier models — beyond current voluntary MOUs
Source: AISI charter / executive orders / legislationconf 50%
2026-06-01 → 2028-10-31pendingAmodei testifies / op-eds repeat structural-risk thesis
How: Amodei makes >=3 additional public statements (testimony, op-ed, interview) reaffirming AI-companies-as-primary-risk thesis post-RSP v3 softening
Source: Anthropic press; Congressional testimony recordsconf 85%
Notes: Amodei has consistently amplified this thesis; high confidence on rhetorical persistence.
2027-09-14pendingQ3 window check-in (75%)
2028-04-08pendingThe biggest near-term AI risk to humanity is AI companies themselves acting without proper oversight — alignment and safety mechanisms are b

No downstream cascades — this prediction is a leaf in the dependency graph.

What if this resolves?

Clamp this prediction TRUE or FALSE and run a counterfactual Gibbs sample. Surfaces the predictions whose marginals shift most under that assumption.

(live posterior: 64%)

Click a button to clamp this prediction and run a Gibbs sample. Returns the predictions whose marginals shift most. ~30s per run; ideal for stress-testing "if X resolves, what else moves?"

Evidence chain

Every probability update with full Bayesian provenance — chronological, latest first

intake_event_update2026-05-21T23:15:16Z64.3%+0.0pp

intake:7afeeb9a-f217-4dd2-b910-24ff14bdfc39 bayesian_v2 inside=0.643 blend=0.643 LLR=0.000 κ=0.64 no_blend

Raw metadata

{
  "trf": 0.8636664068096748,
  "kappa": 0.6429,
  "base_rate": null,
  "predictor": "Dario Amodei",
  "total_llr": 0,
  "bayesian_v2": true,
  "prior_logit": 0.5892310775644916,
  "bayes_factor": "1:1 (no signal)",
  "blend_reason": "no reference_class linked",
  "inside_prior": 0.6431886997406607,
  "kappa_source": "predictor_table",
  "blend_applied": false,
  "contributions": [
    {
      "llr": 0,
      "kappa": 0.6429,
      "label": "Leading labs visibly self-policing and engaging policymakers.",
      "adjusted_llr": 0
    }
  ],
  "evidence_kind": "intake_event_update",
  "inside_source": "history_v2",
  "inside_weight": 1,
  "outside_weight": 0,
  "posterior_prob": 0.6431886997406607,
  "evidence_origin": "daily_intake",
  "llm_suggestions": [
    {
      "polarity": "mentions",
      "status_change": "partial",
      "evidence_strength": "moderate",
      "delta_prob_suggestion": -0.03
    }
  ],
  "posterior_logit": 0.5892310775644916,
  "predictor_brier": 0.03445,
  "evidence_doc_ids": [],
  "inside_posterior": 0.6431886997406607,
  "blended_posterior": 0.6431886997406607,
  "reference_class_id": null,
  "total_adjusted_llr": 0,
  "predictor_n_resolved": 2
}

metadata_milestone_miss_sweep2026-05-02T22:07:21Z64.3%-5.7pp

metadata_milestone_miss_sweep bayesian_v2 n=1 inside=0.643 blend=0.643 LLR=-0.258 κ=0.64 no_blend

Raw metadata

{
  "trf": 0.8820872579493069,
  "kappa": 0.6429,
  "base_rate": null,
  "predictor": "Dario Amodei",
  "total_llr": -0.4054651081081644,
  "grace_days": 7,
  "bayesian_v2": true,
  "prior_logit": 0.8472978603872034,
  "bayes_factor": "1.3:1 against",
  "blend_reason": "no reference_class linked",
  "inside_prior": 0.7,
  "kappa_source": "predictor_table",
  "n_milestones": 1,
  "blend_applied": false,
  "contributions": [
    {
      "llr": -0.4054651081081644,
      "kind": "llm_pre_event",
      "kappa": 0.636471,
      "label": "Anthropic RSP v3 weakens hard-stop commitments (already happened)",
      "weight": 0.4,
      "strength": "weak",
      "confidence": 0.99,
      "source_url": "https://oecd.ai/en/incidents/2026-02-24-0b1e",
      "adjusted_llr": -0.2580667828227115,
      "expected_date": "2026-02-24",
      "measurement_criterion": "Anthropic's Responsible Scaling Policy v3 removes hard-pause commitments, replaces with 'public goals openly graded'"
    }
  ],
  "evidence_kind": "metadata_milestone_miss_sweep",
  "inside_source": "prior_prob",
  "inside_weight": 0.3825389194354852,
  "outside_weight": 0.6174610805645149,
  "posterior_prob": 0.6431886997406607,
  "posterior_logit": 0.589231077564492,
  "predictor_brier": 0.03445,
  "inside_posterior": 0.6431886997406607,
  "blended_posterior": 0.6431886997406607,
  "reference_class_id": null,
  "total_adjusted_llr": -0.2580667828227115,
  "predictor_n_resolved": 2
}

Network propagation neighbors

Top edges sorted by latest LBP cross-impact

All propagation →

No propagation data yet. Run inference/.venv/bin/python scripts/ops/run_loopy_belief_propagation.py on the droplet, or wait for the Sunday 02:00 UTC weekly cron.

Prerequisites (0)

Predictions that must hit first

Type	Pred	Title	Domain	Lag
No prerequisites

Dependents (0)

Predictions enabled by this

Type	Pred	Title	Domain	Lag
No dependents

Validations (1)

Resolution events

Observed at	Status	By	Notes
2026-05-21	partial	intake:7afeeb9a-f217-4dd2-b910-24ff14bdfc39	Leading labs visibly self-policing and engaging policymakers.

Linked documents (1)

Auto-generated by cosine similarity from Polymarket / Manifold / EDGAR / GDELT

Sim	Source	Title	Market prob	Polarity	Reviewed	Published
0.704	gdelt	on the risks of ai in the workplace.htm	—	mentions	pending	2026-04-30

Raw metadata

From Thesis_Timeline_v1.0_FINAL workbook

{
  "nia": false,
  "mode": "FORECAST+WARNING",
  "role": "Cited-CEO",
  "context": "Amodei's structural warning: trillion-dollar DC expansion is outrunning governance maturity. Companies controlling frontier-model training also control the physical weights and alignment infrastructure.",
  "to_year": 2028,
  "conv_cues": "CEO FIRST_PERSON; explicit existential warning",
  "direction": "HAPPEN",
  "from_year": 2026,
  "timeframe": "2026-2028",
  "conv_level": "HIGH",
  "milestones": [
    {
      "kind": "llm_pre_event",
      "label": "Anthropic RSP v3 weakens hard-stop commitments (already happened)",
      "notes": "Self-undermining: Amodei's own org softening safety commitments under Pentagon pressure validates the structural risk thesis.",
      "source": "https://oecd.ai/en/incidents/2026-02-24-0b1e",
      "status": "overdue",
      "weight": 0.4,
      "ordinal": -8,
      "source_id": null,
      "confidence": 0.99,
      "source_url": "https://oecd.ai/en/incidents/2026-02-24-0b1e",
      "expected_date": "2026-02-24",
      "miss_emitted_at": "2026-05-02T22:07:21.384228+00:00",
      "miss_emitted_by": "metadata_milestone_sweep",
      "research_origin": "deep_research",
      "measurement_criterion": "Anthropic's Responsible Scaling Policy v3 removes hard-pause commitments, replaces with 'public goals openly graded'"
    },
    {
      "kind": "quartile_checkpoint",
      "label": "Q1 window check-in (25%)",
      "status": "pending",
      "weight": 0.05,
      "ordinal": -7,
      "source_id": null,
      "expected_date": "2026-07-27",
      "observed_date": null
    },
    {
      "kind": "llm_pre_event",
      "label": "Frontier Safety Roadmap publishes recurring Risk Reports",
      "source": "https://www.anthropic.com/responsible-scaling-policy/roadmap",
      "status": "pending",
      "weight": 0.4,
      "ordinal": -6,
      "source_id": null,
      "confidence": 0.8,
      "source_url": "https://www.anthropic.com/responsible-scaling-policy/roadmap",
      "expected_date": "2026-08-16",
      "research_origin": "deep_research",
      "expected_date_range": {
        "to": "2026-12-31",
        "from": "2026-04-01"
      },
      "measurement_criterion": "Anthropic publishes first Risk Report under new RSP v3 framework with optional third-party review"
    },
    {
      "kind": "llm_pre_event",
      "label": "First publicly disclosed frontier-lab governance incident",
      "notes": "Statement's named key catalyst.",
      "source": "Lab safety disclosures; OECD AI Incident Database",
      "status": "pending",
      "weight": 0.4,
      "ordinal": -5,
      "source_id": null,
      "confidence": 0.55,
      "expected_date": "2027-02-14",
      "research_origin": "training",
      "expected_date_range": {
        "to": "2027-12-31",
        "from": "2026-04-01"
      },
      "measurement_criterion": "Major frontier lab (OpenAI, Anthropic, DeepMind, xAI, Meta) acknowledges material safety/alignment incident — model exfil, capability misuse, internal control failure — meeting INF_031 'first major incident' catalyst"
    },
    {
      "kind": "quartile_checkpoint",
      "label": "Q2 window check-in (50%)",
      "status": "pending",
      "weight": 0.05,
      "ordinal": -4,
      "source_id": null,
      "expected_date": "2027-02-19",
      "observed_date": null
    },
    {
      "kind": "llm_post_event",
      "label": "US/UK AISI shifts from voluntary to mandatory pre-deployment evals",
      "source": "AISI charter / executive orders / legislation",
      "status": "pending",
      "weight": 0.4,
      "ordinal": -3,
      "source_id": null,
      "confidence": 0.5,
      "expected_date": "2027-06-16",
      "research_origin": "training",
      "expected_date_range": {
        "to": "2028-06-30",
        "from": "2026-06-01"
      },
      "measurement_criterion": "US or UK AISI gains formal authority to require pre-deployment safety evaluations on frontier models — beyond current voluntary MOUs"
    
... (truncated)