From here forward, training data will be synthetic (pre-training era of human internet data is over)

From here forward, training data will be synthetic (pre-training era of human internet data is over) | There was no data ceiling. It was completely elucory... We we've reached orbit. We've reached escape velocity and now it's synthetic data from here on out.

There was no data ceiling. It was completely elucory... We we've reached orbit. We've reached escape velocity and now it's synthetic data from here on out.

Beneficiaries (23)

Adverse (6)

{
  "nia": false,
  "url": "https://www.youtube.com/watch?v=d__HRChE2ZE",
  "mode": "THESIS",
  "role": "Host",
  "context": "There was no data ceiling. It was completely elucory. And I I think history will look back at this moment... Similarly, the internet which was collected by a bunch of fat fingers punching keyboards... was just the biological bootloadader for an era of synthetic data... We've reached escape velocity and now it's synthetic data from here on out.",
  "verbatim": "There was no data ceiling. It was completely elucory... We we've reached orbit. We've reached escape velocity and now it's synthetic data from here on out.",
  "conv_cues": "no data ceiling; escape velocity",
  "direction": "HAPPEN",
  "timeframe": "Ongoing",
  "conv_level": "HIGH",
  "milestones": [
    {
      "kind": "llm_pre_event",
      "label": "Microsoft Research SynthLLM and arXiv 2510.01631 establish synthetic-data scaling laws",
      "notes": "HIT — Microsoft Research published SynthLLM and 'Demystifying Synthetic Data in LLM Pre-training' establishing 33% synthetic / 67% natural as optimal mix.",
      "source": "https://arxiv.org/abs/2510.01631",
      "status": "hit",
      "weight": 0.4,
      "ordinal": -9,
      "source_id": null,
      "confidence": 0.95,
      "source_url": "https://arxiv.org/abs/2510.01631",
      "expected_date": "2025-10-31",
      "observed_date": "2025-10-31",
      "research_origin": "deep_research",
      "measurement_criterion": "Peer-reviewed/preprint research establishes synthetic-data scaling laws for LLM pre-training with quantified mix ratios"
    },
    {
      "kind": "quartile_checkpoint",
      "label": "Q1 window check-in (25%)",
      "status": "pending",
      "weight": 0.05,
      "ordinal": -8,
      "source_id": null,
      "expected_date": "2026-11-01",
      "observed_date": null
    },
    {
      "kind": "llm_pre_event",
      "label": "Frontier lab publicly confirms majority synthetic data in latest model pre-training",
      "notes": "Wissner-Gross's strong claim that pre-training is 'completely synthetic from here on out' — current research suggests 33% mix is near-optimal, not 100%.",
      "source": "https://invisibletech.ai/blog/ai-training-in-2026-anchoring-synthetic-data-in-human-truth",
      "status": "pending",
      "weight": 0.4,
      "ordinal": -7,
      "source_id": null,
      "confidence": 0.45,
      "source_url": "https://invisibletech.ai/blog/ai-training-in-2026-anchoring-synthetic-data-in-human-truth",
      "expected_date": "2027-03-17",
      "research_origin": "deep_research",
      "expected_date_range": {
        "to": "2027-12-31",
        "from": "2026-06-01"
      },
      "measurement_criterion": "OpenAI/Anthropic/Google DeepMind technical report or paper states >=50% of pre-training tokens for flagship model are synthetic"
    },
    {
      "kind": "llm_pre_event",
      "label": "Epoch AI confirms exhaustion of high-quality public text corpus",
      "source": "https://epoch.ai/blog/will-we-run-out-of-data-limits-of-llm-scaling-based-on-human-generated-data",
      "status": "pending",
      "weight": 0.4,
      "ordinal": -6,
      "source_id": null,
      "confidence": 0.7,
      "source_url": "https://epoch.ai/blog/will-we-run-out-of-data-limits-of-llm-scaling-based-on-human-generated-data",
      "expected_date": "2027-03-17",
      "research_origin": "deep_research",
      "expected_date_range": {
        "to": "2027-12-31",
        "from": "2026-06-01"
      },
      "measurement_criterion": "Epoch AI or equivalent research org publishes update confirming utilization of stock of human-generated public text"
    },
    {
      "kind": "quartile_checkpoint",
      "label": "Q2 window check-in (50%)",
      "status": "pending",
      "weight": 0.05,
      "ordinal": -5,
      "source_id": null,
      "expected_date": "2027-05-05",
      "observed_date": null
    },
    {
      "kind": "llm_post_event",
      "label": "Frontier labs continue licensi
... (truncated)