Assumptions & weak signals: how to build a stronger empirical basis for reasoning about the pace of AI progress
A proposal to surface the AI research and safety fields' key assumptions about what drives AI progress, and the weak signals that would tell us if they're holding.
Summary
The AI field is sharply divided about how fast AI progress - and with it, the possibility of RSI - is coming. These divisions between short and long timelines run deep: experts are often reasoning from different, sometimes unexamined assumptions about what actually drives progress (sample efficiency, scaling, generalizability, and more). Our dominant measurement tools - benchmarks, evaluations, and forecasts - attempt to gauge model capability and survey expert opinion, but don't surface or test those underlying assumptions. This means that researchers, policymakers, and funders have to make consequential decisions - about how fast to scale, what regulations to pursue, and the most important agendas to support - on shaky foundations. This project proposes a structured, mixed-methods study to surface and synthesize key assumptions about the drivers of AI progress across the field, including in China and the global south, and to identify the weak signals that would tell us whether each is holding or breaking - thereby delivering a stronger, more evaluable basis for reasoning about pace, and for updating as evidence emerges. This project is deliberately intended to sit upstream of other existing efforts: it aims to go beyond benchmarks and forecasts, and help deliver the empirical grounding we need for better measurement, stronger forecasts, and effective governance decisions.
The problem
AI capabilities are advancing fast. But exactly how fast, and how close we are to a point where AI could begin automating AI R&D - potentially igniting an intelligence explosion - is hotly contested. The AI research and safety fields split into two broad camps. On one side, short-timeline advocates (the authors of Kokotajlo, Alexander, Larsen, Lifland, & Dean - AI 2027, Aschenbrenner - Situational Awareness - The Decade Ahead, the leaders of most frontier labs) argue that artificial general intelligence (AGI) is imminent, set to arrive by 2030, with artificial superintelligence (ASI) to follow shortly thereafter. Others, like Narayanan and Kapoor, think the pace of progress is likely to be much slower, with AGI at least a decade away, if not more, be that due to fundamental limitations with the underlying Transformer architecture, or the fact that AI is simply a normal technology.
These camps have different conclusions. But more than that, they are reasoning from different underlying assumptions about the causal mechanisms that are most important to AI progress, and often siloed by domain and/or geography. For example, Narayanan and Kapoor think that Sample efficiency - and how inefficient current AI systems are relative to humans - will hold back the pace of progress. Others, like Cotra and Kokotajlo, argue that current trends in scaling - in which parameters, Compute efficiency, and algorithmic efficiency are all improving rapidly (though to different degrees) - will be sufficient to deliver AGI soon. These people, all of whom are experts, are relying on different assumptions - some tacit, many explicit - about how AI progress emerges. Despite some reporting that picks out some key assumptions (Bye - The data bottleneck could slow the superintelligence race), the field lacks a systematic basis for naming, synthesizing across, and monitoring and assessing which of these assumptions hold most weight. As a result, those working on AI, from researchers to regulators to civil society groups, do not have a strong, consistent basis on which to reason about the pace of what's coming, and make decisions accordingly.
Why it matters
The dominant tools available for assessing model capability - capability evaluations, including benchmarks - and forecasting future trends don't resolve these discrepancies. Existing tools might help to provide a strong sense of point in time capabilities, and for surveying different perspectives about the future. They do not, however, help us triangulate and track the likely pace of AI progress. And they often fail to incorporate the views of people in China and the global south.
This means that the AI community is left to make consequential decisions - about what to fund, about what research bets to prioritize, and what governance interventions will be most effective in averting catastrophic risks - on shaky foundations. To make better decisions, the AI safety and research fields need a better grounded, more empirically evaluable model for making sense of what's actually driving progress, and whether and how it's emerging, and at what pace. This initiative aims to deliver exactly that model, and help the field update as evidence emerges, rather than remaining divided in irreconcilable timeline camps.* Note: this project is not about producing policy directly. It's focused on delivering a strong information base, on which others working on AI safety, including policy, can build scaffolds for making better decisions
What I propose
To fill this gap, I propose a structured, mixed methods research project that aims to surface and synthesize the key assumptions the global field - including those working in China and elsewhere - holds about the most important factors that drive AI progress. The project consists of three initial phases, with the potential for additional phases being bolted on upon completion:
Phase 1: Surface and synthesize key assumptions
- Comprehensive literature review of the AI safety and AI research literature, through the specific lens of identifying what assumptions about the drivers of progress are being made
- An expert survey, conducted using the Delphi technique, applied to experts across the field and - crucially - across geographies, such that we reflect the perspectives of academics, thinkers, and researchers across the timelines spectrum, and across the world, including China
- Robust key informant interviews with a representative sub sample of survey respondents Collected data across these three workstreams would be thematically coded, analyzed, and synthesized into a manageable and informative set of key assumptions that most characterize and shape the field's collective thinking about the drivers of progress and pace.* Note: "most important" in this context doesn't necessarily mean "shared," though that is likely part of what we would want to pull out of the evidence. We're also looking for compelling outliers that may not yet be reflected in many timelines discussions, with a target of ~10-15 assumptions for the sake of tractability
Phase 2: Derive weak signals of change
For each key assumption, identify the weak signals - "the first symptom of change or a sign of an emerging phenomenon that may be significant in the future" (for more, see Sitra) - that would enable us to assess whether and to what extent the assumption is holding or breaking. By the end of phase 2, we would have not just a set of key assumptions, but a clear framework against which to assess each of those assumptions moving forward.
Phase 3: Monitor, test, and reassess
This new framework based on assumptions and weak signals would become the basis for a new monitoring effort. That effort - which would ideally be picked up by, or folded into the work of organizations already doing great monitoring, evaluation, and benchmarking, like EPOCH, Apollo Research, or METR - would then drive the collection and use of evidence to assess our key assumptions.
What weak signals might make possible in the future
Completing phases 1-3 could seed subsequent work to convene global stakeholders across the timelines spectrum for participatory sensemaking processes aimed at generating incompletely theorized agreements (building on the work of Stix & Maas - Bridging the Gap - the case for an Incompletely Theorized Agreement on AI policy) to guide the governance of AI systems. This is not, however, the core of my initial proposal, and should be seen as an extra step. Regardless, delivering a stronger evidence base for assessing the pace of AI progress would benefit the field even without jumpstarting policy.
Where this initiative fits in the broader safety landscape
- It goes deeper than existing expert forecasts: forecasts of AI timelines and risks have been around for years (AI Impacts, for example. Longitudinal Expert AI Panel for another). But this project goes one level deeper than existing surveys and forecasts, seeking to make visible and synthesize the causal assumptions that often remain implicit in those forecast results, and derive monitorable, weak signals.
- It sits upstream of existing measurement efforts: Most existing measurement efforts focus on AI system capability, and most forecasts aim to make visible what experts predict will happen. This work seeks to dig deeper, and pull out the assumptions that underpin existing measurement models and predictions, such that we can assess the underlying thinking on which existing methods tacitly rely.
- Relevant for scenario scrutiny: The AI Futures Project's notion of scenario scrutiny Kokotajlo, Alexander, Larsen, Lifland, Greenblatt, Halstead, & Dean - AI 2040 (Plan A) is intended to help stress-test policy proposals against potential future scenarios. This initiative sits upstream of scenario scrutiny, seeking to interrogate assumptions about the future itself. In this light, weak signals would help ground the premises that scenario scrutiny and other measurement efforts depend on.
Why me
This work sits at the core of what I've spent my career doing with major foundations seeking to understand, navigate, and shape complex systems across the world: using mixed methods to surface the tacit thinking and assumptions that guide strategy, and developing rigorous, testable methods that draw from systems thinking, futures and foresight, and participatory sensemaking to measure things that are hard to assess, and inform decisionmaking and action. I can't out benchmark Epoch or METR - but surfacing, synthesizing, and assessing the assumptions that underpin thinking about the pace of progress is neglected, plausibly more important, and genuinely tractable for me.
Open questions
- Does this feel demonstrably different from existing evaluation, benchmarking, and forecasting efforts? From my perspective, the answer to this question is yes - and that carrying out this kind of exercise would help us to make existing methods even more useful - but I want to sensecheck the gap I'm specifying.
- Would measurement orgs and decisionmakers actually use a synthesized assumptions and weak signals monitoring framework? Would this add value to their work, or feel like just another framework measuring something that isn't decision relevant?
- Would it be useful to explore a minimum viable version (such as, for example, a light version of phase 1) to test value before pursuing this full agenda?
References
Stix & Maas - Bridging the Gap - the case for an Incompletely Theorized Agreement on AI policy Cotra - Ajeya Cotra on whether it’s crazy that every AI company’s safety plan is ‘use AI to make AI safe’ (80k Hours Podcast) Toner - In search of a dynamist vision for safe superhuman AI Toner - Long timelines to advanced AI have gotten crazy short Delaney - Strategic Visions in AI Governance Nanda - Become a person who actually does things AI Futures Project (AI Futures Project) Kokotajlo, Alexander, Larsen, Lifland, Greenblatt, Halstead, & Dean - AI 2040 (Plan A) Bye - The data bottleneck could slow the superintelligence race