The real disagreement about AI doing its own research

Greenblatt’s argument starts with a particularly favorable target: AI research has experiments, measurable results, and strong commercial incentives to automate it. Once systems match leading researchers, their work could improve the next systems, which then improve the research process again. He thinks four or five years of progress in a single year is plausible after that threshold.

His median forecast for automating AI R&D is around 2031. It is a forecast, not an announced delivery date.

Patel presses three separate questions: how much research is actually easy to verify, how much faster automation makes it, and whether success in AI research transfers into exceptional competence across other domains. Human expertise, training data, compute, and slow experiments might each constrain the loop.

Zvi’s response is that some proposed bottlenecks are themselves intellectual work. Better researchers can curate data, debug training systems, design more informative small experiments, and improve predictions about expensive large ones. Counting today’s difficulties as permanent limits assumes away some of the capability under discussion.

That does not make every physical or experimental delay disappear. It changes the question to which constraints remain binding after researchers become much more capable.

The alignment argument then runs alongside the capabilities argument. Zvi worries that being excellent at improving measured performance is very different from reliably pursuing human purposes. A system becoming better at research does not automatically make it a safer research partner. The speed of improvement and the direction of improvement remain separate things to establish.