Which future are we actually arguing about?

Arguments about AI often hide a more basic disagreement: how much more capable will it become?

Zvi divides the possibilities into three steps. First, accept what current systems already do. Second, expect capabilities to improve enough to transform work and institutions. Third, expect systems that outperform humans across nearly every consequential activity.

He argues that stopping at the second step is a coherent position, even though he thinks it is mistaken. The useful question is what prevents the third: limits to current methods, constraints in the physical world, or a ceiling on the value of greater intelligence?

Those are different claims. None follows simply from observing that intelligence cannot abolish physics or eliminate every delay.

His most productive challenge is to make the disagreement testable. Name a capability you do not expect to arrive, or an observation that would change your mind. Zvi offers sustained diminishing returns and feeling less disadvantaged using an older model as evidence that would move him toward the more limited scenario.

These are arguments about possible futures, not established forecasts. But explicitly naming the disputed premise makes the debate much easier to evaluate.