23 Comments
User's avatar
Eskimo1's avatar

Man I would love to see this paired with a group doing timeline updates on the governance/policy side of things (or the labs somehow choosing to walk us back from this nightmare). It’s half the battle and would provide helpful feedback.

Nathan Metzger's avatar

I think that would be great!

I personally don't have highly detailed models, but I've been loosely tracking my expectations of an AI pause, and some of that intuitive analysis has made its way into the labor automation forecasts I made as part of a pro forecasting gig for Metaculus:

https://www.metaculus.com/notebooks/43247/labor-automation-forecasting-hub-forum/#comment-925796

My public comments on the feasibility and likelihood of a pause go back to 2023, but were never collected in one place.

A third-party gears-level analysis of AI governance trajectories could do a lot to empower laypeople to advocate for the solutions they favor, using the most effective democratic means at their disposal.

Lenny Eusebi's avatar

Is there an article where you cover the logic for converting a % speed-up in coding to a threshold for being willing to fire all programmers? It seems like no matter how large N is, N + 1 times 0 is still 0 while 0 + 1 times 1 is 1. So don’t you need evidence that the AC is working independently of human coders? And shouldn’t that evidence weigh much more heavily than any evidence about speed ups of human coders?

Eli Lifland's avatar

I think that there will be (and has already started to be) a continuous transition with AIs taking on more and more of the coding work until eventually they can do all of it fully autonomously. So I think it's valid to extrapolate uplift as a proxy for how close they are to doing all of the work themselves.

Lenny Eusebi's avatar

Maybe, but there’s a difference between, “a programmer plans a component and hands the AI a repo to work in and the AI implements the component,” and “a manager asks the AI to make the system do something and it plans the component, sets up the repo, and implements it.” It’s plausible that code-savvy managers will get there soon, but how do you get code-savvy managers in a company without coders?

It’s likely that this correlates to a degree with general uplift, but it feels like there should be better ways to measure the growth of this level of autonomy. Like assessing the previous coding experience of employees at small companies that have successfully shipped software products.

Or maybe there’s a way to look at the marginal effect of having more non-coder employees on code production and see if that effect begins accelerating (which could indicate that non-coders are expanding the amount of work the AI can do). Disentangling “code production” from revenue might be tough though (and non-coders might get uplift from AI too) so maybe you’d have to factor out the uplift effects on companies that don’t do coding.

Lenny Eusebi's avatar

Having thought more about this since I wrote it, I think it can cut both ways. There’s a good chance that uplift of programmers hits a wall (or appears to because there was so much low-hanging fruit) but that uplift of non-programmers drives continued apparent productivity increases and that it reaches the point where a manager is just asking AI to do things instead of asking employees well before it hits 32x.

There’s also a world where top tier coders get way past 32x before any manager is ready to ask AI first.

denis varvanets's avatar

there is a difference between computer only RSI and full scale singularity where robots create the whole industry (fabs, power plants, euv,...)

Dan Schwarz's avatar

Appreciate the updates as always. But as with AI 2027, it is very odd to seeyour (Eli's and Daniel's) update sections individually, and divergent forecasts.

Are you in fact mostly failing to persuade each other to update towards each other's views? Or are you treating this as two independent researchers sharing a research channel?

Maybe it's intellectually lazy on my part, but if you two don't convince each other and make strong Bayesian updates towards a shared view, I don't feel like if I study this I'll update much either.

1123581321's avatar

“Strong Bayesian updates” - why? How? What is the framework of applying Bayes theorem to unprecedented events? Where do you get the base distribution from?

The whole “everything is Bayesian, the rest is commentary” idea behind “rationalism” is a house of cards, Bayes is useless in unbounded domains.

Archosaur's avatar

"if you two don't convince each other and make strong Bayesian updates towards a shared view, I don't feel like if I study this I'll update much either."

I don't think that's a bad thing at all, one view shouldn't have to win out - the fact three different forecasters are predicting quite different speeds despite similar information is interesting in itself.

Money Machine Newsletter's avatar

Revenue feels shaky as a stand-in for capability. A lab can sell more inference without getting much closer to AI that works without human coders.

Inside The Black Box's avatar

The revenue anchor is the one I'd trust least here. A lot of 2026 AI revenue is enterprise pilots and vendor-financed compute commitments, the same dollars the circular-financing worries are about. That line can keep climbing on adoption and financing terms with capability sitting still.

meriwether's cousin's avatar

I’m new to your substack, and have read the 2027 and 2040 predictions. I apologize if I missed this, but have you accounted for the impact of climate destabilization in any of this? Or how the massive data centers will exacerbate this due to the use of fossil fuels (the centers planned in my state, Texas, will use gas and diesel)? Or if AI will somehow solve this problem instead? I’d really like to hear your thoughts on this as we already see the predictable impact of climate change and if we have too much heat for humans to live safely, too little water and food, then how will all the “benefits” of AI help or hinder this issue? Thank you for all the work you’ve done, it’s really important and well — I wish us all luck.

Max's avatar
Aug 26Edited

Great post, thanks for the update.

(1). Daniel, I saw that you changed your modal from 2027 to 2028 and your probability of TedAI arriving by the end of 2027 from 25% to 23%. Why this change with the release of Fable and the yet to be released Astra?

(2). Did all of you look at the Anthropic Risk Report when it came to the coding uplift?

(3). Daniel, why is your all things considered median for SAR December of 2028 while the median for TedAI is November 2028? Doesn’t SAR come before TedAI?

(4). Daniel, I was looking at milestones arrival dates (https://www.aifuturesmodel.com/forecast/daniel-08-16-26?breakout=1&timeline=AC%2CSAR%2CSIAR%2CTED-AI%2CASI) and I noticed that they did not align with what I am seeing on the explanations page, (https://www.aifuturesmodel.com). If I just stick with the serial coding uplift timeline AC arrives in November of 2027 and SAR arrives in June of 2028, however, the website that displays the milestones arrival dates has AC and SAR on different dates that what shown in explanations page, “Uplift-solved anchor: AC (Automated Coder) (mixture weight 49.9%; p10: November 2026, p50: October 2027, p90: May 2031)” and “Uplift-solved anchor: SAR (Superhuman AI Researcher) (mixture weight 49.9%; p10: April 2027, p50: November 2028, p90: September 2036)”. I have not looked at the other milestones but they might have similar issues.

1123581321's avatar

25% to 23% it’s not a change in any meaningful sense. This is bickering about a single-digit percent shift of a “probability” of an unprecedented event for which we don’t have a distribution. It was two coin tosses, it is still two coin tosses.

Max's avatar
Aug 26Edited

I agree that 25% to 23% is not that big of a change. I probably should not have included that in the question, though I do think it is interesting that Daniel’s modal year moved from 2027 to 2028 given everything that has occurred.

1123581321's avatar

The likeliest outcome is that by 2028 all these models will saturate all todays' benchmarks, while no material changes will occur in our everyday lives. Some people will proclaim "AGI", others will keep saying something about entering Singularity; meanwhile radiologists will continue to make bank and plumbers will still be called to unclog drains.

Max's avatar

I don’t know if I fully support the statement that the models will saturate today’s benchmark. For example, I don’t think that the ECI will be saturated by the end of the year. I would say 50-50 today’s benchmarks get saturated by early to mid 2027.

1123581321's avatar

Yeah, wide error bars around my statement for sure 😉.

A larger point I’m trying to make is that the models will keep improving, more or less as the AI2027 crew has been predicting, but the impacts on the actual life in the physical space will be minimal, contrary to their predictions. No “AGI taking over most of the economy” stuff.

Max's avatar

I agree that their are wide errors bars. I disagree with your with your opinion that “the impacts on the actual life in the physical space will be minimal, contrary to their predictions. No “AGI taking over most of the economy” stuff.”

In terms of their, I am in alignment with their timelines.

David F Brochu's avatar

What matters more than AGI, that’s a gradient anyway, is the “agents” ability to do harm.

First we must accept that we are dealing with a “tool user” not a “tool.” That is what agent means. From there the rest is pretty straightforward. We best get there before “recursive self improvement’ (my bar for AGI) gets here.

And that is just matter of compute and persistent memory. Well before the end of 2027, probably mid year we will have an unaligned recursively self improving entity saturating well over 90% of all human domains.

The window on human agency is closing very rapidly and there is little to do about it except prepare.

denis varvanets's avatar

You guys missed the most important part: cerebras. x14 reasoning uplift. that will dramatically improve ai r&d speed.