This document is amazing. I don't think I've read scenario analysis this detailed anywhere. And it is so good to finally read something like this. Whenever I took an AI course, read about predictions, I kept asking: but what does that mean, how does that actually happen, what is the chain, why would it work like that? This paper maps it all out.
So the rest of what I say this is not really a critique, more a view on which assumptions are not made explicit: the ones about the politics of it all.
I think this dynamic has a general and a specific dimension.
On the general side, the paper seems to assume that by 2028 the political leadership will have adopted the AI safety community's framework about AI — maybe because facts on the ground forced them to. By framework I mean not just that AI matters, but that there is a race to superintelligence, so anything we do about AI has to reflect our relationship with China and its dynamics. That AI development is inseparable from the competition with China. Without accepting this framework, the connection of AI and China are simply not made. These Are two completely separate issues in a politician's mind..
I know this sounds absurd, but the notion of super intelligence sounds more absurd from where your general presidential candidate is sitting. Politicians are not rational policy-wise — they follow a different rationality. Look at what is happening today. China is clearly the most important foreign policy challenge even today. Does that seem to shape policy? The United States just launched a war against Iran, intervened in Venezuela, has a love-hate relationship with Putin, and sometimes wants to occupy Greenland.
But the more important element is the specific one: the AI companies. Many scenarios are considered, but the one where the US government simply takes over the AI companies is absent, even though the political motivations for it are strong.
"Both presidential candidates keep getting asked what they'll do about AI, and try out increasingly dramatic ideas on the campaign trail" the paper says. Exactly. Having concluded that Congress is worried and that AI companies are too powerful, why doesn't the government — with or without a shutdown — just take over? (It would serve the Butlerian Jihad notion of Plan S even better.)
The legal authority exists in the he Defense Production Act and nationalization precedents And American politics is capable of far more drastic moves than takeover: the United States once banned alcohol outright — a decision I'm sure many of the regulators personally disliked, and a bad one. Neither stopped it from happening. A
And especially if AI is the most salient issue, there is an escalation race The one with the most simple solution wins. I won't quote here data how much people dislike AI companies, Silicon Valley, chief executives and AI in general.The takeover of AI companies simply outflanks anything else.
In the end, I think a case can be made that you cannot both have a deal with China and have private companies owning. If you believe that super intelligence is coming and is as dangerous as we believe, I am very doubtful that the government would leave it in the hands of private companies. I'm not even sure that once weaponization is concerned, it would be constitutional to do so. And if you don't believe in super intelligence then you have no reason for a deal with China.
Of course, we all know what predictions are worth, so I may be wrong on many of these. But on one thing I'm fairly confident: the political dimension deserves more elaborate treatment — even if that is nothing more than listing the assumptions the paper relies on.
> According to our economic model, world GDP grows roughly 200x orders of magnitude during the 2030s (and would grow more if not for the severe limits on AI progress and robot production negotiated by the governments of the world). Because the human population isn’t increasing much during this period, and because of the Citizen’s Dividend, real wealth for the average human also goes up by about 200x.
This document is amazing. I don't think I've read scenario analysis this detailed anywhere. And it is so good to finally read something like this. Whenever I took an AI course, read about predictions, I kept asking: but what does that mean, how does that actually happen, what is the chain, why would it work like that? This paper maps it all out.
So the rest of what I say this is not really a critique, more a view on which assumptions are not made explicit: the ones about the politics of it all.
I think this dynamic has a general and a specific dimension.
On the general side, the paper seems to assume that by 2028 the political leadership will have adopted the AI safety community's framework about AI — maybe because facts on the ground forced them to. By framework I mean not just that AI matters, but that there is a race to superintelligence, so anything we do about AI has to reflect our relationship with China and its dynamics. That AI development is inseparable from the competition with China. Without accepting this framework, the connection of AI and China are simply not made. These Are two completely separate issues in a politician's mind..
I know this sounds absurd, but the notion of super intelligence sounds more absurd from where your general presidential candidate is sitting. Politicians are not rational policy-wise — they follow a different rationality. Look at what is happening today. China is clearly the most important foreign policy challenge even today. Does that seem to shape policy? The United States just launched a war against Iran, intervened in Venezuela, has a love-hate relationship with Putin, and sometimes wants to occupy Greenland.
But the more important element is the specific one: the AI companies. Many scenarios are considered, but the one where the US government simply takes over the AI companies is absent, even though the political motivations for it are strong.
"Both presidential candidates keep getting asked what they'll do about AI, and try out increasingly dramatic ideas on the campaign trail" the paper says. Exactly. Having concluded that Congress is worried and that AI companies are too powerful, why doesn't the government — with or without a shutdown — just take over? (It would serve the Butlerian Jihad notion of Plan S even better.)
The legal authority exists in the he Defense Production Act and nationalization precedents And American politics is capable of far more drastic moves than takeover: the United States once banned alcohol outright — a decision I'm sure many of the regulators personally disliked, and a bad one. Neither stopped it from happening. A
And especially if AI is the most salient issue, there is an escalation race The one with the most simple solution wins. I won't quote here data how much people dislike AI companies, Silicon Valley, chief executives and AI in general.The takeover of AI companies simply outflanks anything else.
In the end, I think a case can be made that you cannot both have a deal with China and have private companies owning. If you believe that super intelligence is coming and is as dangerous as we believe, I am very doubtful that the government would leave it in the hands of private companies. I'm not even sure that once weaponization is concerned, it would be constitutional to do so. And if you don't believe in super intelligence then you have no reason for a deal with China.
Of course, we all know what predictions are worth, so I may be wrong on many of these. But on one thing I'm fairly confident: the political dimension deserves more elaborate treatment — even if that is nothing more than listing the assumptions the paper relies on.
Any chance the narration by mr. Miles< https://www.youtube.com/channel/UCLB7AzTwc6VFZrBsO2ucBMg >
somehow lands on YouTube for convenience? The embedded narration player felt more than clunky.
Potential typo in footnote 99:
> According to our economic model, world GDP grows roughly 200x orders of magnitude during the 2030s (and would grow more if not for the severe limits on AI progress and robot production negotiated by the governments of the world). Because the human population isn’t increasing much during this period, and because of the Citizen’s Dividend, real wealth for the average human also goes up by about 200x.
200x orders of magnitude? That can't be right...
Yes, that is a typo. Output grows by a factor of 200, not 10^200. Thank you!