Most of the Western media collectively lost their minds at the news last week that an Anthropic researcher had quit because of his concerns that developers genuinely believed superintelligent AI could “kill us all by the end of the decade”.
Jacob Coxon — who had only spent four months at Anthropic, as well as several years at OpenAI — notified The Wall Street Journal before publishing his ultra-viral thread on X.
This made it clear the media frenzy was at least partly orchestrated, and lends credence to the theory it was a marketing tactic to instil a sense of awe and destructive power into the industry’s sycophantic conversation simulators.
That theory, however, is closer to regulatory capture than marketing.
The fossil fuel industry realised in the early 2000s that outright climate science denial wasn’t going to work forever.
It became important to admit that climate science was real, while soberly downplaying its severity and insisting that the fossil fuel industry (experts and hyper-rational technologists that they are) should be in charge of the solutions.
This is why each year’s COP events are still sodden with fossil fuel lobbyists, and why they’ve infiltrated the Intergovernmental Panel on Climate Change for decades.
It is why media outlets so frequently interview fossil fuel company CEOs to comment on energy issues.
It is why carbon capture and storage is a technology immune to its own persistent failures, and why methane gas — responsible for a material chunk of all climate impacts — is still presented and understood as a climate solution.
You can see the parallels in the modern generative AI industry.
Weak, toothless councils and roundtables are packed with former tech employees and adherents writing about superintelligence and extinction, while ensuring actual threats remain unaddressed and profits remain untouched.
We even see AI companies offering AI solutions to AI-caused problems.
Big tech rattles the bell about human extinction because it knows it’ll help capture regulators and allow the companies to self-regulate.
This is part of it, but it’s still not enough to understand the current landscape.
The fossil fuel industry is capitalist, but the AI industry is cultist.
The Anthropic engineer who dramatically quit is the latest public face of a long-running, cult-like movement, originating in Silicon Valley, that is obsessed with the sci-fi risk of deadly computers.
“Cult-like” is probably putting it way too mildly; these people are close to being actual cultists.
“Effective altruism”, “longtermism” and other sub-sects all orbit this central idea that the only real threat is a self-aware language model.
This was laid out....
