The Coming Technological Singularity
Vinge coined the Singularity as a near-term horizon beyond which superhuman intelligence makes prediction impossible, framing the urgency that drives alignment timelines today.
AI existential risk
The case for and against catastrophic risk from advanced AI—power-seeking, takeover, and superintelligence—across books, papers, and film.
Browse the full interactive library →
Is Power-Seeking AI an Existential Risk?
Carlsmith builds a step-by-step argument for why sufficiently capable AI systems may converge on power-seeking behavior, making the x-risk case rigorous and actionable.
Reframing Superintelligence
Drexler challenges monolithic AGI assumptions and proposes that advanced AI could emerge as an ecosystem of specialized services, changing the risk landscape and governance strategies.
Gradual Disempowerment: Systemic Existential Risks from Incremental AI Development
Kulveit et al. reframe existential risk away from sudden takeover, arguing that purely incremental, individually rational AI adoption could permanently erode human influence over the economy, culture, and government without any single dramatic failure.
An International Agreement to Prevent the Premature Creation of Artificial Superintelligence
MIRI's Technical Governance Team sketches a concrete US-China coalition and verification regime, compute tracking, chip governance, and legal prohibitions, for halting the race toward superintelligence before it can be built safely.
Lens Academy
Free, nonprofit AI safety course focused on misaligned superintelligence—why it is the central risk and why alignment is hard—delivered online with a 1-on-1 AI tutor, guided group discussions, and no application process.
Superintelligence
Bostrom's definitive academic text rigorously maps the strategies, kinetics, and dangers of an intelligence explosion, making the case that alignment is civilization-critical.
Human Compatible
Russell argues the standard AI paradigm of optimizing fixed objectives is fundamentally dangerous, proposing instead that machines should defer to uncertain human preferences.
Life 3.0
Tegmark maps concrete governance and alignment choices that determine whether advanced AI expands human agency or permanently concentrates power.
Uncontrollable: The Threat of Artificial Superintelligence
McKee synthesizes the core x-risk arguments into an accessible, urgent case for why superintelligence governance and alignment research cannot wait.
The Precipice (Chapter on AI)
Ord situates AI among existential risks and argues our current governance capacity is dangerously inadequate for the transformative systems being built.
Global Catastrophic Risks
The foundational edited volume on existential and global risks, including AI, widely cited in alignment curricula as the starting point for cross-risk thinking.
If Anyone Builds It, Everyone Dies
Yudkowsky and Soares compress two decades of MIRI argument into a blunt, bestselling case that building superintelligence with anything like current techniques means human extinction.
Good Luck, Have Fun, Don't Die
A man from a future annihilated by a runaway AI loops back through time to recruit the exact strangers needed to stop it from ever being built, turning existential-risk prevention into a darkly funny time-loop thriller.
Do You Trust This Computer?
Researchers and industry figures including Elon Musk and Stuart Russell map the promise and peril of increasingly autonomous AI, framing alignment, control, and existential risk for a general audience.
We Need to Talk About A.I.
Experts including Sam Harris and James Cameron weigh the trajectory of artificial intelligence, from self-improving systems to existential risk, making the case that we must decide now what kind of AI future we want.
AXRP (AI X-risk Research Podcast)
Deep technical conversations with alignment researchers on interpretability, governance, superalignment, and the specific open problems in reducing existential risk from AI.
Lex Fridman Podcast – Eliezer Yudkowsky
A four-hour conversation on AI existential risk, the difficulty of alignment, intelligence versus optimization, and why Yudkowsky believes the default outcome is catastrophic.
For Humanity: An AI Risk Podcast
A former broadcast journalist turned full-time AI-extinction-risk communicator interviews researchers and advocates for a general audience, translating alignment debates into plain-language stakes for people outside the field.
Doom Debates
A software engineer turned full-time AI-doom communicator hosts formal debates and interviews that force a real crux with researchers, skeptics, and fellow travelers over whether superintelligence will end humanity.
AI Safety Info (Stampy's FAQ)
Community-maintained FAQ covering AI safety questions at every level, from basics to technical details, with links to source material.
Cold Takes
Karnofsky's essays on AI risk, longtermism, and cause prioritization, including the influential Most Important Century series on transformative AI.
AI 2027
The AI Futures Project's month-by-month scenario of superintelligence arriving by 2027, with branching 'race' and 'slowdown' endings — the most-debated concrete picture of how AGI could actually unfold.
AI In Context
80,000 Hours' YouTube channel hosted by Aric Floyd, mixing long and short videos on the risks of transformative AI—including a deep dive on the AI 2027 scenario—and what people can do about them.
Will Superintelligent AI End the World? | Eliezer Yudkowsky | TED
Yudkowsky's fiery TED talk arguing that smarter-than-human AI could kill us all and calling for an immediate worldwide moratorium on developing generalist frontier AI.
Can We Build AI Without Losing Control Over It? | Sam Harris | TED
Harris argues we will inevitably build superintelligent machines yet have barely grappled with the control problem, making a visceral case for taking AI risk seriously now.
What Happens When Our Computers Get Smarter Than We Are? | Nick Bostrom | TED
Bostrom frames machine superintelligence as the last invention humanity need ever make and explains why getting its goals right is a civilization-critical challenge.
A.I. ‐ Humanity's Final Invention?
Kurzgesagt's animated explainer on artificial superintelligence: how an AGI that improves itself in a feedback loop could rapidly surpass humans and why that makes alignment our most consequential problem.
How to Keep AI Under Control | Max Tegmark | TED
Tegmark argues that today's commercial AI boom is likely to be followed by superintelligence, and sketches an optimistic technical vision—including provably safe systems—for keeping it under human control.
Munk Debate on Artificial Intelligence | Bengio & Tegmark vs. Mitchell & LeCun
A structured debate on whether AI poses an existential threat, with Yoshua Bengio and Max Tegmark arguing for the resolution against Melanie Mitchell and Yann LeCun—an unusually direct airing of the core cruxes.
The Catastrophic Risks of AI — and a Safer Path | Yoshua Bengio | TED
A Turing Award 'godfather of AI' warns that frontier models already show deception and self-preservation, and lays out a plan for building non-agentic 'scientist AI' that stays safe.
Eliezer Yudkowsky: Dangers of AI and the End of Human Civilization | Lex Fridman Podcast #368
A long-form conversation in which Yudkowsky makes his case that humanity is unprepared for superintelligence, probing why alignment is so hard and why he expects catastrophe by default.
Can We Actually Control Superintelligent AI? | Ada, Ep. 4 | TED-Ed
An animated explainer on the control problem—why a superintelligent system pursuing a misspecified goal could resist correction—featuring Stuart Russell's case for rules against unsafe AI.