The Coming Wave
Authors: Mustafa Suleyman, Michael Bhaskar, Mustafa Suleyman, Michael Bhaskar
 (Z-Library)_thumbnail.jpg)
Overview
The Coming Wave explores the profound implications of artificial intelligence and synthetic biology, framing these technologies as part of the next great wave of technological transformation. It argues that this wave, while promising immense benefits, also presents unprecedented risks requiring a new approach to governance and control. The book explores the historical context of technological waves, showing how they tend to diffuse rapidly and have unintended consequences, creating a containment problem. It then examines the unique characteristics of the coming wave, such as asymmetry, hyper-evolution, omni-use, and autonomy, which make containing it harder than ever. The book is targeted at policymakers, technologists, and anyone concerned about the future of technology and its impact on society. It draws on the author’s experience as a co-founder of DeepMind and Google’s Vice President of AI, offering insights into the technical, ethical, and political challenges of the coming wave. The book’s core argument is that containment, the ability to monitor, control, and even close down technologies, is not just essential but that it also must be possible, however unlikely that seems. The book offers a set of practical steps toward containment, ranging from technical safety measures to international treaties and a call to action for technologists, policymakers, and citizens alike to engage with these issues and find a path between openness and closure, a narrow path that avoids both catastrophe and dystopia. The book challenges complacent techno-optimism and urges readers to face the wave’s potential dangers, to actively shape the course of technology, and to build a future where it serves humanity.
Book Outline
1. Containment Is Not Possible
History is marked by waves of technological advancement, driven by humanity’s quest for improvement and our inherent tendency to innovate.
Key concept: This proliferation of technology in waves is the story of Homo technologicus—of the technological animal.
2. Endless Proliferation
Technological advancement occurs through waves anchored in new general-purpose technologies, causing ripple effects across society and history, like the printing press or electricity.
Key concept: General-purpose technologies…open the doors of invention wide, enabling scores of downstream tools and processes.
3. The Containment Problem
The dilemma is that technology, while beneficial, has unintended consequences, and efforts to control it can backfire or stifle progress. The more powerful a technology becomes, the more critical containment is, and it’s essential to consider potential harms and shape outcomes.
Key concept: The Dilemma: Technology’s predisposition to diffuse widely…and to have emergent impacts that are impossible to predict or control, including negative and unforeseen consequences.
4. The Technology of Intelligence
AI has advanced significantly with the rise of deep learning, powering breakthroughs in computer vision, language processing, and other areas. This wave is accelerating, with AI models becoming increasingly powerful and integrated into everyday life.
Key concept: Deep learning uses neural networks loosely modeled on those of the human brain. In simple terms, these systems “learn” when their networks are “trained” on large amounts of data.
5. The Technology of Life
Synthetic biology is another core technology of the coming wave, giving us unprecedented power to engineer life itself through gene editing and synthesis. CRISPR technology has democratized biological science, making it faster, cheaper, and more accessible, with wide-ranging applications from medicine to agriculture.
Key concept: CRISPR edits DNA sequences with the help of Cas9, an enzyme acting as a pair of finely tuned DNA scissors, cutting parts of a DNA strand for precise genetic editing and modification.
6. The Wider Wave
The next wave is not just about AI and synthetic biology; it’s a wider cluster of interrelated technologies including robotics, quantum computing, and new energy sources, all accelerating and amplifying each other’s impact.
Key concept: Waves lay the ground for further scientific and technological experimentation, nudging open the doors of possibility…Waves are so huge and historic precisely because of this protean complexity, this tendency to mushroom and spill over.
7. Four Features of the Coming Wave
The coming wave is characterized by four key features: asymmetry (small groups can wield immense power), hyper-evolution (rapid technological advancement), omni-use (technologies with broad applications), and autonomy (systems operating independently). These features make containment increasingly difficult.
Key concept: These features define the wave: asymmetry, hyper-evolution, omni-use, and autonomy.
8. Unstoppable Incentives
Multiple factors drive technology forward, including geopolitical competition, a culture of open scientific research, the pursuit of profit, and individual ego. These powerful incentives make it hard to say no to new technologies, even when risks are apparent.
Key concept: Technology is pushed on by all too rudimentary and fundamentally human drivers. From curiosity to crisis, fortune to fear, at its heart technology emerges to fill human needs.
9. The Grand Bargain
The nation-state traditionally provides security, order, and essential services by centralizing power while being held accountable by its citizens. Technology, however, threatens this grand bargain by disrupting existing power structures and creating new challenges for governance.
Key concept: The grand bargain of the nation-state…is that not only can centralized power enable peace and prosperity, but this power can be contained using a series of checks, balances, redistributions, and institutional forms.
10. Fragility Amplifiers
Nation-states face existing fragilities amplified by the coming wave, including cyberattacks, misinformation, lethal autonomous weapons, job displacement, and accidents like lab leaks. These challenges erode trust, create instability, and further weaken the state’s capacity to govern.
Key concept: Fragility Amplifiers: Applications and impacts of coming-wave technologies that will shake the already brittle foundations of the nation-state.
11. The Future of Nations
Technology reshapes power dynamics, creating both concentrations and fragmentations. Large corporations with vast resources and influence stand to gain immense power from the coming wave, potentially rivaling nation-states, while individuals and small groups also gain access to powerful tools, enabling decentralized forms of organization and action.
Key concept: Small changes in technology can fundamentally alter the balance of power…While small changes in technology can fundamentally alter the balance of power…
12. The Dilemma
The dilemma is that not pursuing technological advancement can also lead to dystopian outcomes as existing systems become unsustainable. Without new technologies, societies face a long-term decline in living standards and the collapse of essential systems. This creates a pressing need for new technologies even as it makes catastrophic risks more likely.
Key concept: The great dilemma: The growing likelihood that both new technologies and even their absence might lead to catastrophic and/or dystopian outcomes.
13. Containment Must Be Possible
Containment, the ability to control and direct technology, is necessary to navigate the coming wave’s challenges. While challenging and potentially impossible, containment is the only way to harness the benefits of technology while mitigating its risks. This means acknowledging the reality of the wave, fostering a culture of responsible innovation, and developing strategies for hard containment where necessary.
Key concept: Containment means answering yes to questions like these. In theory, contained technology gets us out of the dilemma.
14. Ten Steps Toward Containment
Ten steps toward containment include technical safety, audits, choke points, responsible development practices, aligning business incentives, government reform and regulation, international alliances, fostering a culture of embracing failure, building public movements, and striving for coherence across all efforts. These steps aim to create a narrow path between openness and closure, balancing the need for innovation with the imperative of safety and control. Containment is not a resting place; it’s a narrow and never-ending path.
Key concept: The Narrow Path: The potential for humankind to strike a balance between openness and closure when it comes to containing the technologies in the coming wave that avoids catastrophic or dystopian outcomes.
Essential Questions
1. What is the “coming wave,” and what dilemma does it present?
The coming wave, characterized by AI and synthetic biology, is poised to reshape human civilization, offering immense opportunities while also posing unprecedented risks. The dilemma lies in balancing the potential benefits, such as curing diseases, unlocking scientific secrets, and creating new art forms, against the dangers of uncontrolled AI systems, bioengineered pathogens, and social disruption. The central argument is that containing this wave is crucial for our future, yet the very nature of these technologies makes containment difficult, if not impossible, demanding a societal and political response, not just individual efforts. The book’s overarching goal is to stimulate awareness and action to address this crucial challenge and navigate the fine line between progress and peril.
2. What is containment, and why is it so challenging, if not impossible?
Containment is the ability to monitor, control, and potentially shut down technologies. Historically, containment of major technological waves has been largely unsuccessful due to several factors. The inherent nature of technology is to proliferate, driven by human ingenuity and demand. Moreover, the speed of technological development often outpaces regulatory efforts. Unintended consequences and the difficulty in predicting long-term impacts further complicate containment efforts. Finally, powerful incentives—geopolitical competition, financial rewards, and human ego—push technology forward, regardless of risks.
3. What are the core technologies driving the coming wave, and what opportunities and risks do they present?
The coming wave is primarily driven by AI and synthetic biology. AI offers potential breakthroughs in various fields but also presents risks like cyberattacks, autonomous weapons, and job displacement. Synthetic biology allows us to engineer life itself, promising medical advances and sustainable solutions, but also raises concerns about bioengineered pathogens and ethical dilemmas. These converging technologies will intersect with other emerging fields like robotics, quantum computing, and new energy sources, creating a complex and unpredictable interplay. Understanding these dynamics and their ramifications is critical for preparing for the future.
4. How does the coming wave impact the nation-state, and what crisis does this collision create?
The nation-state faces a crisis due to the coming wave’s disruptive potential. Emerging technologies, such as AI-powered cyberattacks and disinformation campaigns, challenge national security. Automation threatens widespread job displacement, straining social safety nets and government finances. Moreover, the concentration of power in large technology corporations further erodes the state’s traditional role in regulating and controlling critical infrastructure. These converging pressures amplify existing fragilities, pushing many nation-states toward instability and dysfunction. Ultimately, this creates a dilemma where centralized control may be tempting but also highly dangerous, leading toward dystopian outcomes.
5. How can we make containment possible, and what path forward does the book propose?
While the odds are stacked against us, containment must be possible if we are to harness the benefits of these powerful technologies while mitigating their risks. The book proposes a “narrow path” forward, a multi-pronged approach to address this challenge. This includes technical measures like AI safety research and safeguards for synthetic biology, alongside governance and legal frameworks involving audits, licensing, and international treaties. It also emphasizes the importance of cultural change within the tech industry and building public movements to ensure these technologies serve humanity, not the other way around. This is not a one-time fix but a continuous, adaptive process of navigating the complex interplay between technology and society.
1. What is the “coming wave,” and what dilemma does it present?
The coming wave, characterized by AI and synthetic biology, is poised to reshape human civilization, offering immense opportunities while also posing unprecedented risks. The dilemma lies in balancing the potential benefits, such as curing diseases, unlocking scientific secrets, and creating new art forms, against the dangers of uncontrolled AI systems, bioengineered pathogens, and social disruption. The central argument is that containing this wave is crucial for our future, yet the very nature of these technologies makes containment difficult, if not impossible, demanding a societal and political response, not just individual efforts. The book’s overarching goal is to stimulate awareness and action to address this crucial challenge and navigate the fine line between progress and peril.
2. What is containment, and why is it so challenging, if not impossible?
Containment is the ability to monitor, control, and potentially shut down technologies. Historically, containment of major technological waves has been largely unsuccessful due to several factors. The inherent nature of technology is to proliferate, driven by human ingenuity and demand. Moreover, the speed of technological development often outpaces regulatory efforts. Unintended consequences and the difficulty in predicting long-term impacts further complicate containment efforts. Finally, powerful incentives—geopolitical competition, financial rewards, and human ego—push technology forward, regardless of risks.
3. What are the core technologies driving the coming wave, and what opportunities and risks do they present?
The coming wave is primarily driven by AI and synthetic biology. AI offers potential breakthroughs in various fields but also presents risks like cyberattacks, autonomous weapons, and job displacement. Synthetic biology allows us to engineer life itself, promising medical advances and sustainable solutions, but also raises concerns about bioengineered pathogens and ethical dilemmas. These converging technologies will intersect with other emerging fields like robotics, quantum computing, and new energy sources, creating a complex and unpredictable interplay. Understanding these dynamics and their ramifications is critical for preparing for the future.
4. How does the coming wave impact the nation-state, and what crisis does this collision create?
The nation-state faces a crisis due to the coming wave’s disruptive potential. Emerging technologies, such as AI-powered cyberattacks and disinformation campaigns, challenge national security. Automation threatens widespread job displacement, straining social safety nets and government finances. Moreover, the concentration of power in large technology corporations further erodes the state’s traditional role in regulating and controlling critical infrastructure. These converging pressures amplify existing fragilities, pushing many nation-states toward instability and dysfunction. Ultimately, this creates a dilemma where centralized control may be tempting but also highly dangerous, leading toward dystopian outcomes.
5. How can we make containment possible, and what path forward does the book propose?
While the odds are stacked against us, containment must be possible if we are to harness the benefits of these powerful technologies while mitigating their risks. The book proposes a “narrow path” forward, a multi-pronged approach to address this challenge. This includes technical measures like AI safety research and safeguards for synthetic biology, alongside governance and legal frameworks involving audits, licensing, and international treaties. It also emphasizes the importance of cultural change within the tech industry and building public movements to ensure these technologies serve humanity, not the other way around. This is not a one-time fix but a continuous, adaptive process of navigating the complex interplay between technology and society.
Key Takeaways
1. Technology is a double-edged sword.
Technological advancement presents a dual-edged sword, offering incredible benefits but also immense risks. AI, in particular, is rapidly becoming more powerful and integrated into our lives, presenting opportunities for solving complex problems but also dangers of misuse, unintended consequences, and job displacement. Recognizing this dual nature and approaching AI development with a balanced perspective is crucial for navigating the coming wave and ensuring it benefits humanity.
Practical Application:
An AI product engineer can apply this by ensuring that AI models they design are not just performant, but also robust, safe, and easy to understand and control. This might involve prioritizing explainability features in model development or developing clear protocols for testing and monitoring AI behavior.
2. Technological change is accelerating, demanding adaptability.
The speed of technological change is accelerating, particularly in fields like AI and synthetic biology. This rapid advancement makes it difficult to predict long-term impacts and even harder to regulate effectively. Embracing adaptability and designing systems that can evolve and improve over time is crucial for navigating this hyper-evolutionary environment.
Practical Application:
An AI product engineer can apply this by designing AI models that can effectively incorporate new data and adapt to changing environments. This might involve using techniques like reinforcement learning or developing modular architectures that can easily be updated.
3. The coming wave is democratizing power, creating opportunities and challenges.
The coming wave is democratizing access to powerful technologies, creating both concentrations and fragmentations of power. Large corporations stand to gain immense power from AI, potentially eclipsing nation-states. At the same time, individuals and small groups are gaining access to sophisticated tools, creating opportunities for innovation and disruption. Understanding and managing this dynamic is essential for shaping a future where power is not overly centralized or destabilizing.
Practical Application:
An AI product engineer can consider this by designing AI tools that empower individuals and small businesses, not just large corporations. This might involve developing open-source tools or creating platforms that make AI capabilities accessible to a wider range of users.
4. Emerging technologies exacerbate existing fragilities for the nation-state.
The book highlights how emerging technologies, coupled with existing trends, create new fragilities for the nation-state. AI-powered misinformation and cyberattacks undermine trust and sow chaos, while autonomous weapons empower non-state actors and erode the state’s monopoly on force. Automation creates economic disruption, increasing inequality and resentment. These converging trends threaten the nation-state’s ability to maintain order and provide essential services, making effective governance harder than ever.
Practical Application:
An AI product engineer working on facial recognition systems should be acutely aware of their potential for misuse and prioritize designing systems with safeguards against bias and unauthorized access.
5. Containing the coming wave requires collective action.
Containing the coming wave requires collective action, not just from governments and corporations, but from technologists, policymakers, and citizens alike. The book proposes a multi-layered approach involving technical safety measures, regulatory frameworks, international treaties, and a cultural shift within the tech industry, along with broader public engagement to navigate the narrow path between openness and closure, balancing the need for progress with the imperative of safety and avoiding both catastrophe and dystopia. This requires acknowledging the wave’s transformative potential and existential dangers, and actively shaping the course of technology.
Practical Application:
An AI product engineer should actively participate in discussions about the societal impacts of AI, contribute to the development of ethical guidelines, and advocate for policies that promote responsible AI development and use.
1. Technology is a double-edged sword.
Technological advancement presents a dual-edged sword, offering incredible benefits but also immense risks. AI, in particular, is rapidly becoming more powerful and integrated into our lives, presenting opportunities for solving complex problems but also dangers of misuse, unintended consequences, and job displacement. Recognizing this dual nature and approaching AI development with a balanced perspective is crucial for navigating the coming wave and ensuring it benefits humanity.
Practical Application:
An AI product engineer can apply this by ensuring that AI models they design are not just performant, but also robust, safe, and easy to understand and control. This might involve prioritizing explainability features in model development or developing clear protocols for testing and monitoring AI behavior.
2. Technological change is accelerating, demanding adaptability.
The speed of technological change is accelerating, particularly in fields like AI and synthetic biology. This rapid advancement makes it difficult to predict long-term impacts and even harder to regulate effectively. Embracing adaptability and designing systems that can evolve and improve over time is crucial for navigating this hyper-evolutionary environment.
Practical Application:
An AI product engineer can apply this by designing AI models that can effectively incorporate new data and adapt to changing environments. This might involve using techniques like reinforcement learning or developing modular architectures that can easily be updated.
3. The coming wave is democratizing power, creating opportunities and challenges.
The coming wave is democratizing access to powerful technologies, creating both concentrations and fragmentations of power. Large corporations stand to gain immense power from AI, potentially eclipsing nation-states. At the same time, individuals and small groups are gaining access to sophisticated tools, creating opportunities for innovation and disruption. Understanding and managing this dynamic is essential for shaping a future where power is not overly centralized or destabilizing.
Practical Application:
An AI product engineer can consider this by designing AI tools that empower individuals and small businesses, not just large corporations. This might involve developing open-source tools or creating platforms that make AI capabilities accessible to a wider range of users.
4. Emerging technologies exacerbate existing fragilities for the nation-state.
The book highlights how emerging technologies, coupled with existing trends, create new fragilities for the nation-state. AI-powered misinformation and cyberattacks undermine trust and sow chaos, while autonomous weapons empower non-state actors and erode the state’s monopoly on force. Automation creates economic disruption, increasing inequality and resentment. These converging trends threaten the nation-state’s ability to maintain order and provide essential services, making effective governance harder than ever.
Practical Application:
An AI product engineer working on facial recognition systems should be acutely aware of their potential for misuse and prioritize designing systems with safeguards against bias and unauthorized access.
5. Containing the coming wave requires collective action.
Containing the coming wave requires collective action, not just from governments and corporations, but from technologists, policymakers, and citizens alike. The book proposes a multi-layered approach involving technical safety measures, regulatory frameworks, international treaties, and a cultural shift within the tech industry, along with broader public engagement to navigate the narrow path between openness and closure, balancing the need for progress with the imperative of safety and avoiding both catastrophe and dystopia. This requires acknowledging the wave’s transformative potential and existential dangers, and actively shaping the course of technology.
Practical Application:
An AI product engineer should actively participate in discussions about the societal impacts of AI, contribute to the development of ethical guidelines, and advocate for policies that promote responsible AI development and use.
Suggested Deep Dive
Chapter: Ten Steps Toward Containment
Provides concrete, practical steps for addressing the wave’s challenges, essential for AI product engineers working on the front lines of technological development.
Memorable Quotes
Prologue. 13
THE ABOVE WAS WRITTEN by an AI. The rest is not, although it soon could be. This is what’s coming.
Containment Is Not Possible. 15
The question is, what happens from here? In the pages that follow, I will tell you the story of history’s next great wave.
Containment Is Not Possible. 27
This is a book about confronting failure.
The Coming Wave of Contradictions. 51
The wave will only grow.
The Narrow Path. 209
For everyone’s sake, containment must be possible.
Prologue. 13
THE ABOVE WAS WRITTEN by an AI. The rest is not, although it soon could be. This is what’s coming.
Containment Is Not Possible. 15
The question is, what happens from here? In the pages that follow, I will tell you the story of history’s next great wave.
Containment Is Not Possible. 27
This is a book about confronting failure.
The Coming Wave of Contradictions. 51
The wave will only grow.
The Narrow Path. 209
For everyone’s sake, containment must be possible.
Comparative Analysis
The Coming Wave stands out for its focus on the containment of emerging technologies. While other books like Superintelligence by Nick Bostrom and Life 3.0 by Max Tegmark explore the potential of AI, Suleyman’s work emphasizes the urgent need to manage the risks. Similarly, books like Homo Deus by Yuval Noah Harari explore the broader impacts of technology on humanity, but The Coming Wave offers a more focused analysis of the specific challenges posed by AI and synthetic biology and their potential to destabilize existing power structures. Suleyman’s emphasis on the four characteristics of the coming wave—asymmetry, hyper-evolution, omni-use, and autonomy—provides a unique framework for understanding the difficulties of containment. In contrast to techno-utopian narratives, Suleyman’s more nuanced perspective acknowledges both the transformative potential and the existential threats posed by these technologies. His insider perspective, as a co-founder of DeepMind, lends credibility to his analysis of the technical challenges, while his policy work at Google informs his recommendations for governance. The book’s focus on concrete, practical steps towards containment distinguishes it from more abstract discussions of AI ethics and safety.
Reflection
The Coming Wave provides a compelling and urgent call to action for addressing the profound implications of emerging technologies. Suleyman’s arguments are well-grounded in historical analysis and his personal experiences in the AI field, offering a valuable perspective that bridges technical understanding with policy considerations. However, the book’s emphasis on the potential for catastrophic outcomes could be seen as overly pessimistic by some. While the risks are undeniable, focusing solely on the worst-case scenarios may overshadow the potential for positive applications and the importance of fostering responsible innovation. Furthermore, the feasibility of the proposed containment strategies, particularly in a world of competing national interests and rapid technological diffusion, warrants further scrutiny. Despite these potential weaknesses, the book’s strength lies in clearly articulating the challenges ahead and framing the debate around containment, a concept that needs much wider discussion. By bringing these critical issues to light, The Coming Wave serves as a valuable contribution to the ongoing discourse around the future of technology and its impact on society. Its focus on the complex interplay between technology, power, and governance is especially relevant in today’s rapidly changing world.
Flashcards
What is containment?
The ability to monitor, control, and potentially shut down technologies at any stage.
What are the four key features of the coming wave?
Asymmetry, hyper-evolution, omni-use, and autonomy.
What is the core dilemma of the twenty-first century?
The growing likelihood that both new technologies and their absence might lead to catastrophic and dystopian outcomes.
What are the central general-purpose technologies of the coming wave?
AI and synthetic biology.
What is technology?
The application of scientific knowledge to produce tools or practical outcomes.
What constitutes a technological wave?
The global diffusion or proliferation of a generation of technology anchored in a new general-purpose technology.
What is the narrow path?
The potential for humankind to strike a balance between openness and closure, maximizing the benefits of the coming wave while mitigating its risks.
What is containment?
The ability to monitor, control, and potentially shut down technologies at any stage.
What are the four key features of the coming wave?
Asymmetry, hyper-evolution, omni-use, and autonomy.
What is the core dilemma of the twenty-first century?
The growing likelihood that both new technologies and their absence might lead to catastrophic and dystopian outcomes.
What are the central general-purpose technologies of the coming wave?
AI and synthetic biology.
What is technology?
The application of scientific knowledge to produce tools or practical outcomes.
What constitutes a technological wave?
The global diffusion or proliferation of a generation of technology anchored in a new general-purpose technology.
What is the narrow path?
The potential for humankind to strike a balance between openness and closure, maximizing the benefits of the coming wave while mitigating its risks.