No image of AI risk is better known than Skynet, the defence network in The Terminator that is trusted to run the military's systems and turns them on humanity (42, Cameron, 1984). For the questions states actually negotiate, the image is misleading. The debate about military AI is not about a network that rebels. It is about weapons that select and engage targets without a human decision for each engagement, about who answers in law when such a weapon kills the wrong people, and about a competition that may push everyone to field systems before anyone understands them.
This chapter tests three tropes from Chapter 2. T6 The weapon in human hands is its main subject. T1 The creature that turns on its maker is the trope the public reaches for, and the chapter separates it from T6. T5 The intelligence explosion enters through the race: the argument that whoever first builds a self-improving system gains a decisive military advantage. The chapter stays at the level of evidence, law and policy and describes no weapon design or targeting method. Its practical concern is open question Q12, new in this book: what a single organisation can do about risks that are systemic, international and decided by a few actors.
Table 21.1: The spectrum and the points of meaningful human control.
Primary topics explored
- Skynet and the real debate
- What has been reported
- Meaningful human control
- The race
- The civilian side of a military technology
- The debate
About AI Threat
AI Threat is Book 5 of The Transformed Architecture. It investigates whether the dystopias of science fiction, with AI as the threat, are coming true. It covers every scale of the threat, from the human mind and society to misuse, machines beyond control and the end of the world, stages every debate constructively, and turns the verdict into what individuals, European enterprises and public authorities can do.
It builds on the conclusions of Architecture as Code, AI Governance, AI Platform and Security and AI Red Team.
Boards, chief risk officers, CISOs, AI governance leads, architects and public-sector leaders.
Gunnar Nordqvist
Conceived, directed and published the books of The Transformed Architecture.
Continue with the full chapter
Chapter 21: Autonomous Weapons and the AI Arms Race in AI Threat gives the detailed explanations, practical examples and implementation patterns behind this summary.
This whitepaper belongs to “Humans Wielding Machines” and is one of 39 drawn from the manuscript.