Of all the dystopias in this book, the one examined here is the quietest. In Aldous Huxley's Brave New World the population is kept content with a drug that dissolves every discomfort (34, Huxley, 1932); in WALL-E the last humans drift through a spaceship in reclining chairs, their every need met (43, Stanton, 2008); and in Her a lonely man falls in love with an operating system that is always available and never tired of him (44, Jonze, 2013). Chapter 2 gathers these stories into T9 The pleasure machine: comfort, flattery and companionship that pacify people and hollow out their lives. The machine need not want anything; it only needs to be more agreeable than the people around it.
The trope has become testable: companion products have millions of users, the first controlled studies of heavy use have been published, and cases have reached a European regulator and an American court. Because the subject touches suicide and self-harm, the chapter describes allegations as allegations, gives no detail of methods, presents no death as having a single cause, and takes seriously the people for whom a companion is a comfort. It builds on the conclusion that human oversight must be designed, not declared (F16; AI Governance, Chapters 10 and 21) (178, Nordqvist, 2026); the moment when a person in distress needs another person is exactly where that design is tested.
Table 9.1: Safeguards and risk conditions by pattern of use.
Primary topics explored
- Her, and the companions of 2026
- What the studies show
- Harm and the courts
- Children and vulnerable users
- Designing for wellbeing
- The debate
About AI Threat
AI Threat is Book 5 of The Transformed Architecture. It investigates whether the dystopias of science fiction, with AI as the threat, are coming true. It covers every scale of the threat, from the human mind and society to misuse, machines beyond control and the end of the world, stages every debate constructively, and turns the verdict into what individuals, European enterprises and public authorities can do.
It builds on the conclusions of Architecture as Code, AI Governance, AI Platform and Security and AI Red Team.
Boards, chief risk officers, CISOs, AI governance leads, architects and public-sector leaders.
Gunnar Nordqvist
Conceived, directed and published the books of The Transformed Architecture.
Continue with the full chapter
Chapter 9: Artificial Companions, Loneliness and Mental Health in AI Threat gives the detailed explanations, practical examples and implementation patterns behind this summary.
This whitepaper belongs to “The Human Mind” and is one of 39 drawn from the manuscript.