Science fiction imagined the machine as a weapon that fights, but an older fear is the counterfeit: the forged voice, the face that belongs to someone else. Orwell's Ministry of Truth needed no robots, only the capacity to manufacture a record that people would accept as real (37, Orwell, 1949). Generative AI has made part of that capacity cheap: a few minutes of recorded voice and a few images of a face now suffice to produce speech and video that a colleague or relative may accept as genuine.
This chapter tests two tropes from Chapter 2. T6 The weapon in human hands asks whether people use AI to cause harm at a scale or speed not otherwise possible; here the harm is fraud. T10 The end of truth asks whether synthetic content erodes the shared basis of facts; here the question is whether influence operations armed with generative models can move elections. Chapters 8 and 10 examined persuasion and the information ecosystem; this chapter looks at the hostile, organised uses. It builds on a conclusion of AI Platform and Security, recorded as F19 in SERIES_FOUNDATIONS.md: among the four commitments against AI-enabled adversaries are identity and approvals that resist synthetic media (179, Nordqvist, 2026).
Table 20.1: The control that breaks each step.
Primary topics explored
- The video call that moved 25 million
- Fraud at scale
- Influence operations and elections
- Labelling and platform duties
- Approvals that do not trust appearance
- The debate
About AI Threat
AI Threat is Book 5 of The Transformed Architecture. It investigates whether the dystopias of science fiction, with AI as the threat, are coming true. It covers every scale of the threat, from the human mind and society to misuse, machines beyond control and the end of the world, stages every debate constructively, and turns the verdict into what individuals, European enterprises and public authorities can do.
It builds on the conclusions of Architecture as Code, AI Governance, AI Platform and Security and AI Red Team.
Boards, chief risk officers, CISOs, AI governance leads, architects and public-sector leaders.
Gunnar Nordqvist
Conceived, directed and published the books of The Transformed Architecture.
Continue with the full chapter
Chapter 20: Fraud, Impersonation and Influence Operations in AI Threat gives the detailed explanations, practical examples and implementation patterns behind this summary.
This whitepaper belongs to “Humans Wielding Machines” and is one of 39 drawn from the manuscript.