AI Threat · Whitepaper

Chapter 11 · Society and Truth

The Automated State

When Public Authorities Got It Wrong

Author Conceived and directed by Gunnar Nordqvist; text written by AI agents under his direction.
Published 2 November 2026
Build number 2026.11
ai-risk loss-of-control ai-misuse threat-register
Figure 11.1: The harm path runs from data through a model or rule, a flag, an official action and a burden on the citizen to redress or its absence; complaints, appeals and inquiries feed back to the model only if someone listens.
Figure 11.1: The harm path runs from data through a model or rule, a flag, an official action and a burden on the citizen to redress or its absence; complaints, appeals and inquiries feed back to the model only if someone listens.

This chapter is about a dystopia that has already happened. In the Netherlands and in Australia, public authorities used automated risk selection and calculation to decide whom to suspect and pursue for money. Large numbers of people on low incomes were treated as cheats on the strength of a flag, a score or an average, and for years the institutions meant to serve them would not listen. None of these systems was advanced AI.

The chapter tests T7 The watching state and T8 The machine that stops, and the humans who forgot, here in its institutional form of officials who stopped checking because the system had already decided. It answers the series' open question Q11 for one class of harm and turns the lessons into a review any authority can run. It builds on F30 from AI Governance, Chapter 17: in public authorities legality comes before capability, the line between decision support and automated decision is drawn in the data model and the audit log, and every case has a reasons log (178, Nordqvist, 2026). That chapter described the controls; this one shows the failures they answer.

Table 11.1: The control that blocks each step.

Primary topics explored

  1. SyRI: a court stops a risk model
  2. Unprecedented injustice: the Dutch childcare benefits scandal
  3. Robodebt: income averaging and the Royal Commission
  4. The common mechanisms
  5. What the AI Act now requires
  6. The quiet dystopia as a design failure

About AI Threat

AI Threat is Book 5 of The Transformed Architecture. It investigates whether the dystopias of science fiction, with AI as the threat, are coming true. It covers every scale of the threat, from the human mind and society to misuse, machines beyond control and the end of the world, stages every debate constructively, and turns the verdict into what individuals, European enterprises and public authorities can do.

It builds on the conclusions of Architecture as Code, AI Governance, AI Platform and Security and AI Red Team.

Who should read this

Boards, chief risk officers, CISOs, AI governance leads, architects and public-sector leaders.

Portrait of Gunnar Nordqvist

Gunnar Nordqvist

Conceived, directed and published the books of The Transformed Architecture.

Continue with the full chapter

Read the full chapter online

Chapter 11: The Automated State: When Public Authorities Got It Wrong in AI Threat gives the detailed explanations, practical examples and implementation patterns behind this summary.

This whitepaper belongs to “Society and Truth” and is one of 39 drawn from the manuscript.