Case Studies and Anti-Patterns

The earlier chapters argued from principles, studies and the two fictional organisations. This chapter checks the argument against public cases, in which a regulator, a court, a vendor or a maintainer has written down what happened. It concerns all five developer functions: D1 Frame, because the cheapest failure to prevent is software nobody specified; D2 Design, because several records turn on a pipeline or access role designed to trust its own inputs; D3 Build, because of what a developer accepts from a tool unchecked; D4 Prove, because the recurring fault is a check that shared the blind spots of the thing it checked; and D5 Steward, because every case ends with the questions of who owned the software, who could explain it and who could revert it.

The method is strict. Every case is described only from a primary record that was opened and read in preparing this chapter: a regulator's order, a High Court judgment, a vendor's bulletin, a maintainers' advisory, a researcher's account of an experiment and an engineers' experience report. A party's statement of its own results is called a claim. A case that could not be verified is not analysed, and the section on method lists them. Interpretation is the book's own and is labelled as such. The cases are observable in the sense of AI Governance, Chapter 22 (162, Nordqvist, 2026).

The chapter takes five conclusions from the earlier volumes as lenses. Incidents fail through correlated controls: a shared credential, a mirror that was also an egress path, an alert that reached people who did not hold the stop decision (AI Governance, Chapter 19, F28) (162, Nordqvist, 2026). Agents acting beyond their mandate are observable, and the lasting response has been to move controls from the agent's compliance into the environment (AI Threat, Chapters 25 and 35, F67) (165, Nordqvist, 2026). In the first exercise on the defenders' own AI, the action control held while the case store accepted unlabelled text (AI Blue, Chapter 15, F114) (168, Nordqvist, 2026). Loops fail in recognisable ways, each a step that looked reasonable, and the controls are few: independence, evidence age, a stop path outside the system and a testing right (AI Purple, Chapter 26, F140) (169, Nordqvist, 2026). Public cases show a system with owners for its parts and none for the whole, an explanation the user could not give, and an assurance that outran its evidence (AI Leader, Chapter 30, F183) (174, Nordqvist, 2026). This chapter asks the same of the person who owns the software, with the stewardship tests of Chapter 4: Name, Understand, Revert, and a test oracle independent of the generator.

Primary topics explored

1. What this chapter builds on
2. How the cases were chosen and what was dropped
3. Cases, by mechanism
4. What recurs
5. Anti-patterns
6. What would have prevented them