This chapter states the design rule on which the rest of the book depends. A loop in which an emulation system produces the test and a detection system is tested by it, and in which a third system reports whether the result was good, can agree with itself. The chapter concerns P3 Observe, where telemetry, detection and response are compared with the expectation and a finding is raised; P4 Improve, where a finding becomes a change in a detection, control, playbook or mandate; and P5 Verify, where a re-test passes and evidence is attached before a finding closes. It touches P1 Target, where the scenario that is held out is chosen and by whom (Chapter 5), and P2 Emulate, whose scenarios the detection side must not see in advance. Its subject is what makes the result of such a loop independent of the thing it checks, how that independence is shown to a person who was not there, and what a loop closed by machines alone can and cannot establish.
The rule, called the independence principle in Chapter 4, reads as follows. A check counts as verification only when the organisation can show that the ways in which the check can fail are not the ways in which the checked thing can fail. Independence is therefore a property of a particular claim, designed and evidenced, and never a property that a tool, a vendor or a model family can be assumed to have. Chapter 4 declares independence in three layers: five kinds of separation (model, data, authorship, vendor and time), one rule between roles (the incentive rule) and three proofs (held-out scenarios, seeded gaps and seeded errors, and reproduction by a person). The chapter develops the five kinds and the three proofs, adds a limit that comes from test-aware behaviour, a debate on whether machines can close the loop alone, and a checklist for supervisors and boards.
Consider the loop as the earlier volumes left it. An emulation planner drafts a scenario for a technique on the priority list. An emulation runner executes it inside the signed scope. An observation correlator compares the telemetry with the expectation. A detection drafter proposes a rule that closes the gap, and a critique agent reviews the rule. Each step is useful. If the same model family, the same vendor, the same retrieval corpus and the same prompts sit behind all of them, the steps are less independent than the diagram suggests, because whatever the family does not know or systematically mistakes, it mistakes everywhere in the chain.
Primary topics explored
- Correlated failure
- Kinds of independence
- Held-out scenarios
- Seeded gaps
- Test-aware behaviour
- Reproduction by a person
About AI Purple
AI Purple is Book 9 of The Transformed Architecture. It takes the loop between the red team and the blue team as its subject and asks what happens when AI sits on both sides of the table: the five stages of the loop (target, emulate, observe, improve, verify), rules of engagement and safe emulation, continuous emulation, the independence principle, coverage as a claim that needs evidence, the loop's own AI and how to defend it, operating models, decision rights, regulation, and the people involved, for European enterprises and public authorities.
It builds on the conclusions of Architecture as Code, AI Governance, AI Platform and Security, AI Red Team, AI Threat, The Transformed Security Architect, AI Boss and AI Blue.
Chief information security officers, heads of security operations, red-team leads, penetration testers, SOC managers, detection engineers, purple-team coordinators and incident commanders, security and AI architects, boards and management bodies, procurement leads, works councils and unions, data protection and audit functions, and public-sector leaders.
Gunnar Nordqvist
Conceived, directed and published the books of The Transformed Architecture.
Continue with the full chapter
Chapter 15: Independence: Verifying the Loop When AI Is on Both Sides in AI Purple gives the detailed explanations, practical examples and implementation patterns behind this summary.
This whitepaper belongs to “The Loop's Own AI” and is one of 33 drawn from the manuscript.