This chapter opens Part IV, on security. It extends security architecture to the parts of an AI system that classical controls do not reach: the model, the prompts and retrieved content that steer it, the data that trained it, and the agents that act on its output. It maps the attack surface with OWASP and MITRE ATLAS, examines injection, poisoning and leakage, applies supply-chain practice to models, and closes with a codified threat model and guardrail policy. The security patterns of Architecture as Code (Chapters 9 and 9B) remain the foundation (72, Nordqvist, 2025). Chapters 16 and 17 treat AI as the target of attack; Chapters 18 and 19 treat AI as the source of threat.
Three properties of AI systems break long-standing assumptions. First, the instruction channel and the data channel are the same: a large language model (LLM) receives its system prompt, the user's request and any retrieved document as one sequence of tokens, and nothing in the model enforces a boundary between them. Second, the weights are an asset that can be stolen, inverted or manipulated, and they encode fragments of their training data. Third, agents hold credentials and call tools, so a manipulated output becomes an action with the agent's privileges.
The European Union Agency for Cybersecurity (ENISA) frames the response in layers: cybersecurity foundations for every ICT system, AI-specific practices for the model lifecycle, and a sector-specific layer (19, ENISA, 2023). The most common failure we see is treating AI security as a new discipline and neglecting the first layer; zero trust, identity, secrets management, segmentation and logging come first, and the AI-specific controls assume them.
Primary topics explored
- What changes when the system contains a model
- Two catalogues: OWASP Top 10 for LLM applications and MITRE ATLAS
- Prompt injection, direct and indirect
- Handling of untrusted content (section 4 of casework-assistant prompt v1.4.0)
- Poisoning, extraction, and training-data leakage
- Supply-chain security for models
About AI Platform and Security
AI Platform and Security is Book 3 of The Transformed Architecture. It shows how large European enterprises and public authorities build, secure and operate the platform on which artificial intelligence runs: data, models and DevSecMLOps, retrieval and agents, infrastructure and sovereignty, AI security engineering, evaluation, observability and cost.
The companion volume AI Governance covers risk management, governance, assurance, organisation and leadership; Architecture as Code supplies the underlying discipline.
Platform engineers, security architects, MLOps teams, AI architects and technical decision-makers.
Gunnar Nordqvist
Conceived, directed and published the books of The Transformed Architecture.
Continue with the full chapter
Chapter 16: Security for AI Systems in AI Platform and Security gives the detailed explanations, practical examples and implementation patterns behind this summary.
This whitepaper belongs to “Security” and is one of 29 drawn from the manuscript.