A Reference Architecture for AI in Large Organisations

A reference architecture is a shared answer to the question of what an organisation's AI estate should consist of, so that each new use case extends a known structure rather than inventing one. This chapter presents a layered reference architecture for AI in large enterprises and public authorities, its four central patterns, its mapping to the AI Act roles, and a declarative model of it that frames every later chapter.

A reference architecture earns its cost when the alternative is twenty teams making twenty incompatible decisions. Kreuzberger, Kühl, and Hirschl (2023) derived an MLOps architecture from literature and practitioner interviews, and found a stable set of components recurring across organisations: a source code repository, CI/CD, workflow orchestration, a feature store, training infrastructure, a model registry, metadata stores, model serving, and monitoring (68, Kreuzberger et al., 2023). Huyen (2022) describes the same components as an iterative design process (74, Huyen, 2022). Both accounts, however, predate the shift that Zaharia et al. (2024) describe, in which the unit of delivery becomes a compound AI system that combines models, retrieval, tools, and control logic (63, Zaharia et al., 2024). The reference architecture in this chapter therefore takes the MLOps components as its foundation and adds the layers that compound systems and agents require.

Value flows upward from data to applications, and control flows inward from the control plane to every layer.

Primary topics explored

1. Why a reference architecture
2. The six layers
3. Four patterns that carry the architecture
4. Mapping the architecture to AI Act roles
5. Expressing the architecture as a declarative model
6. Deployment variants