Data Architecture for AI

Part II begins at the lowest layer of the reference architecture of Chapter 4, the data foundation. This chapter explains why data products and data contracts are the unit of trust between data owners and model teams, how lineage makes a model's claims about its data verifiable, where feature stores and vector stores fit, why synthetic data has sharp limits, and how the GDPR, the Data Act, and the Data Governance Act shape what the architecture may do.

In classical systems the architect designs the behaviour and the data supports it. In an AI system the relationship inverts, because the behaviour is derived from the data. Kleppmann's distinction between systems of record and derived data frames this (77, Kleppmann, 2017): a trained model is a lossy, cached transformation of its training set that can be regenerated only if the inputs are known, versioned, and still available.

The quality, legality, and reproducibility of the model are therefore bounded by those of the data. The architectural response is to make data a product with an accountable owner, to make its interface a testable contract, and to record lineage and lawful basis when a dataset enters the AI estate rather than when a regulator asks. AI Governance, Chapter 5 names what its data readiness assessment looks for: named data owners, known quality and lineage, and a lawful basis for each purpose. This chapter describes the architecture that closes them.

Primary topics explored

1. Why the data layer is the architecture
2. Data products and the data mesh in large organisations
3. Composition
4. Known gaps and skews
5. Uses
6. Maintenance