Most organisations that I meet have no shortage of AI pilots; they have a shortage of AI in production. A demonstration that impressed a steering committee in March is still a demonstration in November, and the team that built it has moved on. This chapter explains why pilots stall and how to codify the path from idea to governed production service, treating the platform, the intake process, and the readiness review as versioned artefacts rather than meetings, so that the second use case is cheaper than the first and the tenth is routine.
The pilot trap is the state in which an organisation can start AI initiatives but not finish them. Its symptoms are proofs of concept with no owner, model code in a notebook on a laptop, and a sponsor who no longer answers emails about funding. Paleyes, Urma and Lawrence (2022) surveyed published case studies of machine learning deployment and found difficulties at every stage of the lifecycle, many rooted in organisational rather than algorithmic causes (67, Paleyes et al., 2022). The trap is not a failure of talent but of architecture.
A pilot optimises for showing that a capability is possible, and defers everything production demands. Sculley and colleagues (2015) catalogued the technical debt this creates: entanglement between features, undeclared consumers of model output, untracked data dependencies, glue code around a small core of learning logic, and configuration that grows faster than the code it configures (43, Sculley et al., 2015). A pilot accumulates all of these in weeks, and the debt falls due at the move to production. Each of the differences between machine learning and conventional software engineering that Amershi and colleagues observed at Microsoft, described in Chapter 7, becomes a stalling point: the pilot used a data extract that cannot be refreshed lawfully, fine-tuned a model the organisation has no licence to deploy, or bundled retrieval, prompting, and post-processing into one untestable script (65, Amershi et al., 2019).
Primary topics explored
- The pilot trap
- Platform thinking and golden paths for AI
- Use-case intake and prioritisation
- Production readiness reviews
- Scaling across business units and agencies
- Summary
About AI Platform and Security
AI Platform and Security is Book 3 of The Transformed Architecture. It shows how large European enterprises and public authorities build, secure and operate the platform on which artificial intelligence runs: data, models and DevSecMLOps, retrieval and agents, infrastructure and sovereignty, AI security engineering, evaluation, observability and cost.
The companion volume AI Governance covers risk management, governance, assurance, organisation and leadership; Architecture as Code supplies the underlying discipline.
Platform engineers, security architects, MLOps teams, AI architects and technical decision-makers.
Gunnar Nordqvist
Conceived, directed and published the books of The Transformed Architecture.
Continue with the full chapter
Chapter 23: From Pilot to Production: Scaling AI Across the Organisation in AI Platform and Security gives the detailed explanations, practical examples and implementation patterns behind this summary.
This whitepaper belongs to “Delivery and Operations” and is one of 29 drawn from the manuscript.