This chapter compares consuming AI as a cloud service, whether a model API, a managed platform, or a feature embedded in a software-as-a-service product, with running open-weight models on accelerators the organisation controls. Chapter 13 treated placement at the level of infrastructure; this chapter asks what the organisation takes on with each choice, challenge by challenge, so that the choice is deliberate and its obligations are staffed and funded.
The binary framing of cloud against on-premises hides most of what matters, because there are three consumption models and responsibilities shift at each boundary. With an AI feature embedded in a SaaS product, such as a drafting assistant in the office suite, the organisation configures a tenant and signs a contract but does not choose the model, see its version, or control when it changes. With a model API or managed AI platform, it builds its own application and owns the prompts, retrieval, evaluation, and gateway, but not the weights or the serving stack. With a self-hosted open-weight model on owned hardware, in colocation, or in a private cloud, it owns every layer from the licence to the incident response.
Amazon Web Services describes the same gradient in its Generative AI Security Scoping Matrix, which orders five scopes "from least ownership to greatest ownership" (196, AWS, n.d.), and most differences in this chapter follow from where a workload sits on it.
Primary topics explored
- Three ways of consuming AI
- Cost and its predictability
- Capacity, accelerator supply and power
- Model change: who controls the version
- Security and the shared-responsibility split
- Data protection, residency and retention terms
About AI Platform and Security
AI Platform and Security is Book 3 of The Transformed Architecture. It shows how large European enterprises and public authorities build, secure and operate the platform on which artificial intelligence runs: data, models and DevSecMLOps, retrieval and agents, infrastructure and sovereignty, AI security engineering, evaluation, observability and cost.
The companion volume AI Governance covers risk management, governance, assurance, organisation and leadership; Architecture as Code supplies the underlying discipline.
Platform engineers, security architects, MLOps teams, AI architects and technical decision-makers.
Gunnar Nordqvist
Conceived, directed and published the books of The Transformed Architecture.
Continue with the full chapter
Chapter 14: Cloud AI Services versus On-Premises AI: Different Challenges in AI Platform and Security gives the detailed explanations, practical examples and implementation patterns behind this summary.
This whitepaper belongs to “Infrastructure” and is one of 29 drawn from the manuscript.