Cost, Capacity and Sustainability of AI

This chapter treats cost and energy as architectural constraints of AI systems rather than as bills that arrive after the design is finished. It extends the cost optimisation discipline of Architecture as Code (Chapter 15B) (72, Nordqvist, 2025), where budgets and tagging are declared in code and enforced by pipelines, to the specific economics of models: token-based pricing, scarce accelerators, and an energy footprint now visible to regulators, investors, and citizens. It covers inference cost, GPU capacity, cost levers, FinOps, and carbon accounting with the Software Carbon Intensity specification (30, Green Software Foundation, 2024).

The response is the same as for any other characteristic: declare the constraint in a cost policy under version control, measure it through telemetry attributed to system, team, and use case, enforce it through pipeline gates and runtime controls, and design for it through the levers in Figure 22.1. Principle eight in AI Governance, Chapter 4, sustainability and cost as design constraints, is the statement of intent; this chapter is its implementation.

A hosted LLM is priced per token, with separate rates for input and output, and sometimes a reduced rate for cached input. Output tokens cost more because each requires a full forward pass, while input tokens are processed in parallel. The cost of a request is therefore roughly input tokens times the input rate plus output tokens times the output rate, multiplied by request volume for the system.

Primary topics explored

1. Cost as an architectural constraint
2. Token economics and inference cost models
3. Capacity planning for GPUs
4. Architectural levers: caching, routing, and batching
5. FinOps for AI and chargeback
6. Energy and carbon accounting