An AI boss cannot direct or evaluate what it cannot see. Every system that assigns visits, allocates claims or scores performance runs on data about the people it manages: when they start and stop, where they are, how fast they work and, in some products, how they sound and look. This chapter concerns two management functions: M2 Evaluation, because monitoring data becomes the evidence on which performance and conduct are judged, and M5 Care, because the same data can protect health and safety or erode the voice and psychological safety on which a healthy workplace depends. It stages the debate between those who see pervasive monitoring as a chilling force and those who see proportionate, negotiated monitoring as lawful and protective.
Algorithmic management turns work into data before it manages it. Of the six mechanisms of algorithmic control that Kellogg, Valentine and Christin identify (Chapter 2), recording underlies the rest (122, Kellogg et al., 2020): an allocation rule needs to know who is free, a rate needs a count, a score needs a history. The joint ILO and Joint Research Centre case studies found algorithmic management already established in regular workplaces in logistics and healthcare, and named surveillance among its risks to job quality where legal safeguards are weak (131, International Labour Organization, 2024).
The Article 29 Working Party warned that an all-in-one office suite may observe all use of information technology at work (50, Article 29 Data Protection Working Party, 2017). The distinction that matters is between what a management system requires and what it invites. A visit-routing system requires the start and end of each visit, not location between tasks or an estimate of mood. Yet the by-products invite further use: once times exist per person, a figure per person can be computed, then a ranking, then a pay input. Karen Levy's study of long-haul trucking shows the mechanism: fleet management systems aggregated real-time data at a remote dispatcher, who could then challenge drivers' accounts of their own conditions (114, Levy, 2015).
Primary topics explored
- What the AI boss needs to see
- What courts and authorities decided
- Article 88 and national law
- Suited to monitor
- What research measured
- The debate
About AI Boss
AI Boss is Book 7 of The Transformed Architecture. It examines what happens when artificial intelligence takes the manager's chair: algorithmic management of direction, evaluation, consequence, development and care, the effects on people, the management of AI agents, Europe's legal answer, and how to design an accountable AI boss, for European enterprises and public authorities.
It builds on the conclusions of Architecture as Code, AI Governance, AI Platform and Security, AI Red Team, AI Threat and The Transformed Security Architect.
Executives, line managers, HR and people leaders, works councils and trade unions, public employers, architects and AI governance leads, data protection and legal officers, procurement and programme leaders, and risk, compliance and audit.
Gunnar Nordqvist
Conceived, directed and published the books of The Transformed Architecture.
Continue with the full chapter
Chapter 13: Surveillance at Work and the Chilling Effect in AI Boss gives the detailed explanations, practical examples and implementation patterns behind this summary.
This whitepaper belongs to “What It Does to People” and is one of 37 drawn from the manuscript.