A courier in Bologna who booked an evening delivery session and then joined a strike would lose points in the ranking that decided when she could book work next; the ranking recorded only that she had stayed away, not why. Drivers in the United Kingdom deactivated for suspected fraud were told of the decision, but the court that examined their cases could not establish that any person had weighed their side of the story. Neither case involves a manager who disliked anyone; both involve a rule nobody had tested for its effect on groups and a route of appeal that led nowhere. This chapter stages the debate on fairness, bias and contestability, and concerns M2 Evaluation, because scores, rankings and performance figures are where bias enters, and M3 Consequence, because pay, promotion, deactivation and access to work are where it does its damage and where contest matters most.
The chapter builds on AI Threat, Chapters 11 and 12, on discrimination in automated judgement and on public authorities whose automated systems went wrong (34, Nordqvist, 2026), and on conclusion F31, that the line between decision support and automated decision is drawn in the data model and the audit log, with a reasons log for every case (AI Governance, Chapter 17) (31, Nordqvist, 2026). What is new is the workplace, where the relationship continues after the decision; Chapter 27 designs contestability in full.
The evidence comes from audits outside the workplace, from litigation and from data sets that show an outcome gap without a biased rule, and the debate depends on keeping them apart.
Primary topics explored
- Documented bias
- Why bias enters
- The appeal that never comes
- When contest fails at scale
- What the law requires of reasons and review
- The debate
About AI Boss
AI Boss is Book 7 of The Transformed Architecture. It examines what happens when artificial intelligence takes the manager's chair: algorithmic management of direction, evaluation, consequence, development and care, the effects on people, the management of AI agents, Europe's legal answer, and how to design an accountable AI boss, for European enterprises and public authorities.
It builds on the conclusions of Architecture as Code, AI Governance, AI Platform and Security, AI Red Team, AI Threat and The Transformed Security Architect.
Executives, line managers, HR and people leaders, works councils and trade unions, public employers, architects and AI governance leads, data protection and legal officers, procurement and programme leaders, and risk, compliance and audit.
Gunnar Nordqvist
Conceived, directed and published the books of The Transformed Architecture.
Continue with the full chapter
Chapter 12: Fairness, Bias and the Appeal That Never Comes in AI Boss gives the detailed explanations, practical examples and implementation patterns behind this summary.
This whitepaper belongs to “What It Does to People” and is one of 37 drawn from the manuscript.