The route from Purpose to Govern
Six camps from the first decision to accountable operation: the question each one must answer before the team moves on, and the sources worth reading on that stretch. Read across for the route, down for the evidence.
- Shape
- Rule standard, specification, regulation
- Practice framework, guide, book
- Evidence paper, postmortem
- Fill
- Stable
- Evolving
- Draft
- Red
- Original note
Edition October 2026 · 3 notes, 52 sources · field guide updated 17 October 2026
The route, camp by camp
-
1Purpose
Is this worth building, and what would prove it?
- Problem-Based Software Requirements SpecificationRafael Gorski M. Souza and Paulo Cézar Stadzisz · 2016. Framework, stable.
- Rules of Machine LearningMartin Zinkevich · living. Guide, stable.
- Hidden Technical Debt in Machine Learning SystemsD. Sculley et al. · 2015. Paper, stable.
- Building effective agentsAnthropic · 2024. Guide, stable.
- Measuring AI Ability to Complete Long Software TasksThomas Kwa et al. · 2025. Paper, evolving.
- Data Cascades in High-Stakes AINithya Sambasivan et al. · 2021. Paper, stable.
Also read here
Move on when there is a named owner, a decision this will change, and a measure of success agreed before building.
-
2Design
Does it work on real data, measured honestly?
GN-0002 · Draft Test triangulation , original note- τ-benchShunyu Yao et al. · 2024. Paper, stable.
- AI Agents That MatterSayash Kapoor et al. · 2024. Paper, stable.
- Judging LLM-as-a-Judge with MT-Bench and Chatbot ArenaLianmin Zheng et al. · 2023. Paper, stable.
- Who Validates the Validators?Shreya Shankar et al. · 2024. Paper, stable.
- Underspecification Presents Challenges for Credibility in Modern…Alexander D'Amour et al. · 2022. Paper, stable.
- InspectUK AI Security Institute and Meridian Labs · living. Framework, evolving.
- Datasheets for DatasetsTimnit Gebru et al. · 2021. Paper, stable.
- Thinking in SystemsDonella H. Meadows · 2008. Book, stable.
- Effective Software TestingMaurício Aniche · 2022. Book, stable.
Also read here
On the radar
- 1. Baseline before model, Adopt
- 2. Workflows before autonomous agents, Adopt
- 6. Feedback-loop mapping, Trial
- 9. Multi-agent decomposition, Assess
- 15. Versioned open evaluation harness, Trial
- 19. Cost reported with accuracy, Adopt
- 23. Repeated-trial reliability (pass^k), Trial
- 24. LLM judges calibrated to human grades, Trial
- 26. Task time horizons, Assess
- 27. LLM judges without human calibration, Hold
- 28. Benchmark scores as launch evidence, Hold
Move on when results on real data beat a simple baseline, and the failures are written down, not only the average.
-
3Launch
Can we ship it safely to real users?
GN-0003 · Draft Eval-driven development , original note- Reliable Product Launches at ScaleGoogle · 2016. Book, stable.
- Canarying ReleasesGoogle · 2018. Book, stable.
- The ML Test ScoreEric Breck et al. · 2017. Paper, stable.
- OWASP Top 10 for LLM Applications 2025OWASP Gen AI Security Project · 2024. Guide, stable.
- OWASP Top 10 for Agentic Applications for 2026OWASP Gen AI Security Project · 2025. Guide, evolving.
- Guidelines for secure AI system developmentUK NCSC et al. · 2023. Guide, stable.
- Expanding on what we missed with sycophancyOpenAI · 2025. Postmortem, stable.
- Guidelines for Human-AI InteractionSaleema Amershi et al. · 2019. Paper, stable.
- Adversarial Machine LearningNIST (NIST AI 100-2 E2025) · 2025. Guide, stable.
- Evals for AI EngineersShreya Shankar and Hamel Husain · 2026. Book, draft.
Also read here
On the radar
- 3. Canary releases for model and prompt changes, Adopt
- 8. Injection-resistant agent design patterns, Trial
- 12. OAuth to the current security BCP, Adopt
- 14. Model Context Protocol, Trial
- 15. Versioned open evaluation harness, Trial
- 22. Answer-quality SLOs, Trial
- 23. Repeated-trial reliability (pass^k), Trial
- 28. Benchmark scores as launch evidence, Hold
- 30. EU AI Act risk classification, Adopt
- 31. Model cards and datasheets, Adopt
- 33. OWASP Top 10 launch review, Adopt
- 39. Shifting accountability to the model or provider, Hold
Move on when a staged rollout, a tested rollback and a human escalation path are live before the first user.
-
4Scale
Can it grow in users, agents and cost without losing control?
- Model Context Protocol SpecificationModel Context Protocol project (LF Projects) · 2026. Specification, evolving.
- Agent2Agent (A2A) Protocol SpecificationA2A Project · 2026. Specification, evolving.
- The Tail at ScaleJeffrey Dean and Luiz André Barroso · 2013. Paper, stable.
- Addressing Cascading FailuresGoogle · 2016. Book, stable.
- FrugalGPTLingjiao Chen et al. · 2023. Paper, stable.
- FinOps for AI OverviewFinOps Foundation · living. Guide, evolving.
- Why Do Multi-Agent LLM Systems Fail?Mert Cemri et al. · 2025. Paper, stable.
- Design Patterns for Securing LLM Agents against Prompt InjectionsLuca Beurer-Kellner et al. · 2025. Paper, stable.
- Best Current Practice for OAuth 2.0 SecurityIETF (RFC 9700) · 2025. Standard, stable.
Also read here
On the radar
- 6. Feedback-loop mapping, Trial
- 7. Model cascades and routing, Trial
- 8. Injection-resistant agent design patterns, Trial
- 9. Multi-agent decomposition, Assess
- 10. Unbudgeted automatic retries, Hold
- 11. W3C Trace Context across agent hops, Adopt
- 12. OAuth to the current security BCP, Adopt
- 13. OpenTelemetry GenAI semantic conventions, Trial
- 14. Model Context Protocol, Trial
- 16. Agent2Agent protocol, Assess
- 19. Cost reported with accuracy, Adopt
- 20. Tail latency per agent step, Adopt
- 25. Cost and calls per completed task, Trial
- 32. Per-agent identity and least privilege, Adopt
Move on when the cost of a completed task is known, and the platform enforces limits on spend and agent loops.
-
5Operate
Can we see, fix and improve it in production?
GN-0001 · Draft Agentic observability , original note- Semantic conventions for generative AI systemsOpenTelemetry · living. Specification, evolving.
- Trace ContextW3C · 2021. Standard, stable.
- Dapper, a Large-Scale Distributed Systems Tracing InfrastructureBenjamin H. Sigelman et al. · 2010. Paper, stable.
- Service Level ObjectivesGoogle · 2016. Book, stable.
- Postmortem CultureGoogle · 2016. Book, stable.
- A postmortem of three recent issuesAnthropic · 2025. Postmortem, stable.
- How to Measure AnythingDouglas W. Hubbard · 2014. Book, stable.
- Lean Software DevelopmentMary Poppendieck and Tom Poppendieck · 2003. Book, stable.
Also read here
On the radar
- 3. Canary releases for model and prompt changes, Adopt
- 4. Blameless postmortems for AI incidents, Adopt
- 10. Unbudgeted automatic retries, Hold
- 11. W3C Trace Context across agent hops, Adopt
- 13. OpenTelemetry GenAI semantic conventions, Trial
- 17. Telemetry only in a vendor's console, Hold
- 18. Continuous quality evaluation in production, Adopt
- 20. Tail latency per agent step, Adopt
- 21. Value-of-information measurement, Adopt
- 22. Answer-quality SLOs, Trial
- 24. LLM judges calibrated to human grades, Trial
- 25. Cost and calls per completed task, Trial
- 27. LLM judges without human calibration, Hold
- 36. Controls enforced outside the model, Trial
- 37. Shared AI incident disclosure, Assess
- 38. Human approval as the only safeguard, Hold
- 40. Capturing prompts and outputs by default, Hold
In control when every request leaves one connected trace, incidents are reviewed, and each lesson becomes an evaluation.
-
6Govern
Who is accountable, and can we prove it?
- EU Artificial Intelligence ActEuropean Parliament and Council of the European Union · 2024. Regulation, evolving.
- ISO/IEC 42001 AI management systemISO/IEC JTC 1/SC 42 · 2023. Standard, stable.
- ISO/IEC 42005 AI system impact assessmentISO/IEC JTC 1/SC 42 · 2025. Standard, stable.
- AI RMF 1.0NIST (NIST AI 100-1) · 2023. Framework, stable.
- AI RMF Generative AI ProfileNIST (NIST AI 600-1) · 2024. Framework, stable.
- Practices for Governing Agentic AI SystemsYonadav Shavit et al. · 2023. Paper, evolving.
- Models as Insider Risks in the Super Intelligence EraSatya Nadella · 2026. Guide, evolving.
- Model Cards for Model ReportingMargaret Mitchell et al. · 2019. Paper, stable.
- Moffatt v. Air Canada, 2024 BCCRT 149Civil Resolution Tribunal of British Columbia · 2024. Postmortem, stable.
- Ironies of automationLisanne Bainbridge · 1983. Paper, stable.
Also read here
On the radar
- 4. Blameless postmortems for AI incidents, Adopt
- 29. NIST AI RMF and Generative AI Profile, Adopt
- 30. EU AI Act risk classification, Adopt
- 31. Model cards and datasheets, Adopt
- 32. Per-agent identity and least privilege, Adopt
- 34. ISO/IEC 42001 AI management system, Adopt
- 35. AI impact assessment before build, Adopt
- 36. Controls enforced outside the model, Trial
- 37. Shared AI incident disclosure, Assess
- 38. Human approval as the only safeguard, Hold
- 39. Shifting accountability to the model or provider, Hold
- 40. Capturing prompts and outputs by default, Hold
In control when every system has an accountable owner and a record of what it did, on whose authority and why.
What production teaches goes back to Purpose.