Field guide

1. Problem-Based Software Requirements Specification

   Rafael Gorski M. Souza and Paulo Cézar Stadzisz, Revista Eletrônica de Sistemas de Informação · 2016

   Framework · Stable · also Design

   On the radar: Problem-based requirements (Trial)

   Why it matters

   Starts from named customer problems, not requested features, and derives customer needs and requirements from them. In an AI programme, every capability, including each agent, should then trace to a problem it solves.

   What to check

   Confirm every requirement derives from a customer need, and each need from a named customer problem; cut the rest. MIT-licensed agent skill: github.com/RafaelGorski/Problem-Based-SRS.

2. Rules of Machine Learning: Best Practices for ML Engineering

   Martin Zinkevich, Google · living document

   Guide · Stable · also Design

   On the radar: Baseline before model (Adopt)

   Why it matters

   Its first rule is not to be afraid to launch without machine learning. That sets the right opening question for any sponsor: what simple baseline must the AI beat to justify its cost?

   What to check

   Ship a heuristic or non-ML baseline first, instrument it, and require every model to beat it on the same metric.

3. Hidden Technical Debt in Machine Learning Systems

   D. Sculley et al., NeurIPS · 2015

   Paper · Stable · also Operate

   On the radar: Feedback-loop mapping (Trial)

   Why it matters

   The ML code is a small fraction of a production ML system; the surrounding data, configuration and serving code carry most of the long-term cost. Budget for that, not for the demo.

   What to check

   Inventory undeclared consumers, data dependencies and feedback loops around each model before committing to scale it.

4. Building effective agents

   Anthropic · 2024

   Guide · Stable · also Design

   On the radar: Workflows before autonomous agents (Adopt), Multi-agent decomposition (Assess)

   Why it matters

   Separates predefined workflows from autonomous agents and argues for the simplest design that works. The publisher flags that the tooling described has since changed; read it for the design reasoning.

   What to check

   For each proposed agent, document why a fixed workflow would not meet the requirement before granting it open-ended control.

5. Measuring AI Ability to Complete Long Software Tasks

   Thomas Kwa et al., METR · 2025

   Paper · Evolving · also Design

   On the radar: Task time horizons (Assess)

   Why it matters

   Measures capability as the length of task, in skilled human time, that a model completes half the time. A sober way to set scope: delegate work inside that horizon, not beyond it.

   What to check

   Estimate how long your target tasks take a skilled person and compare that with the published time horizon of the model you plan to use.

6. "Everyone wants to do the model work, not the data work": Data Cascades in High-Stakes AI

   Nithya Sambasivan et al., ACM CHI · 2021

   Paper · Stable · also Design

   Why it matters

   In interviews with 53 practitioners, 92% had met compounding data failures that surfaced late. The study ties them to practices that undervalue data work, which leaders can change.

   What to check

   Log when each data issue was introduced and when it was detected; the gap is your cascade exposure.

Also read here: AI Agents That Matter (Design), How to Measure Anything (Operate), ISO/IEC 42005 AI system impact assessment (Govern), AI RMF 1.0 (Govern)

On the radar: Baseline before model (Adopt), Workflows before autonomous agents (Adopt), Problem-based requirements (Trial), Value-of-information measurement (Adopt), Task time horizons (Assess), NIST AI RMF and Generative AI Profile (Adopt), AI impact assessment before build (Adopt)

1. τ-bench: A Benchmark for Tool-Agent-User Interaction in Real-World Domains

   Shunyu Yao et al. · 2024

   Paper · Stable

   On the radar: Repeated-trial reliability (pass^k) (Trial), Benchmark scores as launch evidence (Hold)

   Why it matters

   Introduces pass^k: the chance an agent succeeds on every one of k repeated attempts at the same task. A pilot that passes once is a demo; consistency is what customers experience.

   What to check

   Run each evaluation task several times and report pass^k alongside the single-run success rate.

2. AI Agents That Matter

   Sayash Kapoor et al. · 2024

   Paper · Stable · also Purpose

   On the radar: Baseline before model (Adopt), Model cascades and routing (Trial), Cost reported with accuracy (Adopt), Benchmark scores as launch evidence (Hold)

   Why it matters

   Shows that agent evaluations often ignore cost and lack proper holdout sets, so headline accuracy can mislead. Ask for accuracy and cost together, measured on tasks the agent has not seen.

   What to check

   Plot every candidate agent on cost versus accuracy, including simple baselines, and require results on a held-out task set.

3. Judging LLM-as-a-Judge with MT-Bench and Chatbot Arena

   Lianmin Zheng et al., NeurIPS · 2023

   Paper · Stable

   On the radar: LLM judges calibrated to human grades (Trial), LLM judges without human calibration (Hold)

   Why it matters

   Strong models can approximate human preference ratings, which makes evaluation at scale affordable. The paper also documents position, verbosity and self-enhancement biases that must be controlled before trusting the scores.

   What to check

   Swap answer order, vary length and use a different model family as judge; report agreement with a human-labelled sample.

4. Who Validates the Validators? Aligning LLM-Assisted Evaluation of LLM Outputs with Human Preferences

   Shreya Shankar et al. · 2024

   Paper · Stable · also Operate

   On the radar: LLM judges calibrated to human grades (Trial), LLM judges without human calibration (Hold)

   Why it matters

   Finds that people refine their evaluation criteria while grading outputs, so automated graders must be revisited as understanding grows. Evaluation is a managed asset, not a one-time setup.

   What to check

   Version your grading criteria and LLM-judge prompts, and re-measure their agreement with human grades whenever criteria change.

5. Underspecification Presents Challenges for Credibility in Modern Machine Learning

   Alexander D'Amour et al., JMLR · 2022

   Paper · Stable · also Launch

   On the radar: Benchmark scores as launch evidence (Hold)

   Why it matters

   Models with equally strong test scores can behave very differently in deployment. A strong pilot result does not, on its own, predict production behaviour.

   What to check

   Evaluate candidate models on shifted and subgroup data that the pilot test set does not represent.

6. Inspect

   UK AI Security Institute and Meridian Labs · living document

   Framework · Evolving · also Launch

   On the radar: Versioned open evaluation harness (Trial)

   Why it matters

   An open-source evaluation framework co-developed by the UK AI Security Institute, covering coding, agentic and reasoning tasks. A shared, inspectable harness makes results reproducible and comparable across model vendors.

   What to check

   Express pilot evaluations as versioned Inspect tasks so the same suite reruns unchanged on every model or prompt change.

7. Datasheets for Datasets

   Timnit Gebru et al., Communications of the ACM · 2021

   Paper · Stable · also Govern

   On the radar: Model cards and datasheets (Adopt)

   Why it matters

   A standard set of questions about how a dataset was collected, labelled and intended to be used. Without those answers, no one can say whether pilot data represents production.

   What to check

   Require a datasheet for every evaluation and fine-tuning dataset, including its collection process, known gaps and permitted uses.

8. Thinking in Systems: A Primer

   Donella H. Meadows, edited by Diana Wright, Chelsea Green Publishing · 2008

   Book · Stable · also Scale

   On the radar: Feedback-loop mapping (Trial)

   Why it matters

   Shows how a system's structure of stocks, flows, feedback loops and delays drives its behaviour. AI systems that learn from their own outputs, or agents that call other agents, create exactly such loops.

   What to check

   Draw the feedback loops the system creates, including outputs that return as inputs, and name the delay and the limit on each.

9. Effective Software Testing: A developer's guide

   Maurício Aniche, Manning · 2022

   Book · Stable · also Launch

   Why it matters

   A systematic method for developer testing: examples from the specification, then the code's structure, contracts and property-based tests. When coding agents write code faster than people can review it, tests must find what nobody tried.

   What to check

   For each rule that money, permissions or limits depend on, write a property stating what must never happen and let a generator search for counterexamples.

Also read here: Problem-Based Software Requirements Specification (Purpose), Rules of Machine Learning (Purpose), Building effective agents (Purpose), Measuring AI Ability to Complete Long Software Tasks (Purpose), Data Cascades in High-Stakes AI (Purpose), The ML Test Score (Launch), Expanding on what we missed with sycophancy (Launch), Why Do Multi-Agent LLM Systems Fail? (Scale), Lean Software Development (Operate)

On the radar: Baseline before model (Adopt), Workflows before autonomous agents (Adopt), Feedback-loop mapping (Trial), Multi-agent decomposition (Assess), Versioned open evaluation harness (Trial), Cost reported with accuracy (Adopt), Repeated-trial reliability (pass^k) (Trial), LLM judges calibrated to human grades (Trial), Task time horizons (Assess), LLM judges without human calibration (Hold), Benchmark scores as launch evidence (Hold)

1. Reliable Product Launches at Scale

   Google, Site Reliability Engineering (O'Reilly) · 2016

   Book · Stable

   On the radar: Canary releases for model and prompt changes (Adopt)

   Why it matters

   Describes how Google turned launches into a repeatable discipline with a dedicated coordination role and a checklist. AI launches need the same rigour, plus questions specific to model behaviour.

   What to check

   Extend the launch checklist with rows for evaluation results, abuse testing, rollback of model and prompt versions, and cost limits.

2. Canarying Releases

   Google, The Site Reliability Workbook (O'Reilly) · 2018

   Book · Stable · also Operate

   On the radar: Canary releases for model and prompt changes (Adopt)

   Why it matters

   Expose a change to a small share of traffic, compare it against a control, then widen. For models and prompts, whose failures are statistical rather than crashes, this comparison is an essential safety net.

   What to check

   Route model and prompt changes through a canary with a control group and predefined quality, latency and cost thresholds.

3. The ML Test Score: A Rubric for ML Production Readiness and Technical Debt Reduction

   Eric Breck et al., IEEE Big Data · 2017

   Paper · Stable · also Design

   Why it matters

   Twenty-eight specific tests and monitoring needs drawn from a wide range of production ML systems. It turns 'is this ready for production?' into a score a review board can track.

   What to check

   Score each system against the rubric at launch review and treat untested sections as open risks with named owners.

4. OWASP Top 10 for LLM Applications 2025

   OWASP Gen AI Security Project · 2024

   Guide · Stable

   On the radar: OWASP Top 10 launch review (Adopt)

   Why it matters

   The shared vocabulary security teams use for LLM application risk, from prompt injection to unbounded consumption. Asking which items a launch has mitigated is a fast, credible review question.

   What to check

   Map each of the ten risks to a named control or an accepted-risk decision in the launch review record.

5. OWASP Top 10 for Agentic Applications for 2026

   OWASP Gen AI Security Project · 2025

   Guide · Evolving · also Scale

   On the radar: OWASP Top 10 launch review (Adopt)

   Why it matters

   A peer-reviewed list of the most critical security risks for autonomous and agentic AI systems. Use it alongside the LLM Top 10 once a system can act, not just answer.

   What to check

   Verify each agent runs with its own scoped identity and least-privilege tool permissions, never a shared human or service account.

6. Guidelines for secure AI system development

   UK NCSC, US CISA and international partners · 2023

   Guide · Stable · also Govern

   Why it matters

   Joint guidance from national cyber security agencies, organised by lifecycle: secure design, development, deployment, and operation and maintenance. A security baseline a board can point to.

   What to check

   Confirm model weights, prompts and tool credentials are covered by the same asset inventory, access control and incident process as code.

7. Expanding on what we missed with sycophancy

   OpenAI · 2025

   Postmortem · Stable · also Design

   On the radar: Blameless postmortems for AI incidents (Adopt), Continuous quality evaluation in production (Adopt)

   Why it matters

   A model update shipped on positive evaluations and A/B results despite expert testers saying it felt off, then was rolled back. OpenAI committed to treating behaviour issues as launch-blocking.

   What to check

   Give named expert reviewers authority to block a model release on qualitative findings, even when quantitative evaluations pass.

8. Guidelines for Human-AI Interaction

   Saleema Amershi et al., Microsoft Research (ACM CHI) · 2019

   Paper · Stable

   Why it matters

   Eighteen guidelines, validated with 49 design practitioners, for how AI features set expectations, handle mistakes and support correction. They shape how users experience the system on the days it is wrong.

   What to check

   Review the launch UI against the guidelines on efficient correction, efficient dismissal and explaining why the system did what it did.

9. Adversarial Machine Learning: A Taxonomy and Terminology of Attacks and Mitigations

   NIST (NIST AI 100-2 E2025) · 2025

   Guide · Stable · also Scale

   On the radar: Injection-resistant agent design patterns (Trial)

   Why it matters

   NIST's reference taxonomy of attacks on predictive and generative AI, including direct and indirect prompt injection and the security of agents. One vocabulary for security, engineering and risk teams.

   What to check

   Use its attack categories as the headings of your AI threat model so findings map to a public reference.

10. Evals for AI Engineers: Systematically Measuring and Improving AI Applications

    Shreya Shankar and Hamel Husain, O'Reilly · 2026

    Book · Draft · also Operate

    Why it matters

    Evals turn an LLM system's limits into measured, rerunnable claims a release can be gated on. Announced to cover error analysis, LLM judges and production monitoring; forthcoming November 2026, so cited for scope only.

    What to check

    Begin with error analysis: find and rank failures in real outputs, build evaluations for the largest, then monitor production for regressions.

Also read here: Underspecification Presents Challenges for Credibility in Modern… (Design), Inspect (Design), Effective Software Testing (Design), Model Context Protocol Specification (Scale), Design Patterns for Securing LLM Agents against Prompt Injections (Scale), Best Current Practice for OAuth 2.0 Security (Scale), Service Level Objectives (Operate), EU Artificial Intelligence Act (Govern), AI RMF Generative AI Profile (Govern), Model Cards for Model Reporting (Govern), Moffatt v. Air Canada, 2024 BCCRT 149 (Govern)

On the radar: Canary releases for model and prompt changes (Adopt), Injection-resistant agent design patterns (Trial), OAuth to the current security BCP (Adopt), Model Context Protocol (Trial), Versioned open evaluation harness (Trial), Answer-quality SLOs (Trial), Repeated-trial reliability (pass^k) (Trial), Benchmark scores as launch evidence (Hold), EU AI Act risk classification (Adopt), Model cards and datasheets (Adopt), OWASP Top 10 launch review (Adopt), Shifting accountability to the model or provider (Hold)

1. Model Context Protocol Specification (2026-07-28)

   Model Context Protocol project (LF Projects) · 2026

   Specification · Evolving · also Launch

   On the radar: OAuth to the current security BCP (Adopt), Model Context Protocol (Trial)

   Why it matters

   An open protocol for connecting AI applications to tools and data. Standardising on one protocol reduces bespoke integration work and gives security teams a single interface to review.

   What to check

   Pin the protocol revision your servers support and review its authorization section before exposing any MCP server beyond localhost.

2. Agent2Agent (A2A) Protocol Specification

   A2A Project, Linux Foundation · 2026

   Specification · Evolving

   On the radar: Agent2Agent protocol (Assess)

   Why it matters

   An open protocol for agents built by different teams or vendors to discover each other and delegate tasks. It matters once agents multiply and must cooperate across organisational boundaries.

   What to check

   Treat each Agent Card as an interface contract: version it, sign it, and review its declared skills and security schemes.

3. The Tail at Scale

   Jeffrey Dean and Luiz André Barroso, Communications of the ACM · 2013

   Paper · Stable

   On the radar: Tail latency per agent step (Adopt)

   Why it matters

   Brief latency spikes that are harmless in small systems come to dominate performance at scale. Agent workflows that chain many model and tool calls inherit this problem directly.

   What to check

   Measure p99 latency per step and end to end for agent runs, and set per-call timeouts and hedging where steps are idempotent.

4. Addressing Cascading Failures

   Google, Site Reliability Engineering (O'Reilly) · 2016

   Book · Stable · also Operate

   On the radar: Unbudgeted automatic retries (Hold)

   Why it matters

   Explains how overload in one component spreads through retries and dependencies until a whole service fails. Agents that retry automatically can turn a model-provider slowdown into exactly this.

   What to check

   Cap retries with budgets and jittered backoff, propagate deadlines through agent steps, and load-test the provider-throttled case.

5. FrugalGPT: How to Use Large Language Models While Reducing Cost and Improving Performance

   Lingjiao Chen, Matei Zaharia and James Zou · 2023

   Paper · Stable

   On the radar: Model cascades and routing (Trial), Cost reported with accuracy (Adopt)

   Why it matters

   Routing each query through a cascade, cheaper models first and stronger ones only when needed, matched the best single model at up to 98% lower cost. Model choice is a per-request decision.

   What to check

   Log which model served each request and at what cost, then test whether a cheaper first-pass model meets the quality bar.

6. FinOps for AI Overview

   FinOps Foundation · living document

   Guide · Evolving · also Govern

   On the radar: Cost and calls per completed task (Trial)

   Why it matters

   Vendor-neutral practice for allocating and controlling AI spend, including token-based pricing and scarce GPU capacity. AI costs move faster and reach more teams than typical cloud spend.

   What to check

   Tag every model call with team, product and use case so cost per outcome can be reported, not just total spend.

7. Why Do Multi-Agent LLM Systems Fail?

   Mert Cemri et al. · 2025

   Paper · Stable · also Design

   On the radar: Workflows before autonomous agents (Adopt), Multi-agent decomposition (Assess)

   Why it matters

   A taxonomy of fourteen failure modes from expert-annotated traces, grouped into system design, misalignment between agents, and task verification. Adding agents adds coordination risk, not only capability.

   What to check

   Classify failed multi-agent runs with the paper's taxonomy, and require an explicit verification step before any agent marks a task complete.

8. Design Patterns for Securing LLM Agents against Prompt Injections

   Luca Beurer-Kellner et al. · 2025

   Paper · Stable · also Launch

   On the radar: Injection-resistant agent design patterns (Trial), Controls enforced outside the model (Trial)

   Why it matters

   Proposes agent designs with provable resistance to prompt injection by limiting what an agent can do after reading untrusted input. Each trades some utility for security, a choice the business should own.

   What to check

   For each agent, identify where untrusted content enters and which pattern, such as plan-then-execute, constrains actions after that point.

9. Best Current Practice for OAuth 2.0 Security

   IETF (RFC 9700) · 2025

   Standard · Stable · also Launch

   On the radar: OAuth to the current security BCP (Adopt), Model Context Protocol (Trial), Per-agent identity and least privilege (Adopt)

   Why it matters

   Agents increasingly act on delegated credentials, and agent protocols such as MCP authorise through OAuth. This is the IETF's current security baseline for OAuth deployments.

   What to check

   Confirm agent access tokens are audience-restricted and sender-constrained, and that public-client refresh tokens are rotated or sender-constrained.

Also read here: Thinking in Systems (Design), OWASP Top 10 for Agentic Applications for 2026 (Launch), Adversarial Machine Learning (Launch), Semantic conventions for generative AI systems (Operate), Trace Context (Operate), Practices for Governing Agentic AI Systems (Govern)

On the radar: Feedback-loop mapping (Trial), Model cascades and routing (Trial), Injection-resistant agent design patterns (Trial), Multi-agent decomposition (Assess), Unbudgeted automatic retries (Hold), W3C Trace Context across agent hops (Adopt), OAuth to the current security BCP (Adopt), OpenTelemetry GenAI semantic conventions (Trial), Model Context Protocol (Trial), Agent2Agent protocol (Assess), Cost reported with accuracy (Adopt), Tail latency per agent step (Adopt), Cost and calls per completed task (Trial), Per-agent identity and least privilege (Adopt)

1. Semantic conventions for generative AI systems

   OpenTelemetry · living document

   Specification · Evolving · also Scale

   On the radar: OpenTelemetry GenAI semantic conventions (Trial), Telemetry only in a vendor's console (Hold), Cost and calls per completed task (Trial), Capturing prompts and outputs by default (Hold)

   Why it matters

   The open standard for recording model calls, agent steps, tool use and token usage as telemetry. It keeps observability portable across vendors. Still marked Development, so expect changes.

   What to check

   Emit gen_ai spans and token-usage metrics for every model and tool call, and record which convention version each service emits.

2. Trace Context

   W3C · 2021

   Standard · Stable · also Scale

   On the radar: W3C Trace Context across agent hops (Adopt), OpenTelemetry GenAI semantic conventions (Trial), Telemetry only in a vendor's console (Hold)

   Why it matters

   The web standard for carrying a trace identifier across services. When one user request passes through several agents, tools and vendors, it is what lets you reconstruct the whole path.

   What to check

   Confirm traceparent headers propagate through every agent hop, tool call and third-party API, including asynchronous queues.

3. Dapper, a Large-Scale Distributed Systems Tracing Infrastructure

   Benjamin H. Sigelman et al., Google · 2010

   Paper · Stable

   On the radar: W3C Trace Context across agent hops (Adopt)

   Why it matters

   Google's account of ubiquitous, low-overhead tracing across production services. Its lesson holds for agents: tracing pays off only when it is everywhere and cheap enough to leave on.

   What to check

   Decide your trace sampling policy explicitly; keep full traces for errors, high-cost agent runs and sampled quality reviews.

4. Service Level Objectives

   Google, Site Reliability Engineering (O'Reilly) · 2016

   Book · Stable · also Launch

   On the radar: Tail latency per agent step (Adopt), Answer-quality SLOs (Trial)

   Why it matters

   Defines reliability through indicators and objectives that reflect what users notice, and makes the target explicit. For AI systems, answer quality belongs beside latency and availability.

   What to check

   Define at least one quality SLI per AI feature, such as graded answer acceptability on sampled traffic, with an explicit SLO target.

5. Postmortem Culture: Learning from Failure

   Google, Site Reliability Engineering (O'Reilly) · 2016

   Book · Stable · also Govern

   On the radar: Blameless postmortems for AI incidents (Adopt), Shared AI incident disclosure (Assess)

   Why it matters

   Blameless postmortems turn incidents into durable fixes and shared knowledge. AI incidents, such as harmful outputs or quality regressions, deserve the same written review as outages.

   What to check

   Add AI-specific triggers to the postmortem policy: a quality regression, a harmful output reaching users, or an agent acting outside its scope.

6. A postmortem of three recent issues

   Anthropic · 2025

   Postmortem · Stable

   On the radar: Blameless postmortems for AI incidents (Adopt), Continuous quality evaluation in production (Adopt), Answer-quality SLOs (Trial)

   Why it matters

   Three infrastructure bugs intermittently degraded responses, and the evaluations in place did not capture what users were reporting. Quality can regress in production through infrastructure alone.

   What to check

   Run quality evaluations continuously against true production endpoints, and keep a privacy-safe path to investigate user-reported bad outputs.

7. How to Measure Anything: Finding the Value of Intangibles in Business, 3rd Edition

   Douglas W. Hubbard, Wiley · 2014

   Book · Stable · also Purpose

   On the radar: Value-of-information measurement (Adopt)

   Why it matters

   Treats measurement as reducing uncertainty for a decision, not reaching certainty, and uses the value of information to choose what to measure. It answers the claim that an AI system's benefits are intangible.

   What to check

   For each production metric, name the decision it informs and estimate what better information is worth; retire metrics that inform no decision.

8. Lean Software Development: An Agile Toolkit

   Mary Poppendieck and Tom Poppendieck, Addison-Wesley · 2003

   Book · Stable · also Design

   Why it matters

   Lean thinking for software. Its value stream map follows one request to delivery and shows where work waits, is handed off or is redone. When agents make coding faster, that waste is what remains.

   What to check

   Map one change from request to release. Record working time, waiting time and where each defect was found, then remove the largest waste first.

Also read here: Hidden Technical Debt in Machine Learning Systems (Purpose), Who Validates the Validators? (Design), Canarying Releases (Launch), Evals for AI Engineers (Launch), Addressing Cascading Failures (Scale), Models as Insider Risks in the Super Intelligence Era (Govern), Ironies of automation (Govern)

On the radar: Canary releases for model and prompt changes (Adopt), Blameless postmortems for AI incidents (Adopt), Unbudgeted automatic retries (Hold), W3C Trace Context across agent hops (Adopt), OpenTelemetry GenAI semantic conventions (Trial), Telemetry only in a vendor's console (Hold), Continuous quality evaluation in production (Adopt), Tail latency per agent step (Adopt), Value-of-information measurement (Adopt), Answer-quality SLOs (Trial), LLM judges calibrated to human grades (Trial), Cost and calls per completed task (Trial), LLM judges without human calibration (Hold), Controls enforced outside the model (Trial), Shared AI incident disclosure (Assess), Human approval as the only safeguard (Hold), Capturing prompts and outputs by default (Hold)