Rafael Gorski

Agents are software. Build them like it.

Launching and scaling an AI system is a climb. I build agents and the software around them, and I work with teams the way a mountain guide does: walk the route with them, name the valleys before we reach them, and check the gear at every camp.

The route

Six camps, from the first decision to accountable operation. At each one the team answers a question before it moves on.

  1. 1Purpose

    Is this worth building, and what would prove it?

    Don't leave base camp with a feature request. Leave with a problem, the person who has it, and a way to know it's solved.

    This stretch on the route6 sources in the field guide

  2. 2Design

    Does it work on real data, measured honestly?

    Most routes change here, once real data arrives. Turning back at this camp is cheap. Turning back after launch is not.

    This stretch on the route9 sources in the field guide

  3. 3Launch

    Can we ship it safely to real users?

    Go up in small groups, with a way back down that someone has already tested.

    This stretch on the route10 sources in the field guide

  4. 4Scale

    Can it grow in users, agents and cost without losing control?

    Higher up, every step costs more. Track the cost of a completed task, and let the platform enforce the limits, not the prompt.

    This stretch on the route9 sources in the field guide

  5. 5Operate

    Can we see, fix and improve it in production?

    Most of the time on the mountain is spent here, after launch. Read the traces, not only the dashboards.

    This stretch on the route8 sources in the field guide

  6. 6Govern

    Who is accountable, and can we prove it?

    The summit is not a moment. It's being able to name the owner of every system and show what it did.

    This stretch on the route10 sources in the field guide

The valleys

Every route has places where climbs stall. These are the failure modes I'd stop before production, in the order a team meets them. Each one is on Hold on the radar, with its evidence beside it.

  1. At Design · also Operate

    LLM judges nobody calibrated

    A model grading another model's output is cheap and fast, and it can be wrong in ways that look confident. Before a judge gates anything, check it against people grading the same samples, and check again when prompts or models change.

  2. At Design · also Launch

    Benchmark scores as launch evidence

    A leaderboard result says a model did well on someone else's tasks, once. It says little about your data, your tools or repeated runs. Launch on results from your own tasks, run several times, with the failures written down.

  3. At Scale · also Operate

    Retries with no budget

    Automatic retries feel like resilience until a slow dependency turns every request into several, and the overload spreads. Retries need a budget, backoff and a stop, enforced by the platform rather than left to each agent.

  4. At Operate

    Telemetry that lives only in a vendor's console

    If the only place to see what your agents did is one vendor's dashboard, you can't join it to the rest of your traces, and you can't leave. Put it in the contract: open conventions, exported to a backend you control.

  5. At Govern · also Operate

    Capturing prompts and outputs by default

    Prompts, outputs and tool arguments often carry personal data. Capturing them makes debugging easier and turns your telemetry into a sensitive data store. Keep capture off unless a written policy turns it on for a named environment, with an owner and a retention period.

  6. At Govern · also Operate

    A person approving every step as the only safeguard

    Approval prompts feel safe, but people stop reading them long before the agent stops asking. Keep approval for the decisions that need judgement, and enforce the rest with permissions, limits and checks outside the model.

  7. At Govern · also Launch

    Blaming the model

    When an agent gets it wrong, "the model did it" is not an answer. A Canadian tribunal held an airline responsible for what its chatbot told a customer. Whoever deploys the system owns what it says and does, so name that owner before launch.

Gear check

Before each stretch, check what you carry. The radar sorts 40 techniques, platforms, measures and governance practices into four rings, and backs each placement with sources.

  1. AdoptTake on the climb

    Use by default, because the evidence is consistent and the practice holds up at production scale.

  2. TrialTest on the approach

    Use on a real workload with an owner and an exit plan; sound, but not yet settled.

  3. AssessScout ahead

    Study it and test it in isolation; it may matter, but evidence does not yet justify production use.

  4. HoldLeave behind

    Avoid, and plan to retire where present; the evidence shows it fails, misleads or hides risk. Every valley above sits in this ring.

Open the radar

Trail log

Notes from the climb, short and dated. Here is the newest from each group; the trail log keeps every entry.

Testing now6 entries

Trying it, not settled on it.

  1. Evals that start from production traces

    · Draft

    I'm treating evals the way test-driven development treats tests. Every failure I find in a production trace becomes a failing eval before anyone fixes it, and the suite gates each release module by module against a baseline. GN-0003 sets out the two waves and the loop between them.

Using more effectively5 entries

It works. I'm learning to use it better.

  1. The problem before the prompt

    · Draft

    Give a coding agent a feature request and it builds the feature. Give it the problem and who has it, and its output has something to be checked against. Without that, it fills the gap with plausible features. Naming the problem is the cheapest fix for agent output I know.

Notes3 notes

Longer notes, each pinned to the camp on the route where it matters.

  1. GN-0001 · Draft Agentic observability , original note

    October 2026 · Charted at Operate

    Agents turn one request into a tree of decisions, model calls and tool calls. The OpenTelemetry GenAI conventions give that tree a shared vocabulary. Eight requirements to adopt.

  2. GN-0002 · Draft Test triangulation , original note

    October 2026 · Charted at Design

    Example tests check only the cases an engineer thought of. Triangulate: state the properties that constrain the behaviour, including what must never happen, and let a framework try thousands of inputs to find a counterexample. Then map the value stream to see where late defects cost time.

  3. GN-0003 · Draft Eval-driven development , original note

    October 2026 · Charted at Launch

    Evals make an LLM system's limits explicit. Eval-driven development treats them as test-driven development treats tests: watch behaviour through telemetry, turn each failure into a failing eval, gate every change module by module against a baseline, and send what fails back to engineers.

Read the whole trail log · 18 entries and 3 notes

The guide

I don't carry the team up the mountain. I know the route, name the hazards early, and make sure the checks happen.

How I lead a climb

What I hold to when building agents and the systems around them, with the evidence beside each point.

  1. Start from the problem, not the model.

    A model can't fix a problem nobody wrote down. I start with who has the problem, what it costs them and how we'll know it's solved. Then every feature has to trace back to it. Agents make this matter more, because they will happily build what nobody asked for.

  2. Evaluations are the spec you can run.

    I don't trust a prompt, skill or model change I can't test. Deterministic checks run on every push. A model grading outputs earns trust only after it agrees with people grading the same ones. A benchmark score says a model managed someone else's task once, not that my system will, every time.

  3. If I can't see what an agent did, I can't own it.

    One request to an agent becomes a tree of decisions, model calls and tool calls that nobody wrote down in advance. I want that whole tree as one connected trace, in an open format, before the agent reaches production. Cost, quality and risk all come out of the same record.

  4. Put the controls outside the model.

    A prompt is not a permission. Each agent gets its own identity and only the access it needs. Limits on spend and loops live in the platform, not in the instructions. People stop reading approval prompts long before an agent stops sending them, so approval can't be the only safeguard. The team that ships it stays accountable.

  5. Coding agents move the bottleneck. They don't remove it.

    With coding agents, writing code stops being the slow part. Deciding what to build and reviewing what comes back become the work. That makes written intent, small branches and fast checks worth more than before. It's the same discipline that keeps a team spread across time zones moving: write the decision down clearly enough that nobody has to be in the room.

What I build

Problem-Based SRS

A way to write requirements that starts from the customer's problem instead of the requested feature. Paulo Cézar Stadzisz and I published it in 2016. In 2026 I turned it into an open-source agent skill for GitHub Copilot and Claude Code, so a coding agent has to trace every requirement back to a problem before it builds anything.

  1. CP.01Customer problemSales teams waste valuable time searching for customer information across multiple disconnected systems.
  2. CN.01.1Customer needCentralized customer database
  3. FR.01.1.1Functional requirementContact and company management
From the CRM example that ships with the skill. Each identifier names its parent, so the chain reads forward to what gets built and backward to why.

Conversations

I'm open to conversations with teams designing or scaling agentic and AI systems. The best way to reach me is a message on LinkedIn.