From e02f30adb68bc6dbd080d535150426ae71336585 Mon Sep 17 00:00:00 2001 From: Thomas Kosiewski Date: Thu, 24 Sep 2026 09:50:23 +0000 Subject: [PATCH 01/10] docs: tighten root README and CONTRIBUTING Change-Id: I41f9bade8b8ace222cf694ce9f0024ebc50ba8cf Signed-off-by: Thomas Kosiewski --- CONTRIBUTING.md | 65 +++++++++++++++--------------- README.md | 102 ++++++++++++++++++++++++------------------------ 2 files changed, 83 insertions(+), 84 deletions(-) diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index a32a50ff..8f2bd75a 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -1,17 +1,15 @@ # Contributing -`coder/ai-sdk` is a pnpm monorepo of three independent, independently-published -packages — `@coder/ai-sdk-sandbox`, `@coder/ai-sdk-agent`, and -`@coder/ai-sdk-provider`; see the [README](./README.md) for what each one is. -All three target Vercel AI SDK v7, and pnpm keeps their dependency trees -isolated. +A pnpm monorepo of three independently published packages, all targeting Vercel +AI SDK v7: `@coder/ai-sdk-sandbox`, `@coder/ai-sdk-agent`, and +`@coder/ai-sdk-provider` ([what each does](./README.md#packages)). -Repo-operational knowledge — review protocol, release quirks, invariants — lives -in [AGENTS.md](./AGENTS.md). +Review protocol, release quirks, and invariants live in +[AGENTS.md](./AGENTS.md). ## Setup -The toolchain — node, pnpm, and the workflow linters — is pinned in +The toolchain (Node, pnpm, workflow linters) is pinned in [`mise.toml`](./mise.toml) and locked in `mise.lock`. With [mise](https://mise.jdx.dev) installed: @@ -22,7 +20,7 @@ pnpm install # install workspace dependencies ## Commands -Everything runs from the repo root: +Run everything from the repo root: ```bash pnpm check # the CI gate: format check + lint + typecheck @@ -31,28 +29,27 @@ pnpm build # build every package pnpm format # auto-format with oxfmt ``` -Lint and format are centralized at the root via [oxc](https://oxc.rs) (`oxlint` + -`oxfmt`) and cover the whole tree at once; typecheck, test, and build fan out to -each package. To work on just one, use pnpm's `--filter`: +Lint and format ([oxc](https://oxc.rs): `oxlint` + `oxfmt`) cover the whole +tree at once. Typecheck, test, and build fan out to each package; target one +with `--filter`: ```bash pnpm --filter @coder/ai-sdk-agent test pnpm --filter @coder/ai-sdk-sandbox build ``` -GitHub Actions workflows are linted with [`actionlint`](https://github.com/rhysd/actionlint) -and audited with [`zizmor`](https://docs.zizmor.sh) (both pinned in `mise.toml`); -run them locally with `actionlint` and `zizmor .github/workflows`. +Workflows are linted with [`actionlint`](https://github.com/rhysd/actionlint) +and audited with [`zizmor`](https://docs.zizmor.sh), both pinned in +`mise.toml`. Run them locally with `actionlint` and `zizmor .github/workflows`. ### The anti-slop lint rules -On top of oxlint's built-in rules, the repo enforces -[anti-slop](https://github.com/dmmulroy/anti-slop) — opinionated rules that -reject low-evidence TypeScript patterns (unjustified type assertions, `unknown` -in signatures, ad hoc `typeof` narrowing, and similar). The plugin is -**vendored** at [`tools/oxlint/anti-slop/`](./tools/oxlint/anti-slop/) as -upstream recommends, and loaded through `jsPlugins` in -[`.oxlintrc.json`](./.oxlintrc.json). Notes: +[anti-slop](https://github.com/dmmulroy/anti-slop) adds opinionated rules that +reject low-evidence TypeScript patterns: unjustified type assertions, `unknown` +in signatures, ad hoc `typeof` narrowing, and similar. It is **vendored** at +[`tools/oxlint/anti-slop/`](./tools/oxlint/anti-slop/) (as upstream +recommends) and loaded through `jsPlugins` in +[`.oxlintrc.json`](./.oxlintrc.json). - **Updating**: copy `skills/install-anti-slop/assets/anti-slop/` from upstream over `tools/oxlint/anti-slop/` and record the upstream commit in the PR. The @@ -74,10 +71,10 @@ upstream recommends, and loaded through `jsPlugins` in ## Commits & pull requests -PRs are **squash-merged**, and the PR title becomes the commit on `main` — which is -what drives releases. So the **PR title must be a valid [Conventional Commit][cc]** -(CI enforces it). Use the package's short name as the scope, or omit the scope for -repo-wide changes: +PRs are **squash-merged**: the PR title becomes the commit on `main` and drives +releases. The **title must be a valid [Conventional Commit][cc]** (CI enforces +it). Scope it with the package's short name, or omit the scope for repo-wide +changes: ```text feat(sandbox): add port leasing @@ -87,11 +84,15 @@ ci: bump actions ## Releases -Releases are fully automated with -[release-please](https://github.com/googleapis/release-please) — no manual version -bumps or `npm publish`. It reads Conventional Commit history and opens a release PR -per package; merging that PR versions the package, tags it (`sandbox-vX.Y.Z` / -`agent-vX.Y.Z` / `provider-vX.Y.Z`), and publishes to npm with provenance. The -three packages version and release independently. +Fully automated with +[release-please](https://github.com/googleapis/release-please); never bump +versions or `npm publish` by hand. + +1. release-please reads the Conventional Commit history and opens one release + PR per package. +2. Merging that PR versions the package, tags it (`sandbox-vX.Y.Z` / + `agent-vX.Y.Z` / `provider-vX.Y.Z`), and publishes to npm with provenance. + +Each package versions and releases independently. [cc]: https://www.conventionalcommits.org diff --git a/README.md b/README.md index 784e87b0..1679da9c 100644 --- a/README.md +++ b/README.md @@ -1,42 +1,41 @@ # coder/ai-sdk -**Coder's integrations with the [Vercel AI SDK](https://ai-sdk.dev).** Run coding -agents inside Coder workspaces, and drive Coder Agents from AI SDK code. +**Coder integrations for the [Vercel AI SDK](https://ai-sdk.dev).** Run coding +agents inside Coder workspaces, drive Coder Agents from AI SDK code, and call +models through your deployment's AI Gateway. > [!NOTE] -> All three packages are pre-1.0 and track experimental upstreams (Coder's -> chat API is experimental). Expect breaking changes. +> All packages are pre-1.0. Expect breaking changes. ## Packages -Each package is published to npm independently and ships its own README with full -install instructions, usage, and API docs. +| Package | Version | Use it to… | +| ----------------------------------------------- | ----------------------------------------------------------------------------------------------------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | +| [`@coder/ai-sdk-provider`](./packages/provider) | [![npm](https://img.shields.io/npm/v/@coder/ai-sdk-provider.svg)](https://www.npmjs.com/package/@coder/ai-sdk-provider) | Call models with `generateText` / `streamText` through [AI Gateway](https://coder.com/docs/ai-coder/ai-gateway). One Coder token, no raw provider keys, per-user auth and audit. | +| [`@coder/ai-sdk-agent`](./packages/agent) | [![npm](https://img.shields.io/npm/v/@coder/ai-sdk-agent.svg)](https://www.npmjs.com/package/@coder/ai-sdk-agent) | Run **Coder Agents**, Coder's server-side agent runtime, as a real AI SDK v7 `Agent` (`generate()`, `stream()`, tool calls). | +| [`@coder/ai-sdk-sandbox`](./packages/sandbox) | [![npm](https://img.shields.io/npm/v/@coder/ai-sdk-sandbox.svg)](https://www.npmjs.com/package/@coder/ai-sdk-sandbox) | Run CLI coding agents (Claude Code, Codex) under the AI SDK v7 `HarnessAgent` in an isolated **Coder workspace**, with your tools, secrets, and network. | -| Package | Version | What it does | -| ----------------------------------------------- | ----------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | -| [`@coder/ai-sdk-sandbox`](./packages/sandbox) | [![npm](https://img.shields.io/npm/v/@coder/ai-sdk-sandbox.svg)](https://www.npmjs.com/package/@coder/ai-sdk-sandbox) | A **sandbox provider** for the Vercel AI SDK v7 `HarnessAgent`. Runs CLI coding agents — Claude Code, Codex — inside a **Coder workspace** instead of on the local machine, so each agent gets a real, isolated dev environment with your tools, secrets, and network. | -| [`@coder/ai-sdk-agent`](./packages/agent) | [![npm](https://img.shields.io/npm/v/@coder/ai-sdk-agent.svg)](https://www.npmjs.com/package/@coder/ai-sdk-agent) | A Vercel AI SDK–compliant **`Agent`** (AI SDK v7) backed by **Coder Agents**, Coder's server-side agent runtime. `new CoderAgent()` returns a real `Agent` — `generate()`, `stream()`, tool calls, the whole interface. | -| [`@coder/ai-sdk-provider`](./packages/provider) | [![npm](https://img.shields.io/npm/v/@coder/ai-sdk-provider.svg)](https://www.npmjs.com/package/@coder/ai-sdk-provider) | A **Vercel AI SDK provider** that routes `generateText` / `streamText` calls through your Coder deployment's [AI Gateway](https://coder.com/docs/ai-coder/ai-gateway). Point it at your deployment with a Coder API token and use any model it proxies — no raw provider keys, with per-user auth and audit. | +Each package ships on npm independently, with its own README. +[`@coder/ai-sdk-effect`](./packages/effect) is an unpublished, experimental +[Effect](https://effect.website) bridge ([#144](https://github.com/coder/ai-sdk/issues/144)). -Experimental: [`@coder/ai-sdk-effect`](./packages/effect) bridges these -packages to [Effect](https://effect.website) (unpublished Phase 1 spike, see -[#144](https://github.com/coder/ai-sdk/issues/144)). +### Which package? -**Which package?** Need a **model** (text, streaming, or schema‑constrained -structured output) through your deployment → `@coder/ai-sdk-provider`. Need Coder's -**server‑side agent** (multi‑step tool loop, MCP, workspace file/shell tools) → -`@coder/ai-sdk-agent`. Need to run a **CLI coding agent** (Claude Code, Codex) -inside a workspace → `@coder/ai-sdk-sandbox`. +| You need… | Use | +| ------------------------------------------------------------------------- | ------------------------ | +| A **model**: text, streaming, or schema-constrained structured output | `@coder/ai-sdk-provider` | +| Coder's **server-side agent**: multi-step tool loop, MCP, workspace tools | `@coder/ai-sdk-agent` | +| A **CLI coding agent** (Claude Code, Codex) inside a workspace | `@coder/ai-sdk-sandbox` | ## Using them together -The packages are independent, but they compose into one application against one -deployment, authenticated by one Coder token: **provision** a workspace with the -sandbox package, **drive** Coder's server-side agent in it with the agent -package, then **extract** a typed result with the provider. The full flow below -audits a project's dependencies — the agent works inside the workspace (calling -one custom tool that executes in _your_ process), and the provider parses the -prose report into a typed object through AI Gateway. +The packages are independent, but they compose against one deployment with one +Coder token. This example audits a project's dependencies: + +1. **Sandbox** provisions a workspace. +2. **Agent** works inside it and calls one custom tool that runs in _your_ + process. +3. **Provider** turns the agent's prose report into a typed object. ```bash pnpm add @coder/ai-sdk-sandbox @coder/ai-sdk-agent @coder/ai-sdk-provider ai zod @@ -54,13 +53,12 @@ import { CoderNativeTransport, ensureCoderWorkspace } from "@coder/ai-sdk-sandbo import { CoderAgent } from "@coder/ai-sdk-agent"; import { createCoder } from "@coder/ai-sdk-provider"; -// The sandbox transport and the agent both default their connection from the -// CODER_URL + CODER_SESSION_TOKEN exported above (explicit `url`/`baseUrl` and -// `token` options remain available). +// The transport and the agent read CODER_URL + CODER_SESSION_TOKEN by default +// (explicit `url`/`baseUrl` and `token` options also exist). // 1. Sandbox: get-or-create a workspace and wait until its agent is ready. -// The native transport talks straight to your deployment — no `coder` CLI -// or `ssh` on the host. Assumes the template checks out your project. +// The native transport talks straight to your deployment (no `coder` CLI +// or `ssh` on the host). Assumes the template checks out your project. const transport = new CoderNativeTransport(); const ws = await ensureCoderWorkspace({ workspace: "dep-audit", @@ -69,9 +67,8 @@ const ws = await ensureCoderWorkspace({ }); if (ws.id === undefined) throw new Error("transport did not report a workspace id"); -// 2. Agent: drive Coder's server-side agent loop in that workspace. The -// server runs the multi-step loop and its file/shell tools; your custom -// tool executes here, in this process, and its result is sent back. +// 2. Agent: the server runs the loop and its file/shell tools; your custom +// tool executes here, and its result is sent back. const agent = new CoderAgent({ organizationId: process.env.CODER_ORG_ID!, model: "claude-sonnet-4-6", @@ -124,32 +121,33 @@ console.table(audit.outdated); await transport.close(); // close cached relay WebSockets on shutdown ``` -Each step stands alone — skip the ones you don't need. For depth on each: -[`ensureCoderWorkspace` and workspace creation settings](./packages/sandbox/README.md#provisioning-a-workspace-without-a-session) -in the sandbox README; [custom tools](./packages/agent/README.md#custom-tools), -[structured output](./packages/agent/README.md#structured-output) (when the -typed answer must come out of the agent run itself, not a follow-up model call), -the [workspaces & quota operations guide](./packages/agent/README.md#workspaces--quota) -(fleet sizing, autostop, troubleshooting stuck turns), -and the [durable-workflow how-to](./packages/agent/README.md#durable-workflows-persist-resume-recover) -(persisting `chatId` across job/queue boundaries, drop & timeout recovery) -in the agent README; -[named providers and authentication modes](./packages/provider/README.md#named-providers-and-the-two-wire-protocols) -and the [enterprise governance & security reference](./packages/provider/README.md#enterprise-governance--security) -(data flow, credential isolation, audit capture, required permissions) -in the provider README. +Each step stands alone; skip the ones you don't need. + +### Go deeper + +- **Sandbox:** [`ensureCoderWorkspace` and create settings](./packages/sandbox/README.md#provisioning-a-workspace-without-a-session) +- **Agent:** [custom tools](./packages/agent/README.md#custom-tools) · + [structured output](./packages/agent/README.md#structured-output) (typed + answer from the agent run itself, no follow-up model call) +- **Agent guides:** [workspaces & quota](./packages/agent/docs/workspaces-and-quota.md) + (fleet sizing, autostop, stuck turns) · + [durable workflows](./packages/agent/docs/durable-workflows.md) (persist + `chatId` across jobs, recover from drops and timeouts) +- **Provider:** [named providers and auth modes](./packages/provider/README.md#named-providers-and-the-two-wire-protocols) · + [enterprise governance & security](./packages/provider/README.md#enterprise-governance--security) + (data flow, credential isolation, audit, permissions) ## Contributing -Development setup, the command reference, and how releases work all live in -[`CONTRIBUTING.md`](./CONTRIBUTING.md). The short version — with -[mise](https://mise.jdx.dev) installed: +With [mise](https://mise.jdx.dev) installed: ```bash -mise install && pnpm install # set up the toolchain + dependencies +mise install && pnpm install # toolchain + dependencies pnpm check && pnpm test # format check, lint, typecheck, then test ``` +Setup, commands, and releases: [`CONTRIBUTING.md`](./CONTRIBUTING.md). + ## License [Apache-2.0](./LICENSE) © Coder Technologies, Inc. From f27ff99553d7807750b809dc1074ce3fa4ce8c3c Mon Sep 17 00:00:00 2001 From: Thomas Kosiewski Date: Thu, 24 Sep 2026 09:54:37 +0000 Subject: [PATCH 02/10] docs(sandbox): tighten README and move template guide to docs Shorten the sandbox README's main reading path: tables for transports, workspace requirements, ensureCoderWorkspace settings, and lifecycle modes; transport rationale in collapsible blocks; Development trimmed to the package-specific e2e commands. Move the zero-install template authoring guide to packages/sandbox/docs/template-authoring.md, keeping a summary under the same README heading so anchors still resolve. Signed-off-by: Thomas Kosiewski --- packages/sandbox/README.md | 506 +++++++----------- packages/sandbox/docs/template-authoring.md | 170 ++++++ packages/sandbox/examples/template/Dockerfile | 2 +- 3 files changed, 358 insertions(+), 320 deletions(-) create mode 100644 packages/sandbox/docs/template-authoring.md diff --git a/packages/sandbox/README.md b/packages/sandbox/README.md index 7f972a0e..520ea432 100644 --- a/packages/sandbox/README.md +++ b/packages/sandbox/README.md @@ -1,18 +1,16 @@ # @coder/ai-sdk-sandbox -A [Coder](https://coder.com) workspace sandbox provider for the Vercel AI SDK v7 -**HarnessAgent**. It lets you run CLI coding agents inside a real Coder workspace -instead of an ephemeral cloud sandbox. Claude Code is verified end-to-end here; -Codex is expected to work via the same bridge mechanism but is not yet verified -in this repo. +Run CLI coding agents from the Vercel AI SDK v7 **HarnessAgent** inside a real +[Coder](https://coder.com) workspace instead of an ephemeral cloud sandbox. -It implements the `HarnessV1SandboxProvider` contract from `@ai-sdk/harness`, so -you pass it as the `sandbox` to a `HarnessAgent` exactly like -`@ai-sdk/sandbox-vercel`. +- Implements the `HarnessV1SandboxProvider` contract from `@ai-sdk/harness`: + pass it as the `sandbox` to a `HarnessAgent`, exactly like + `@ai-sdk/sandbox-vercel`. +- Claude Code is verified end-to-end. Codex is expected to work via the same + bridge mechanism but is not yet verified in this repo. -> **Status:** experimental. This provider tracks the stable AI SDK v7 harness -> packages (see the `@ai-sdk/harness` peer range in -> [`package.json`](./package.json)). +> **Status:** experimental. Tracks the stable AI SDK v7 harness packages (see +> the `@ai-sdk/harness` peer range in [`package.json`](./package.json)). ## Install @@ -22,12 +20,10 @@ pnpm add @coder/ai-sdk-sandbox @ai-sdk/harness @ai-sdk/harness-claude-code @ai-s Choose one host transport: -- `CoderNativeTransport` connects directly to Coderd and requires no `coder` or - `ssh` binary on the host. Pass a deployment URL and token (see - [Native transport](#native-transport)). -- The default `CoderCliTransport` requires the - [`coder` CLI](https://coder.com/docs/install), an authenticated `coder login`, - and an OpenSSH client (`ssh`) on PATH. +| Transport | Host requirements | +| ----------------------------- | -------------------------------------------------------------------------------------------------------------------- | +| `CoderCliTransport` (default) | [`coder` CLI](https://coder.com/docs/install), an authenticated `coder login`, and an OpenSSH client (`ssh`) on PATH | +| `CoderNativeTransport` | A deployment URL and token. No `coder` or `ssh` binary. See [Native transport](#native-transport) | ## Quick start @@ -56,7 +52,7 @@ try { } ``` -See [`examples/claude-code.ts`](./examples/claude-code.ts) for a runnable version. +Runnable version: [`examples/claude-code.ts`](./examples/claude-code.ts). ### Native transport @@ -76,20 +72,19 @@ const sandbox = createCoderWorkspace({ transport, }); -// When the application shuts down, close cached relay WebSockets: +// On application shutdown, close cached relay WebSockets: await transport.close(); ``` -The constructor falls back to `CODER_URL` and `CODER_SESSION_TOKEN`, so -`new CoderNativeTransport()` is sufficient when both are set. The token is sent -only to Coderd in the `Coder-Session-Token` header; it is never copied into the -workspace. +- The constructor falls back to `CODER_URL` and `CODER_SESSION_TOKEN`, so + `new CoderNativeTransport()` is enough when both are set. +- The token is sent only to Coderd in the `Coder-Session-Token` header. It is + never copied into the workspace. ## Creating workspaces on demand -Instead of pointing at an existing workspace, you can have the provider **create -one from a template** — with parameters and/or a preset — and tear it down when -the session ends. Add a `create` block: +Add a `create` block to create a workspace from a template (with parameters +and/or a preset) and tear it down when the session ends: ```ts const agent = new HarnessAgent({ @@ -106,16 +101,19 @@ const agent = new HarnessAgent({ }); ``` -By default this is **fresh-per-session**: the workspace name is derived from the -harness `sessionId` (e.g. `agent-1a2b3c4d5e6f`), so each session gets its own -workspace, and `session.destroy()` deletes it. `resumeSession` re-derives the -same name and reattaches. The provider waits for the workspace agent to finish -connecting and running its startup script (`lifecycle_state: ready`) before the -harness runs — a successful _build_ is not enough on its own. +By default this is **fresh-per-session**: -You can also **get-or-create a named workspace** by combining `workspace` with -`create`: if it exists the provider attaches to it (and never deletes it); if it -doesn't, the provider creates it (and, by default, owns it). +- The workspace name is derived from the harness `sessionId` (e.g. + `agent-1a2b3c4d5e6f`), so each session gets its own workspace. +- `session.destroy()` deletes it. `resumeSession` re-derives the same name and + reattaches. +- The harness runs only after the workspace agent finishes connecting and + running its startup script (`lifecycle_state: ready`). A successful _build_ is + not enough on its own. + +To **get-or-create a named workspace**, combine `workspace` with `create`. If +it exists, the provider attaches to it (and never deletes it). If it doesn't, +the provider creates it (and, by default, owns it). ```ts createCoderWorkspace({ @@ -124,15 +122,20 @@ createCoderWorkspace({ }); ``` -**Parameters vs. presets.** A preset's parameter values take precedence over an -overlapping `parameters` entry of the same name (this is Coder's behavior), so -set a given value via the preset _or_ `parameters`, not both. Every unset -non-ephemeral parameter must be supplied via `parameters`, `parameterFile`, or a -`preset`, unless `useParameterDefaults` accepts its template default. Parameters -marked required have no usable default and must always be supplied; otherwise -creation fails because it cannot prompt non-interactively. If you set a `preset`, -the provider preflight-validates the name against the template's presets and -fails fast with the available names (set `validate: false` to skip). +**Parameters vs. presets:** + +- A preset's parameter values take precedence over an overlapping `parameters` + entry of the same name (Coder's behavior). Set a given value via the preset + _or_ `parameters`, not both. +- Every unset non-ephemeral parameter must be supplied via `parameters`, + `parameterFile`, or a `preset`, unless `useParameterDefaults` accepts its + template default. +- Parameters marked required have no usable default and must always be + supplied; otherwise creation fails because it cannot prompt + non-interactively. +- If you set a `preset`, the provider preflight-validates the name against the + template's presets and fails fast with the available names. Set + `validate: false` to skip. ### Create settings @@ -160,20 +163,26 @@ createCoderWorkspace({ ## Provisioning a workspace without a session -`ensureCoderWorkspace(settings)` runs the same get-or-create → start-if-stopped -→ wait-until-ready pipeline as create mode, but without creating a harness -sandbox session — use it to provision a workspace for **other tools** to bind -to. It takes an explicit `workspace` name (`[owner/]workspace`; there is no -`sessionId` to derive one from), an optional `create` block (same shape as -above; `namePrefix`/`owner` are unused), plus `readyTimeoutMs`, `transport`, and -`abortSignal`. A stopped workspace is always started. Without `create`, the -workspace must already exist. - -It returns an `EnsuredCoderWorkspace`: the workspace's final (ready) status -snapshot plus `created` (whether this call created it) and — when the transport -reports one — `id`, the workspace UUID. That id is the handle other Coder -packages bind to. `@coder/ai-sdk-agent` is intentionally **not** a dependency of -this package; the two compose by a plain string handoff: +`ensureCoderWorkspace(settings)` provisions a workspace for **other tools** to +bind to. It runs the same get-or-create → start-if-stopped → wait-until-ready +pipeline as create mode, without creating a harness sandbox session. + +| Setting | Notes | +| -------------------------------------------- | ------------------------------------------------------------------------------------------------------ | +| `workspace` | Required, explicit `[owner/]workspace` name (there is no `sessionId` to derive one from). | +| `create` | Optional; same shape as above (`namePrefix`/`owner` are unused). Without it, the workspace must exist. | +| `readyTimeoutMs`, `transport`, `abortSignal` | Optional. | + +A stopped workspace is always started. The call returns an +`EnsuredCoderWorkspace`: + +- the workspace's final (ready) status snapshot; +- `created`: whether this call created it; +- `id`: the workspace UUID, when the transport reports one. Other Coder + packages bind to this handle. + +`@coder/ai-sdk-agent` is intentionally **not** a dependency of this package. +The two compose by a plain string handoff: ```ts import { ensureCoderWorkspace } from "@coder/ai-sdk-sandbox"; @@ -193,20 +202,19 @@ const agent = new CoderAgent({ ``` The non-null assertion is safe on `coder` CLIs that emit `id` in -`coder list -o json`; old CLIs omit it, so guard +`coder list -o json`. Old CLIs omit it, so guard (`if (ws.id === undefined) throw …`) when you can't pin the CLI version. ## Terminal UI -For an interactive chat in your terminal instead of one-shot `generate()` calls, -wrap the same agent with the AI SDK terminal UI ([`@ai-sdk/tui`](https://ai-sdk.dev/v7/docs/ai-sdk-harnesses/terminal-ui)): +For an interactive terminal chat instead of one-shot `generate()` calls, wrap +the agent with [`@ai-sdk/tui`](https://ai-sdk.dev/v7/docs/ai-sdk-harnesses/terminal-ui): ```bash pnpm add @ai-sdk/tui ``` -The TUI drives a session-less agent, so adapt the `HarnessAgent` (whose -`generate`/`stream` take a session) by injecting the session for the TUI's +The TUI drives a session-less agent, so inject the session for the TUI's lifetime: ```ts @@ -238,194 +246,37 @@ try { } ``` -See [`examples/claude-code-tui.ts`](./examples/claude-code-tui.ts) for a runnable -version: `CODER_WORKSPACE=my-dev-ws npx tsx examples/claude-code-tui.ts` (exit with -Esc or Ctrl+C). +Runnable version: `CODER_WORKSPACE=my-dev-ws npx tsx examples/claude-code-tui.ts` +([source](./examples/claude-code-tui.ts); exit with Esc or Ctrl+C). ## Workspace requirements -Because the bridge runs inside the workspace, the workspace image must have: - -- **Node.js ≥ 22** on the login-shell PATH. The bridge is `node bridge.mjs`, the - native transport's relay is `node -e`, and the pinned - `@anthropic-ai/claude-code` bridge dependency requires Node 22+. -- **pnpm** on the login-shell PATH (e.g. `npm install -g pnpm`) — the adapter's - bootstrap runs `pnpm install --frozen-lockfile` to install the bridge's - dependencies. -- **Outbound network access** to the npm registry (the bootstrap downloads the - bridge's dependencies — including the Claude Code CLI's ~250 MB platform - binary, which ships as an npm package — on first use) and to the model API - (`api.anthropic.com` for Claude Code, `api.openai.com` for Codex). Pre-bake - the bootstrap into the image to skip the registry entirely — see - [Template authoring](#template-authoring-zero-install-sessions). -- **The model API key** available to the bridge — `ANTHROPIC_API_KEY` / - `OPENAI_API_KEY`. Configure it through the adapter's `auth` option or ensure it - is present in the workspace environment. -- `bash` and `base64` (remote commands run through `bash -lc`; file I/O is - base64-encoded), plus `stty` for the native transport's relay bootstrap. All - standard on any Linux dev image. - -## Template authoring: zero-install sessions - -How to build a Coder template whose workspaces start harness sessions with -**zero runtime install latency**. Generic template authoring is covered by the -[Coder template docs](https://coder.com/docs/admin/templates) and -[Coder's example Docker template](https://github.com/coder/coder/tree/main/examples/templates/docker); -this section covers only the ai-sdk-specific delta. A validated -Dockerfile + `main.tf` pair lives in [`examples/template/`](./examples/template/). - -### What the first session installs - -On the first session in a fresh workspace, the Claude Code adapter applies its -bootstrap recipe relative to the session's default working directory (`$HOME` -for this provider): - -1. It writes `package.json`, `pnpm-lock.yaml`, and `bridge.mjs` (assets shipped - inside `@ai-sdk/harness-claude-code`) to `~/.harness-bootstrap/claude-code/`. -2. It runs `pnpm install --frozen-lockfile --store-dir .pnpm-store` there. This - is the expensive step: ~300 MB from the npm registry — the bridge - dependencies include the `@anthropic-ai/claude-code` platform binary — taking - tens of seconds to minutes depending on the network. -3. It runs the CLI's `install.cjs` (links the platform binary npm delivered; no - extra download) and `./node_modules/.bin/claude --version`. -4. It writes a `.bootstrap-.ok` marker next to them. Later sessions - see the marker and skip all of the above — a single file read. - -Zero-install means baking the results of steps 1–3 into the image **at that -exact path**. The first session in each workspace then replays the recipe as a -fast offline no-op (~4 s measured; pnpm prints `Already up to date` and -downloads nothing) and writes the marker; every session after that is a single -file read. - -### Step 1 — Dockerfile: pre-bake the bootstrap - -[`examples/template/Dockerfile`](./examples/template/Dockerfile) installs the -[workspace requirements](#workspace-requirements) (Node 24 + pnpm on PATH, -bash/coreutils) and replays the recipe at build time — same files, same -commands, same absolute path: - -```dockerfile -ARG HARNESS_CLAUDE_CODE_VERSION=1.0.90 -RUN mkdir -p /home/${USER}/.harness-bootstrap/claude-code \ - && cd /tmp \ - && npm pack @ai-sdk/harness-claude-code@${HARNESS_CLAUDE_CODE_VERSION} \ - && tar -xzf ai-sdk-harness-claude-code-${HARNESS_CLAUDE_CODE_VERSION}.tgz \ - && cp package/dist/bridge/package.json package/dist/bridge/pnpm-lock.yaml \ - /home/${USER}/.harness-bootstrap/claude-code/ \ - && cp package/dist/bridge/index.mjs \ - /home/${USER}/.harness-bootstrap/claude-code/bridge.mjs \ - && rm -rf package ai-sdk-harness-claude-code-${HARNESS_CLAUDE_CODE_VERSION}.tgz \ - && cd /home/${USER}/.harness-bootstrap/claude-code \ - && pnpm install --frozen-lockfile --store-dir .pnpm-store \ - && node node_modules/@anthropic-ai/claude-code/install.cjs \ - && ./node_modules/.bin/claude --version -``` - -Rules that make or break the pre-bake: - -- **Bake at the final absolute path.** pnpm records the store location as an - absolute path (in `node_modules/.modules.yaml`), so a cache baked at, say, - `/opt/...` and copied into `$HOME` later makes the session's `pnpm install` - abort (`ERR_PNPM_ABORTED_REMOVE_MODULES_DIR_NO_TTY` — pnpm wants to purge and - reinstall, and a session has no TTY to confirm). -- **Pin the adapter version your application resolves** (find it with - `pnpm why @ai-sdk/harness-claude-code`, or in your lockfile) and pass it as - `--build-arg HARNESS_CLAUDE_CODE_VERSION=...`. The recipe and its marker hash - derive from the adapter's shipped assets. A mismatch is not fatal — the - recipe re-runs in the same directory and downloads only the delta — but it - reintroduces registry traffic on first sessions. -- **Build for the workspace CPU architecture** — the pre-bake fetches the - builder platform's `claude` binary. - -### Step 2 — main.tf: get the cache into the home volume - -[`examples/template/main.tf`](./examples/template/main.tf) is Coder's example -Docker template minus the IDE modules, with three ai-sdk-specific choices: - -- **The workspace image is the pre-baked one** (`variable "image"`); push it to - a registry your Coder provisioners can pull from. -- **`startup_script_behavior = "blocking"`** on `coder_agent`: this provider's - create mode waits for `lifecycle_state: ready` before the harness runs, and - blocking makes ready mean "startup script finished" (the attribute defaults to - `non-blocking`, in which case ready does not wait for the script). -- **The home volume seeds itself from the image.** Docker populates a fresh - (empty) named volume with the image's content at the mount path, so - `/home/coder/.harness-bootstrap` lands in every new workspace's volume with no - startup-script copying. No port configuration is needed either: the provider - forwards the bridge port itself (OpenSSH `-L` or the native relay), not - through template port shares. - -For volume types that do _not_ copy image content (a Kubernetes PVC mounted at -`/home/coder` shadows it with an empty filesystem), stash the bake outside the -mount and restore it — at the same absolute path — on first start: - -```dockerfile -# Dockerfile, after the pre-bake step: -RUN sudo cp -a /home/${USER}/.harness-bootstrap /opt/harness-bootstrap-stash -``` - -```hcl - startup_script = <<-EOT - set -e - if [ ! -d ~/.harness-bootstrap ]; then - cp -a /opt/harness-bootstrap-stash ~/.harness-bootstrap - fi - EOT -``` - -(`cp -a` preserves the hardlinks between the pnpm store and `node_modules`, so -the stash and the restore each stay ~300 MB rather than doubling.) - -Build, push, and point the provider at the template -([creating templates](https://coder.com/docs/admin/templates/creating-templates)): - -```bash -cd examples/template -docker build -t /ai-sdk-sandbox-workspace:v1 . -docker push /ai-sdk-sandbox-workspace:v1 -coder templates push ai-sdk-sandbox --variable image=/ai-sdk-sandbox-workspace:v1 -``` - -```ts -createCoderWorkspace({ create: { template: "ai-sdk-sandbox" } }); -``` - -### Sizing - -Measured on the example image (linux/amd64): ~730 MB total — ~430 MB for -Ubuntu 24.04 + Node 24 + pnpm, ~300 MB for the bootstrap cache (dominated by -the pnpm store; `node_modules` hardlinks into it). Each workspace's home volume -receives the ~300 MB seeded cache; leave headroom for per-session work -directories (`claude-code-/`) and whatever your agents check out. +The bridge runs inside the workspace, so the workspace image needs: -### Verifying zero installs +| Requirement | Why | +| -------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | +| **Node.js ≥ 22** on the login-shell PATH | The bridge is `node bridge.mjs`, the native transport's relay is `node -e`, and the pinned `@anthropic-ai/claude-code` bridge dependency requires Node 22+. | +| **pnpm** on the login-shell PATH (e.g. `npm install -g pnpm`) | The adapter's bootstrap runs `pnpm install --frozen-lockfile` to install the bridge's dependencies. | +| Outbound access to the **npm registry** | On first use, the bootstrap downloads the bridge's dependencies, including the Claude Code CLI's ~250 MB platform binary (shipped as an npm package). Pre-bake to skip the registry entirely: see [Template authoring](#template-authoring-zero-install-sessions). | +| Outbound access to the **model API** | `api.anthropic.com` for Claude Code, `api.openai.com` for Codex. | +| The **model API key** (`ANTHROPIC_API_KEY` / `OPENAI_API_KEY`) | Available to the bridge: configure it through the adapter's `auth` option or put it in the workspace environment. | +| `bash` and `base64` | Remote commands run through `bash -lc`; file I/O is base64-encoded. | +| `stty` | The native transport's relay bootstrap. | -**Image-level (no Coder deployment needed).** Replay the bootstrap on a fresh -volume with networking disabled — it must succeed offline: +`bash`, `base64`, and `stty` are standard on any Linux dev image. -```bash -docker volume rm -f probe-home && docker volume create probe-home -docker run --rm --network none -v probe-home:/home/coder \ - bash -lc 'cd ~/.harness-bootstrap/claude-code \ - && pnpm install --frozen-lockfile --store-dir .pnpm-store \ - && node node_modules/@anthropic-ai/claude-code/install.cjs \ - && ./node_modules/.bin/claude --version' -``` - -Expect `Already up to date` from pnpm and the CLI version banner in a few -seconds — proof that a session's bootstrap needs no registry access. +## Template authoring: zero-install sessions -**Session-level.** After the first session against a workspace from the -template, `~/.harness-bootstrap/claude-code/` must contain a `.bootstrap-*.ok` -marker (later sessions skip the recipe entirely). If session creation instead -stalls for minutes with pnpm download progress in the bootstrap, the image's -pinned adapter version doesn't match the application's (see the pinning rule -above). +Pre-bake the adapter's bootstrap into the workspace image so sessions start +with zero runtime install latency. The +[template authoring guide](./docs/template-authoring.md) covers the Dockerfile, +`main.tf`, sizing, and verification. A validated pair lives in +[`examples/template/`](./examples/template/). ## Settings -**At least one of `workspace` or `create` is required** (you may set both — see -[Creating workspaces on demand](#creating-workspaces-on-demand)). +**At least one of `workspace` or `create` is required.** You may set both; see +[Creating workspaces on demand](#creating-workspaces-on-demand). ```ts import { @@ -445,9 +296,8 @@ createCoderWorkspace({ ownsLifecycle: false, // see "Lifecycle modes" below ensureStarted: false, // run `coder start` before attaching - // Transport. Defaults to an ambient-login CoderCliTransport. Configure the CLI - // transport (binary paths, url/token, env, login shell, wait mode) — or supply - // a non-CLI/test transport — by constructing one explicitly: + // Transport. Defaults to an ambient-login CoderCliTransport. Construct one to + // configure it, or supply a non-CLI/test transport: transport: new CoderCliTransport({ // coderBinary: 'coder', sshBinary: 'ssh', // url: process.env.CODER_URL, token: process.env.CODER_SESSION_TOKEN, @@ -464,33 +314,33 @@ createCoderWorkspace({ ### Lifecycle modes -- **Wrap an existing workspace (default when there's no `create`, - `ownsLifecycle: false`).** `stop()` and `destroy()` only release host-side - resources (port-forwards); the workspace keeps running. The natural fit for - long-lived dev workspaces. -- **Own the workspace (`ownsLifecycle: true`).** `stop()` runs `coder stop` and - `destroy()` runs `coder delete`. -- **Create mode (`create` set).** `ownsLifecycle` defaults to `true`, so a - workspace the provider creates is deleted on `destroy()` and `onFirstCreate` - runs as its bootstrap hook. As a safety measure, a workspace the provider only - _attached_ to (an explicitly-named, pre-existing one) is **never** deleted — - only ones it actually created. A per-session derived name is always treated as - owned. Set `ownsLifecycle: false` for "create-if-missing but never delete". +| Mode | Selected by | Behavior | +| -------------------------- | ------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------ | +| Wrap an existing workspace | Default without `create` (`ownsLifecycle: false`) | `stop()` and `destroy()` only release host-side resources (port-forwards); the workspace keeps running. The natural fit for long-lived dev workspaces. | +| Own the workspace | `ownsLifecycle: true` | `stop()` runs `coder stop`; `destroy()` runs `coder delete`. | +| Create mode | `create` set | `ownsLifecycle` defaults to `true`: a workspace the provider creates is deleted on `destroy()`, and `onFirstCreate` runs as its bootstrap hook. | + +Create-mode safety rules: + +- A workspace the provider only _attached_ to (an explicitly-named, + pre-existing one) is **never** deleted; only ones it actually created. +- A per-session derived name is always treated as owned. +- Set `ownsLifecycle: false` for "create-if-missing but never delete". ### Ports -The adapter binds its bridge to a port and resolves it from -`createClaudeCode({ port })` or, by default, `sandbox.ports[0]`. Expose that port -via `ports` (default `[4000]`); `getPortUrl` asks the configured transport for a -local TCP forward and returns a loopback `ws://` URL. The CLI transport uses -OpenSSH `-L`; the native transport multiplexes TCP over its Coderd WebSocket. -The forward is plaintext on loopback, so `https`/`wss` requests resolve to their -`http`/`ws` loopback equivalent. +- The adapter binds its bridge to the port from `createClaudeCode({ port })` + or, by default, `sandbox.ports[0]`. Expose it via `ports` (default `[4000]`). +- `getPortUrl` asks the configured transport for a local TCP forward and + returns a loopback `ws://` URL. The CLI transport uses OpenSSH `-L`; the + native transport multiplexes TCP over its Coderd WebSocket. +- The forward is plaintext on loopback, so `https`/`wss` requests resolve to + their `http`/`ws` loopback equivalent. ## How it works -A `HarnessAgent` doesn't run the agent CLI directly. For bridge-backed adapters -(Claude Code, Codex) it installs a small Node "bridge" program **inside the +For bridge-backed adapters (Claude Code, Codex), a `HarnessAgent` doesn't run +the agent CLI directly. It installs a small Node "bridge" program **inside the sandbox**, spawns it, and talks to it over an **authenticated WebSocket**. The bridge runs the vendor SDK in-workspace and streams events back to the host. @@ -504,37 +354,59 @@ This provider maps that contract onto Coder primitives: | `createSession` / `resumeSession` / `id` | CLI workspace lookup | Coderd v2 REST API | | `stop` / `destroy` | `coder stop` / `coder delete` when lifecycle-owned | Coderd workspace-build transitions | -**Why OpenSSH and not `coder ssh -- cmd`?** `coder ssh` allocates a PTY for -the command, which rewrites newlines to CRLF, merges stdout and stderr onto one -stream, and does not reliably propagate exit codes — all fatal for programmatic -use and for the bridge's stdout parsing. `coder ssh`'s own help recommends -`coder config-ssh` "for users who need the full functionality of SSH"; this -provider does the programmatic equivalent, running real OpenSSH over a -`coder ssh --stdio` ProxyCommand. That yields clean, separated streams and -correct exit codes (verified against a live workspace). - -**How the native relay stays byte-clean.** Coderd's browser-terminal endpoint -is a PTY, which by itself merges stdout/stderr and has no process exit-code -channel. The native transport uses it only as a carrier: it bootstraps a small, -dependency-free Node relay, switches the PTY to raw/no-echo mode, and exchanges -versioned newline-delimited frames with base64 byte payloads. The relay launches -commands with separate pipes and also opens TCP sockets for `getPortUrl`. It -does not bind a workspace port or persist credentials/files; one relay is cached -per selected workspace agent and `transport.close()` tears it down. - -The WebSocket the harness opens against `getPortUrl(...)` is the critical path, -and it needs no wildcard access URLs — the host running `HarnessAgent` is already -a Coder client. We forward via OpenSSH `-L` rather than `coder port-forward`: -the bridge sends an _unprompted_ `bridge-hello` frame immediately after the WS -upgrade, and in testing a freshly-created `coder port-forward` tunnel did not +The bridge WebSocket needs no wildcard access URLs: the host running +`HarnessAgent` is already a Coder client. + +
+Why OpenSSH instead of coder ssh <ws> -- cmd + +`coder ssh` allocates a PTY for the command. The PTY rewrites newlines to CRLF, +merges stdout and stderr onto one stream, and does not reliably propagate exit +codes: all fatal for programmatic use and for the bridge's stdout parsing. + +`coder ssh`'s own help recommends `coder config-ssh` "for users who need the +full functionality of SSH". This provider does the programmatic equivalent: it +runs real OpenSSH over a `coder ssh --stdio` ProxyCommand. That yields clean, +separated streams and correct exit codes (verified against a live workspace). + +
+ +
+How the native relay stays byte-clean + +Coderd's browser-terminal endpoint is a PTY, which by itself merges +stdout/stderr and has no process exit-code channel. The native transport uses +it only as a carrier: + +1. It bootstraps a small, dependency-free Node relay. +2. It switches the PTY to raw/no-echo mode. +3. It exchanges versioned newline-delimited frames with base64 byte payloads. + +The relay launches commands with separate pipes and also opens TCP sockets for +`getPortUrl`. It does not bind a workspace port or persist credentials/files. +One relay is cached per selected workspace agent; `transport.close()` tears it +down. + +
+ +
+Why OpenSSH -L instead of coder port-forward + +The WebSocket the harness opens against `getPortUrl(...)` is the critical path. +The bridge sends an _unprompted_ `bridge-hello` frame immediately after the WS +upgrade. In testing, a freshly-created `coder port-forward` tunnel did not reliably deliver that first server-initiated frame to the first WS client, -whereas SSH local forwarding does. This path is verified end-to-end against a -real workspace — both a synthetic WebSocket round-trip (`scripts/verify-real.ts`) -and a full Claude Code turn with tool use (`scripts/e2e-claude.ts`). +whereas SSH local forwarding does. + +This path is verified end-to-end against a real workspace: both a synthetic +WebSocket round-trip (`scripts/verify-real.ts`) and a full Claude Code turn +with tool use (`scripts/e2e-claude.ts`). + +
## Limitations & notes -- `setNetworkPolicy` is not implemented (omitted) — egress is governed by your +- `setNetworkPolicy` is not implemented (omitted). Egress is governed by your Coder template/deployment, not this provider. - File reads buffer the whole file (binary content moves as base64). Fine for bootstrap-sized files; not intended for streaming very large files. @@ -543,48 +415,44 @@ and a full Claude Code turn with tool use (`scripts/e2e-claude.ts`). `relayNodeCommand` when Node lives at a fixed nonstandard path. - A workspace with multiple agents must be selected as `workspace.agent`; the native transport refuses to guess. -- `@ai-sdk/sandbox-just-bash` cannot expose ports and is rejected by bridge-backed - adapters — this provider exists precisely to provide that port. -- To run Claude Code / Codex, the **workspace** image needs Node.js ≥ 22 (the - adapter installs the bridge + CLI on first use) and egress to the npm registry - and the model API — unless the image pre-bakes the bootstrap (see - [Template authoring](#template-authoring-zero-install-sessions)). Provide the - API key via the adapter's `auth` option. +- `@ai-sdk/sandbox-just-bash` cannot expose ports and is rejected by + bridge-backed adapters. This provider exists precisely to provide that port. +- To run Claude Code / Codex, the workspace image must meet the + [workspace requirements](#workspace-requirements). ## Development -```bash -# From this package's directory (packages/sandbox): -pnpm install -pnpm typecheck # tsc against the real harness types -pnpm test # vitest: unit + local integration (fake `coder` + `ssh`) -pnpm build # tsup → dist/ (ESM + d.ts) +Setup and repo-wide gates are in [CONTRIBUTING.md](../../CONTRIBUTING.md). +Package-specific commands, run from `packages/sandbox`: -# Formatting & linting are root-level scripts (`-w` runs them from anywhere in the repo): -pnpm -w format # oxfmt (apply formatting) -pnpm -w lint # oxlint (report lint issues) -pnpm -w check # format check + lint + typecheck (CI gate) +```bash +# Unit + local integration tests against fake `coder`/`ssh` executables (no deployment): +pnpm test # End-to-end against a real workspace (needs the coder CLI + a running workspace): pnpm verify:real my-ws -# The same contract through Coderd directly. The CLI is used only to mint a -# token for this shell; CoderNativeTransport never invokes it: +# The same contract through Coderd directly. The CLI only mints a token for +# this shell; CoderNativeTransport never invokes it: CODER_URL=https://coder.example.com \ CODER_SESSION_TOKEN="$(coder tokens create --name ai-sdk-sandbox)" \ pnpm verify:native my-ws -# End-to-end of create mode (creates a throwaway workspace, then deletes it): +# Create mode: creates a throwaway workspace from a template, waits for +# readiness, runs a command in it, and deletes it: pnpm verify:create docker ``` -The local integration tests exercise the real transport (argument building, -stdin, base64 file round-trips, streaming, the port-forward lifecycle, and the -create/status/presets JSON paths) against fake `coder`/`ssh` executables that run -commands locally — no Coder deployment needed. `scripts/verify-real.ts` runs the -same surface — plus a real WebSocket-over-SSH round-trip — against an actual -workspace; `scripts/verify-create.ts` creates a throwaway workspace from a -template, waits for readiness, runs a command in it, and deletes it. +
+What the local integration tests cover + +They exercise the real transport (argument building, stdin, base64 file +round-trips, streaming, the port-forward lifecycle, and the +create/status/presets JSON paths) against fake `coder`/`ssh` executables that +run commands locally. `scripts/verify-real.ts` runs the same surface, plus a +real WebSocket-over-SSH round-trip, against an actual workspace. + +
## License diff --git a/packages/sandbox/docs/template-authoring.md b/packages/sandbox/docs/template-authoring.md new file mode 100644 index 00000000..7ded2a44 --- /dev/null +++ b/packages/sandbox/docs/template-authoring.md @@ -0,0 +1,170 @@ +# Template authoring: zero-install sessions + +Build a Coder template whose workspaces start harness sessions with **zero +runtime install latency**. A validated Dockerfile + `main.tf` pair lives in +[`examples/template/`](../examples/template/). + +This guide covers only the ai-sdk-specific delta. For generic template +authoring, see the [Coder template docs](https://coder.com/docs/admin/templates) +and [Coder's example Docker template](https://github.com/coder/coder/tree/main/examples/templates/docker). + +## What the first session installs + +On the first session in a fresh workspace, the Claude Code adapter applies its +bootstrap recipe relative to the session's default working directory (`$HOME` +for this provider): + +1. Writes `package.json`, `pnpm-lock.yaml`, and `bridge.mjs` (assets shipped + inside `@ai-sdk/harness-claude-code`) to `~/.harness-bootstrap/claude-code/`. +2. Runs `pnpm install --frozen-lockfile --store-dir .pnpm-store` there. This is + the expensive step: ~300 MB from the npm registry (the bridge dependencies + include the `@anthropic-ai/claude-code` platform binary), taking tens of + seconds to minutes depending on the network. +3. Runs the CLI's `install.cjs` (links the platform binary npm delivered; no + extra download) and `./node_modules/.bin/claude --version`. +4. Writes a `.bootstrap-.ok` marker next to them. Later sessions + see the marker and skip all of the above: a single file read. + +Zero-install means baking the results of steps 1–3 into the image **at that +exact path**. The first session in each workspace then replays the recipe as a +fast offline no-op (~4 s measured; pnpm prints `Already up to date` and +downloads nothing) and writes the marker. Every later session is a single file +read. + +## Step 1 — Dockerfile: pre-bake the bootstrap + +[`examples/template/Dockerfile`](../examples/template/Dockerfile) installs the +[workspace requirements](../README.md#workspace-requirements) (Node 24 + pnpm on +PATH, bash/coreutils) and replays the recipe at build time: same files, same +commands, same absolute path. + +```dockerfile +ARG HARNESS_CLAUDE_CODE_VERSION=1.0.90 +RUN mkdir -p /home/${USER}/.harness-bootstrap/claude-code \ + && cd /tmp \ + && npm pack @ai-sdk/harness-claude-code@${HARNESS_CLAUDE_CODE_VERSION} \ + && tar -xzf ai-sdk-harness-claude-code-${HARNESS_CLAUDE_CODE_VERSION}.tgz \ + && cp package/dist/bridge/package.json package/dist/bridge/pnpm-lock.yaml \ + /home/${USER}/.harness-bootstrap/claude-code/ \ + && cp package/dist/bridge/index.mjs \ + /home/${USER}/.harness-bootstrap/claude-code/bridge.mjs \ + && rm -rf package ai-sdk-harness-claude-code-${HARNESS_CLAUDE_CODE_VERSION}.tgz \ + && cd /home/${USER}/.harness-bootstrap/claude-code \ + && pnpm install --frozen-lockfile --store-dir .pnpm-store \ + && node node_modules/@anthropic-ai/claude-code/install.cjs \ + && ./node_modules/.bin/claude --version +``` + +Rules that make or break the pre-bake: + +- **Bake at the final absolute path.** pnpm records the store location as an + absolute path (in `node_modules/.modules.yaml`). A cache baked at, say, + `/opt/...` and copied into `$HOME` later makes the session's `pnpm install` + abort with `ERR_PNPM_ABORTED_REMOVE_MODULES_DIR_NO_TTY`: pnpm wants to purge + and reinstall, and a session has no TTY to confirm. +- **Pin the adapter version your application resolves.** Find it with + `pnpm why @ai-sdk/harness-claude-code` (or in your lockfile) and pass it as + `--build-arg HARNESS_CLAUDE_CODE_VERSION=...`. The recipe and its marker hash + derive from the adapter's shipped assets. A mismatch is not fatal (the recipe + re-runs in the same directory and downloads only the delta), but it + reintroduces registry traffic on first sessions. +- **Build for the workspace CPU architecture.** The pre-bake fetches the builder + platform's `claude` binary. + +## Step 2 — main.tf: get the cache into the home volume + +[`examples/template/main.tf`](../examples/template/main.tf) is Coder's example +Docker template minus the IDE modules, with three ai-sdk-specific choices: + +- **The workspace image is the pre-baked one** (`variable "image"`). Push it to + a registry your Coder provisioners can pull from. +- **`startup_script_behavior = "blocking"`** on `coder_agent`. This provider's + create mode waits for `lifecycle_state: ready` before the harness runs, and + blocking makes ready mean "startup script finished". The attribute defaults to + `non-blocking`, in which case ready does not wait for the script. +- **The home volume seeds itself from the image.** Docker populates a fresh + (empty) named volume with the image's content at the mount path, so + `/home/coder/.harness-bootstrap` lands in every new workspace's volume with no + startup-script copying. + +No port configuration is needed: the provider forwards the bridge port itself +(OpenSSH `-L` or the native relay), not through template port shares. + +
+Volumes that don't copy image content (e.g. a Kubernetes PVC) + +A Kubernetes PVC mounted at `/home/coder` shadows it with an empty filesystem. +Stash the bake outside the mount and restore it, at the same absolute path, on +first start: + +```dockerfile +# Dockerfile, after the pre-bake step: +RUN sudo cp -a /home/${USER}/.harness-bootstrap /opt/harness-bootstrap-stash +``` + +```hcl + startup_script = <<-EOT + set -e + if [ ! -d ~/.harness-bootstrap ]; then + cp -a /opt/harness-bootstrap-stash ~/.harness-bootstrap + fi + EOT +``` + +`cp -a` preserves the hardlinks between the pnpm store and `node_modules`, so +the stash and the restore each stay ~300 MB rather than doubling. + +
+ +Build, push, and point the provider at the template +([creating templates](https://coder.com/docs/admin/templates/creating-templates)): + +```bash +cd examples/template +docker build -t /ai-sdk-sandbox-workspace:v1 . +docker push /ai-sdk-sandbox-workspace:v1 +coder templates push ai-sdk-sandbox --variable image=/ai-sdk-sandbox-workspace:v1 +``` + +```ts +createCoderWorkspace({ create: { template: "ai-sdk-sandbox" } }); +``` + +## Sizing + +Measured on the example image (linux/amd64): + +| Component | Size | +| ----------------------------- | ------- | +| Ubuntu 24.04 + Node 24 + pnpm | ~430 MB | +| Bootstrap cache | ~300 MB | +| **Total image** | ~730 MB | + +The cache is dominated by the pnpm store; `node_modules` hardlinks into it. +Each workspace's home volume receives the ~300 MB seeded cache. Leave headroom +for per-session work directories (`claude-code-/`) and whatever your +agents check out. + +## Verifying zero installs + +**Image-level (no Coder deployment needed).** Replay the bootstrap on a fresh +volume with networking disabled. It must succeed offline: + +```bash +docker volume rm -f probe-home && docker volume create probe-home +docker run --rm --network none -v probe-home:/home/coder \ + bash -lc 'cd ~/.harness-bootstrap/claude-code \ + && pnpm install --frozen-lockfile --store-dir .pnpm-store \ + && node node_modules/@anthropic-ai/claude-code/install.cjs \ + && ./node_modules/.bin/claude --version' +``` + +Expect `Already up to date` from pnpm and the CLI version banner within a few +seconds. That proves a session's bootstrap needs no registry access. + +**Session-level.** After the first session against a workspace from the +template, `~/.harness-bootstrap/claude-code/` must contain a `.bootstrap-*.ok` +marker (later sessions skip the recipe entirely). If session creation instead +stalls for minutes with pnpm download progress in the bootstrap, the image's +pinned adapter version doesn't match the application's (see the pinning rule +above). diff --git a/packages/sandbox/examples/template/Dockerfile b/packages/sandbox/examples/template/Dockerfile index 8277432c..6fe06cb2 100644 --- a/packages/sandbox/examples/template/Dockerfile +++ b/packages/sandbox/examples/template/Dockerfile @@ -1,5 +1,5 @@ # Workspace image for @coder/ai-sdk-sandbox sessions with zero runtime install -# latency. See "Template authoring" in the package README for the full guide. +# latency. See docs/template-authoring.md in the package for the full guide. # # The pre-bake step below mirrors the Claude Code adapter's bootstrap recipe # (@ai-sdk/harness-claude-code ships the bridge's package.json, pnpm-lock.yaml, From 06bafd05d08d0bba362ca4f5d04a760e33ee2cc0 Mon Sep 17 00:00:00 2001 From: Thomas Kosiewski Date: Thu, 24 Sep 2026 09:57:30 +0000 Subject: [PATCH 03/10] docs(provider): tighten README and move security reference to docs/security.md Change-Id: I7e13f8a135436171e10959f2a8e189e4b7c52fbb Signed-off-by: Thomas Kosiewski --- packages/provider/README.md | 311 ++++++++++----------------- packages/provider/docs/security.md | 159 ++++++++++++++ packages/provider/examples/README.md | 12 +- 3 files changed, 276 insertions(+), 206 deletions(-) create mode 100644 packages/provider/docs/security.md diff --git a/packages/provider/README.md b/packages/provider/README.md index 850e733e..b42c708b 100644 --- a/packages/provider/README.md +++ b/packages/provider/README.md @@ -4,12 +4,11 @@ [![npm](https://img.shields.io/npm/v/@coder/ai-sdk-provider.svg)](https://www.npmjs.com/package/@coder/ai-sdk-provider) [![License](https://img.shields.io/badge/license-Apache--2.0-blue.svg)](./LICENSE) -A **Vercel AI SDK provider that routes requests through your Coder deployment's -[AI Gateway](https://coder.com/docs/ai-coder/ai-gateway)** (formerly "AI Bridge"). Point it -at your deployment URL, hand it a Coder API token, and use any model your -deployment proxies with the AI SDK's `generateText` / `streamText` — the same way -you'd use [OpenRouter](https://ai-sdk.dev/providers/community-providers/openrouter) -or any other provider. +A Vercel AI SDK provider that routes requests through your Coder deployment's +[AI Gateway](https://coder.com/docs/ai-coder/ai-gateway) (formerly "AI Bridge"). +Use any model your deployment proxies, the same way you'd use +[OpenRouter](https://ai-sdk.dev/providers/community-providers/openrouter) or any +other provider. ```ts import { generateText } from "ai"; @@ -26,24 +25,18 @@ const { text } = await generateText({ }); ``` -## Why +**Why:** developers authenticate with their Coder token and never handle raw +provider keys. AI Gateway authenticates each request against a Coder identity, +injects the centrally managed keys for the upstream providers (Anthropic, +OpenAI, Bedrock, Copilot, …), and audits usage per user. The deployment decides +which models and providers are available. -**AI Gateway** is Coder's LLM gateway: it sits between your AI tooling and the -upstream providers (Anthropic, OpenAI, Bedrock, Copilot, …), authenticates each -request against a Coder identity, injects the centrally-managed provider keys, and -audits usage per user. This package lets the Vercel AI SDK speak to it natively, so -your developers never handle raw provider keys — they authenticate with their Coder -token and the deployment decides which models and providers are available. - -## Provider vs. Agent - -This package is for **plain model calls** — `generateText`, `streamText`, and -`generateObject` (schema‑constrained structured output). If you need Coder's -**server‑side agent** — the multi‑step tool loop, built‑in tools, MCP servers, or -workspace file/shell tools — use **[`@coder/ai-sdk-agent`](../agent)** instead. -Rule of thumb: **need a model → provider; need server‑side tools, MCP, or a -workspace → Agent.** They compose: use the provider for pure text/JSON steps and -the Agent for the tool‑driven ones. +> [!TIP] +> This package is for **plain model calls**: `generateText`, `streamText`, and +> `generateObject` (schema-constrained structured output). For Coder's +> **server-side agent** (multi-step tool loop, built-in tools, MCP servers, +> workspace file/shell tools), use [`@coder/ai-sdk-agent`](../agent). They +> compose: provider for pure text/JSON steps, Agent for tool-driven ones. ## Install @@ -51,28 +44,31 @@ the Agent for the tool‑driven ones. pnpm add @coder/ai-sdk-provider ai zod ``` -Requires Node ≥ 22, `ai` v7, and a Coder deployment with **AI Gateway enabled** -(stable since Coder **v2.29**, GA in v2.30, on by default in v2.34; requires the -AI Governance Add-On). +Requirements: + +- Node ≥ 22 and `ai` v7. +- A Coder deployment with **AI Gateway enabled**: stable since Coder **v2.29**, + GA in v2.30, on by default in v2.34. Requires the AI Governance Add-On. ## Named providers and the two wire protocols -Your Coder admins define AI Gateway **providers** — named routes on the -deployment (`/api/v2/aibridge//v1/…`), each speaking one of **two wire -protocols** determined by its admin-configured type: +Coder admins define AI Gateway **providers**: named routes on the deployment +(`/api/v2/aibridge//v1/…`). Each speaks one of two wire protocols, +set by its admin-configured type: | Wire protocol | Provider types behind it | Default name | | ------------------------ | ------------------------------------------------------------------------------- | ------------ | | **OpenAI-compatible** | `openai`, `azure`, `google`, `copilot`, `openai-compat`, `openrouter`, `vercel` | `openai` | | **Anthropic-compatible** | `anthropic` (native Claude), `bedrock` (Bedrock-hosted Claude) | `anthropic` | -Routing is **by URL path, not by model id** — the provider name in the URL -decides which upstream handles the request. `createCoder` fronts the default -`openai` / `anthropic` pair: the bare call `coder(modelId)` picks between them -by heuristic (model ids starting with `claude`/`anthropic` go to the -Anthropic-protocol provider, everything else to the OpenAI-protocol one), and -the explicit accessors override it (e.g. to reach Claude through a -Copilot-typed provider on the OpenAI protocol): +Routing is **by URL path, not by model id**: the provider name in the URL +decides which upstream handles the request. + +`createCoder` fronts the default `openai` / `anthropic` pair. The bare call +`coder(modelId)` picks one by heuristic: ids starting with `claude` or +`anthropic` go to the Anthropic-protocol provider, everything else to the +OpenAI-protocol one. Explicit accessors override the heuristic, e.g. to reach +Claude through a Copilot-typed provider on the OpenAI protocol: ```ts coder("gpt-4o"); // → `openai` provider @@ -81,16 +77,20 @@ coder.openai("claude-sonnet-4"); // → `openai` provider (e.g. Copilot) coder.anthropic("claude-opus-4-5"); // → `anthropic` provider (explicit) ``` +Model ids pass through **unchanged** to the upstream provider (no +`vendor/model` namespacing). Use whatever ids your deployment's providers +accept. + ### Custom-named providers -Provider names are admin-chosen (matching `^[a-z0-9]+(-[a-z0-9]+)*$`), so a -deployment may expose e.g. an Azure-backed `azure-openai` next to a -Bedrock-backed `anthropic-bedrock`. Reach them in two ways: +Provider names are admin-chosen and match `^[a-z0-9]+(-[a-z0-9]+)*$`. A +deployment may expose, say, an Azure-backed `azure-openai` next to a +Bedrock-backed `anthropic-bedrock`. There are two ways to reach them. -**Sub-provider accessors** — `openaiProvider(name)` / `anthropicProvider(name)` -return a full sub-provider bound to that gateway provider, so one -`createCoder` instance can target any number of providers. Pick the accessor -matching the provider's wire protocol: +**Sub-provider accessors.** `openaiProvider(name)` / `anthropicProvider(name)` +return a full sub-provider bound to that gateway provider, so one `createCoder` +instance can target any number of providers. Pick the accessor that matches the +provider's wire protocol: ```ts const azure = coder.openaiProvider("azure-openai"); // OpenAI-compatible type @@ -100,12 +100,14 @@ await generateText({ model: azure("gpt-4o"), prompt: "Hi" }); await generateText({ model: bedrock("claude-sonnet-4-6"), prompt: "Hi" }); ``` -A name outside the gateway's grammar throws the AI SDK's -`InvalidArgumentError` at accessor time (such a name can never be registered); -a well-formed name that is not configured on your deployment fails at request -time with the gateway's 404. +| Name | Fails with | +| -------------------------------------------- | ---------------------------------------------- | +| Outside the gateway's grammar | AI SDK `InvalidArgumentError` at accessor time | +| Well-formed but not configured on deployment | The gateway's 404 at request time | + +A name outside the grammar can never be registered, so it fails early. -**Re-pointing the defaults** — when your deployment simply names its one +**Re-pointing the defaults.** If your deployment just names its one OpenAI/Anthropic pair differently, override the names once and keep using the bare call and the `openai` / `anthropic` accessors: @@ -117,34 +119,40 @@ const renamed = createCoder({ }); ``` -**Provider names come from your platform team.** Discovery is admin-only +**Ask your Coder admins for provider names.** Discovery is admin-only server-side: `GET /api/v2/ai/providers` returns `403` for regular users, and -the models endpoint does not attribute models to providers. Ask your Coder -admins which provider names your deployment defines. +the models endpoint does not attribute models to providers. + +### Embeddings are not supported yet -Model ids are passed through **unchanged** to the upstream provider (no -`vendor/model` namespacing) — use whatever ids your deployment's providers accept. +`coder.textEmbeddingModel(id)` and the embedding accessors on `coder.openai` +throw the AI SDK's `NoSuchModelError` immediately. -**Embeddings are not supported yet.** AI Gateway does not intercept -`/v1/embeddings`, so `coder.textEmbeddingModel(id)` — and the embedding -accessors on `coder.openai` — throw the AI SDK's `NoSuchModelError` -immediately instead of emitting a request the gateway rejects with a 404. The -accessors stay so they can light up without a breaking change once the gateway -adds an embeddings route — see -[coder/ai-sdk#69](https://github.com/coder/ai-sdk/issues/69). +
+Why embeddings throw instead of sending a request + +AI Gateway does not intercept `/v1/embeddings`, so a request would be rejected +with a 404. Throwing up front avoids emitting it. The accessors stay so they +can light up without a breaking change once the gateway adds an embeddings +route. See [coder/ai-sdk#69](https://github.com/coder/ai-sdk/issues/69). + +
## Authentication -**Centralized mode (default).** Pass your Coder API token as `apiKey`. AI Gateway -holds the upstream provider keys and brokers the call — this is all you need: +| Mode | Set | Upstream provider key | +| ------------------------- | ------------------------------------------------------ | ------------------------------------------ | +| **Centralized** (default) | `apiKey`: your Coder API token | Held by AI Gateway, which brokers the call | +| **Bring-your-own-key** | `coderToken`: Coder token; `apiKey`: your upstream key | Yours, forwarded to the upstream | + +Centralized mode is all most apps need: ```ts createCoder({ baseURL: "https://coder.example.com", apiKey: coderToken }); ``` -**Bring-your-own-key (BYOK) mode.** Set `coderToken` (sent in the -`X-Coder-AI-Governance-Token` header to authenticate you to the gateway) and pass -your _upstream_ provider key as `apiKey` (forwarded to the upstream): +BYOK sends `coderToken` in the `X-Coder-AI-Governance-Token` header to +authenticate you to the gateway, and forwards `apiKey` to the upstream: ```ts createCoder({ @@ -156,150 +164,51 @@ createCoder({ ## Configuration -| Option | Type | Default | Description | -| --------------- | ------------------------- | ---------------------- | ----------------------------------------------------------------------------------------------------- | -| `baseURL` | `string` | — (required) | Your Coder deployment URL, e.g. `https://coder.example.com`. The AI Gateway path is appended for you. | -| `apiKey` | `string` | — | Coder API token (centralized) or upstream key (BYOK). | -| `coderToken` | `string` | — | Enables BYOK mode; sent in `X-Coder-AI-Governance-Token`. | -| `headers` | `Record` | — | Extra headers merged into every request. | -| `aiGatewayPath` | `string` | `/api/v2/aibridge` | Override if your deployment uses a different mount path. | -| `providers` | `{ openai?, anthropic? }` | `openai` / `anthropic` | Re-point the default pair at differently-named providers. | -| `fetch` | `typeof fetch` | global `fetch` | Custom fetch (testing / middleware). | +| Option | Type | Default | Description | +| --------------- | ------------------------- | ---------------------- | -------------------------------------------------------------------------------------------------------------------- | +| `baseURL` | `string` | — (required) | Your Coder deployment URL, e.g. `https://coder.example.com`. The AI Gateway path is appended for you. | +| `apiKey` | `string` | — | Coder API token (centralized) or upstream key (BYOK). | +| `coderToken` | `string` | — | Enables BYOK mode; sent in `X-Coder-AI-Governance-Token`. | +| `headers` | `Record` | — | Extra headers merged into every request. | +| `aiGatewayPath` | `string` | `/api/v2/aibridge` | Override if your deployment uses a different mount path. Use the canonical `/api/v2/ai-gateway` with Coder v2.35.0+. | +| `providers` | `{ openai?, anthropic? }` | `openai` / `anthropic` | Re-point the default pair at differently-named providers. | +| `fetch` | `typeof fetch` | global `fetch` | Custom fetch (testing / middleware). | ## Enterprise governance & security -Reference for security reviewers evaluating this package. The boundary between -the two kinds of claims below matters: **client behavior** (what this package -puts on the wire) is verifiable in [`src/provider.ts`](./src/provider.ts) — -~250 lines with no dependencies beyond the official AI SDK provider packages — -while **gateway behavior** (key custody, audit capture, retention) is a -property of your Coder deployment, enforced server-side regardless of what any -client does, and documented in the -[AI Gateway docs](https://coder.com/docs/ai-coder/ai-gateway). - -### Data flow - -```text -your app ──HTTP(S)──▶ your Coder deployment ──▶ upstream provider - (this package) (AI Gateway intercepts (Anthropic, OpenAI, - /api/v2/aibridge/…) Bedrock, Copilot, …) -``` - -**What leaves your app.** This package's only network destination is the -`baseURL` you configure. It never contacts upstream vendors directly and adds -no telemetry of its own; each request is exactly what the AI SDK builds for a -normal provider call: - -- **URL** — `POST /api/v2/aibridge//v1/chat/completions` - (OpenAI surface) or `…/v1/messages` (Anthropic surface). `/api/v2/aibridge` - is this package's default for compatibility. Use the canonical - `/api/v2/ai-gateway` path with Coder v2.35.0 or later (see `aiGatewayPath`). -- **Auth headers** — per the mode matrix below. -- **Body** — standard OpenAI-/Anthropic-format JSON: the model id (passed - through unchanged), your full prompt/message content, tool definitions, and - sampling parameters. Prompt content is visible to the Gateway — that is what - enables auditing. -- Anything you add via the `headers` option. - -Transport security follows the scheme of your `baseURL` — the client does not -enforce `https://` (an `http://` URL sends tokens, keys, and prompts in -plaintext). Always use an HTTPS deployment URL outside trusted local -environments. - -**What the Gateway does before forwarding** _(server-side — -[authentication docs](https://coder.com/docs/ai-coder/ai-gateway/auth))_: it -authenticates the token as an active Coder user and rejects the request -outright when it is missing or invalid (nothing is forwarded upstream), strips -all Coder credentials from the outbound request, and attaches the upstream -credential for the mode in use. - -### Credential isolation: centralized vs. BYOK - -| | **Centralized (default)** | **BYOK** | -| ------------------------------- | --------------------------------------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------- | -| Your app holds | Coder API token only | Coder API token **+** the user's own upstream key | -| On the wire | `Authorization: Bearer ` (both surfaces) | `X-Coder-AI-Governance-Token: `; upstream key in `Authorization: Bearer` (OpenAI surface) / `x-api-key` (Anthropic surface) | -| Upstream provider keys live | On the deployment, admin-configured — never distributed to developers | With the individual user; forwarded per request, bypassing the deployment's central key pool | -| Coder token forwarded upstream? | **No** — stripped and replaced by the deployment's provider key | **No** — the governance header is stripped before forwarding | -| Admin control | Provider and key configuration, rotation, failover | Can be disabled deployment-wide (`CODER_AI_GATEWAY_ALLOW_BYOK=false` rejects requests carrying the governance header with `403`) | - -Both modes are audited identically: audit records store _which credential -kind_ was used (`centralized` / `byok`), not the credential itself. - -### Audit capture - -Every request is attributed to the Coder user whose token authenticated it — -per user, per request, in both modes. Per intercepted request the Gateway -records _(server-side — -[audit docs](https://coder.com/docs/ai-coder/ai-gateway/audit))_: - -- **Identity & metadata** — initiating user, provider, model, client, - credential kind, timestamps. -- **Last user prompt** — earlier turns and system prompts are not stored. -- **Token usage** — input / output / cache counts. -- **Tool calls** — tool name and arguments; tool _results_ are not stored. -- **Model reasoning** — extended-thinking / reasoning-summary content when - present. - -Model-generated response text is **discarded**, not stored. Retention defaults -to **60 days** and is configurable (`CODER_AI_GATEWAY_RETENTION`; `0` keeps -data indefinitely). Auditors browse sessions and causal tool-call chains in -the deployment dashboard under `/ai-gateway/sessions`. - -### Required Coder permissions - -| To… | You need | -| -------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------- | -| Send requests through the Gateway | An active Coder user with a valid API token — no extra role. (RBAC: members hold create/update on `aibridge_interception`, i.e. recording their own traffic.) | -| Read audit data (prompts, tool calls, usage) | The **Owner** or **Auditor** role. Regular members cannot read interceptions back — not even their own. | -| Configure providers / keys, toggle BYOK | Deployment administrator (server flags / deployment configuration). | -| Use the feature at all | A deployment licensed for [AI Governance](https://coder.com/docs/ai-coder/ai-governance) — the Gateway is license-gated server-side. | - -### Security FAQ - -- **Does prompt data ever go anywhere other than my deployment?** This package - only ever _initiates_ requests to `baseURL`; onward traffic to the upstream - vendor originates from your deployment (or its standalone gateway replicas), - using the providers your admins configured. One standard-`fetch` caveat: - redirects are followed by default, so a cross-origin redirect issued by your - deployment or an intermediary would resend the request — prompt body and - non-`Authorization` headers (in BYOK mode that includes `x-api-key` and the - governance token) — to the redirect target. To forbid this, supply a custom - fetch: `fetch: (url, init) => fetch(url, { ...init, redirect: "error" })`. -- **Do developers ever handle raw provider keys?** Centralized mode: no — - developers only ever hold a Coder token. BYOK mode: they supply their own - personal key, which is forwarded per request without entering central custody. -- **What is the blast radius of a leaked Coder token?** A Coder API token is - not an AI-only credential — it grants the bearer the user's **full Coder API - permissions** (workspaces, templates, and anything else that user's roles - allow), and, until revoked or expired, AI usage through your Gateway (fully - attributed to that user). What it can _not_ do is authenticate to upstream - vendors: under normal Gateway forwarding it is stripped and does not leave - your deployment (the cross-origin-redirect caveat above is the exception — - a redirect can resend non-`Authorization` headers, including the governance - header, before the Gateway ever sees them). Treat a leak as a Coder account - compromise — revoke the token — and prefer short-lived, dedicated tokens - for AI workloads. -- **Is model-generated content stored?** Partially. Assistant _response text_ - is discarded, but two model-generated artifacts are retained for auditing: - reasoning content (extended thinking / reasoning summaries) and tool-call - arguments — alongside the last user prompt and token counts. -- **Can I verify the client claims myself?** Yes: [`src/provider.ts`](./src/provider.ts) - is the entire wire-facing surface — it only selects base URLs and auth - headers, then delegates request construction to the official AI SDK provider - packages. [`test/provider.test.ts`](./test/provider.test.ts) asserts the - request URL, auth headers, and model pass-through for the chat/messages - routes in both auth modes; the underlying AI SDK packages add their own - protocol headers (e.g. `anthropic-version`) and are not re-tested here. +The full reference for security reviewers is in +[`docs/security.md`](./docs/security.md): data flow, credential isolation, +audit capture, required permissions, and a security FAQ. + +Key points: + +- **Client vs. gateway.** What this package puts on the wire is verifiable in + [`src/provider.ts`](./src/provider.ts). Key custody, audit capture, and + retention are enforced server-side by your Coder deployment. +- **One destination.** The package only initiates requests to your `baseURL`. + It never contacts upstream vendors directly and adds no telemetry. +- **Use HTTPS outside trusted local environments.** The client does not + enforce `https://`; an `http://` URL sends tokens, keys, and prompts in + plaintext. +- **Redirects are followed by default.** A cross-origin redirect would resend + the prompt body and non-`Authorization` headers (in BYOK mode, including + `x-api-key` and the governance token) to the redirect target. Pass + `fetch: (url, init) => fetch(url, { ...init, redirect: "error" })` to forbid + it. +- **Coder tokens are not AI-only.** A token grants the user's full Coder API + permissions. Treat a leak as a Coder account compromise (revoke the token) + and prefer short-lived, dedicated tokens for AI workloads. +- **Audit.** Every request is attributed to the authenticating Coder user. The + Gateway stores the last user prompt, tool calls (name and arguments), + reasoning content when present, token usage, and metadata. Response text is + discarded. Retention defaults to 60 days and is configurable. ## Examples -Runnable scripts live in [`examples/`](./examples) (run against a real deployment via `tsx`): +Runnable scripts live in [`examples/`](./examples); see its +[README](./examples/README.md) for setup. ```bash -export CODER_URL=https://coder.example.com -export CODER_API_TOKEN=$(coder tokens create --name ai-sdk-provider-example) - pnpm example:generate # non-streaming generateText pnpm example:stream # streaming streamText pnpm example:anthropic # the Anthropic surface (native Claude) diff --git a/packages/provider/docs/security.md b/packages/provider/docs/security.md new file mode 100644 index 00000000..e2fee316 --- /dev/null +++ b/packages/provider/docs/security.md @@ -0,0 +1,159 @@ +# Enterprise governance & security + +Reference for security reviewers evaluating `@coder/ai-sdk-provider`. + +Two kinds of claims appear below. Keep them apart: + +| Claim kind | Covers | Where it is enforced / verifiable | +| -------------------- | ------------------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------- | +| **Client behavior** | What this package puts on the wire | [`src/provider.ts`](../src/provider.ts): ~250 lines, no dependencies beyond the official AI SDK provider packages | +| **Gateway behavior** | Key custody, audit capture, retention | Your Coder deployment, server-side, regardless of what any client does. See the [AI Gateway docs](https://coder.com/docs/ai-coder/ai-gateway) | + +## Data flow + +```text +your app ──HTTP(S)──▶ your Coder deployment ──▶ upstream provider + (this package) (AI Gateway intercepts (Anthropic, OpenAI, + /api/v2/aibridge/…) Bedrock, Copilot, …) +``` + +### What leaves your app + +The only network destination is the `baseURL` you configure. The package never +contacts upstream vendors directly and adds no telemetry of its own. Each +request is exactly what the AI SDK builds for a normal provider call: + +- **URL:** `POST /api/v2/aibridge//v1/chat/completions` + (OpenAI surface) or `…/v1/messages` (Anthropic surface). `/api/v2/aibridge` + is this package's default for compatibility. Use the canonical + `/api/v2/ai-gateway` path with Coder v2.35.0 or later (see `aiGatewayPath`). +- **Auth headers:** per the [mode matrix](#credential-isolation-centralized-vs-byok). +- **Body:** standard OpenAI-/Anthropic-format JSON: the model id (passed + through unchanged), your full prompt/message content, tool definitions, and + sampling parameters. Prompt content is visible to the Gateway; that is what + enables auditing. +- Anything you add via the `headers` option. + +**Transport security follows your `baseURL` scheme.** The client does not +enforce `https://`: an `http://` URL sends tokens, keys, and prompts in +plaintext. Always use an HTTPS deployment URL outside trusted local +environments. + +### What the Gateway does before forwarding + +_Server-side; see the [authentication docs](https://coder.com/docs/ai-coder/ai-gateway/auth)._ + +1. Authenticates the token as an active Coder user. A missing or invalid token + is rejected outright; nothing is forwarded upstream. +2. Strips all Coder credentials from the outbound request. +3. Attaches the upstream credential for the mode in use. + +## Credential isolation: centralized vs. BYOK + +| | **Centralized (default)** | **BYOK** | +| ------------------------------- | --------------------------------------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------- | +| Your app holds | Coder API token only | Coder API token **+** the user's own upstream key | +| On the wire | `Authorization: Bearer ` (both surfaces) | `X-Coder-AI-Governance-Token: `; upstream key in `Authorization: Bearer` (OpenAI surface) / `x-api-key` (Anthropic surface) | +| Upstream provider keys live | On the deployment, admin-configured — never distributed to developers | With the individual user; forwarded per request, bypassing the deployment's central key pool | +| Coder token forwarded upstream? | **No** — stripped and replaced by the deployment's provider key | **No** — the governance header is stripped before forwarding | +| Admin control | Provider and key configuration, rotation, failover | Can be disabled deployment-wide (`CODER_AI_GATEWAY_ALLOW_BYOK=false` rejects requests carrying the governance header with `403`) | + +Both modes are audited identically. Audit records store _which credential +kind_ was used (`centralized` / `byok`), not the credential itself. + +## Audit capture + +_Server-side; see the [audit docs](https://coder.com/docs/ai-coder/ai-gateway/audit)._ + +Every request is attributed to the Coder user whose token authenticated it: +per user, per request, in both modes. For each intercepted request the Gateway +records: + +| Recorded | Details | +| ------------------- | --------------------------------------------------------------------- | +| Identity & metadata | Initiating user, provider, model, client, credential kind, timestamps | +| Last user prompt | Earlier turns and system prompts are not stored | +| Token usage | Input / output / cache counts | +| Tool calls | Tool name and arguments; tool _results_ are not stored | +| Model reasoning | Extended-thinking / reasoning-summary content when present | + +- Model-generated response text is **discarded**, not stored. +- Retention defaults to **60 days** and is configurable + (`CODER_AI_GATEWAY_RETENTION`; `0` keeps data indefinitely). +- Auditors browse sessions and causal tool-call chains in the deployment + dashboard under `/ai-gateway/sessions`. + +## Required Coder permissions + +| To… | You need | +| -------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------- | +| Send requests through the Gateway | An active Coder user with a valid API token — no extra role. (RBAC: members hold create/update on `aibridge_interception`, i.e. recording their own traffic.) | +| Read audit data (prompts, tool calls, usage) | The **Owner** or **Auditor** role. Regular members cannot read interceptions back — not even their own. | +| Configure providers / keys, toggle BYOK | Deployment administrator (server flags / deployment configuration). | +| Use the feature at all | A deployment licensed for [AI Governance](https://coder.com/docs/ai-coder/ai-governance) — the Gateway is license-gated server-side. | + +## Security FAQ + +### Does prompt data ever go anywhere other than my deployment? + +This package only ever _initiates_ requests to `baseURL`. Onward traffic to the +upstream vendor originates from your deployment (or its standalone gateway +replicas), using the providers your admins configured. + +**Redirect caveat (standard `fetch`):** redirects are followed by default. A +cross-origin redirect issued by your deployment or an intermediary would +resend the request to the redirect target: the prompt body and +non-`Authorization` headers (in BYOK mode that includes `x-api-key` and the +governance token). To forbid this, supply a custom fetch: + +```ts +createCoder({ + baseURL: "https://coder.example.com", + apiKey: process.env.CODER_API_TOKEN!, + fetch: (url, init) => fetch(url, { ...init, redirect: "error" }), +}); +``` + +### Do developers ever handle raw provider keys? + +- **Centralized mode:** no. Developers only ever hold a Coder token. +- **BYOK mode:** they supply their own personal key, which is forwarded per + request without entering central custody. + +### What is the blast radius of a leaked Coder token? + +A Coder API token is not an AI-only credential. Until revoked or expired, it +grants the bearer: + +- the user's **full Coder API permissions** (workspaces, templates, and + anything else that user's roles allow), and +- AI usage through your Gateway, fully attributed to that user. + +It can _not_ authenticate to upstream vendors. Under normal Gateway forwarding +it is stripped and does not leave your deployment. The exception is the +[cross-origin-redirect caveat](#does-prompt-data-ever-go-anywhere-other-than-my-deployment): +a redirect can resend non-`Authorization` headers, including the governance +header, before the Gateway ever sees them. + +Treat a leak as a Coder account compromise: revoke the token. Prefer +short-lived, dedicated tokens for AI workloads. + +### Is model-generated content stored? + +Partially. Assistant _response text_ is discarded, but two model-generated +artifacts are retained for auditing: reasoning content (extended thinking / +reasoning summaries) and tool-call arguments. The last user prompt and token +counts are retained alongside them. + +### Can I verify the client claims myself? + +Yes. + +- [`src/provider.ts`](../src/provider.ts) is the entire wire-facing surface. It + only selects base URLs and auth headers, then delegates request construction + to the official AI SDK provider packages. +- [`test/provider.test.ts`](../test/provider.test.ts) asserts the request URL, + auth headers, and model pass-through for the chat/messages routes in both + auth modes. +- The underlying AI SDK packages add their own protocol headers (e.g. + `anthropic-version`) and are not re-tested here. diff --git a/packages/provider/examples/README.md b/packages/provider/examples/README.md index 9bd46338..74b84f93 100644 --- a/packages/provider/examples/README.md +++ b/packages/provider/examples/README.md @@ -1,7 +1,10 @@ # Examples -Runnable scripts for `@coder/ai-sdk-provider`. They import from `../src/index.js` -so you can run them straight against the source with [`tsx`](https://tsx.is). +Runnable scripts for `@coder/ai-sdk-provider`, run against the source +(`../src/index.js`) with [`tsx`](https://tsx.is). + +**Prerequisite:** a deployment with **AI Gateway enabled** and at least one +provider configured. ## Setup @@ -14,9 +17,8 @@ export CODER_MODEL=claude-sonnet-4-6 # any model id your deployment proxies export CODER_ANTHROPIC_MODEL=claude-sonnet-4-6 # used by 03-anthropic.ts ``` -Your deployment must have **AI Gateway enabled** and at least one provider -configured. The default model ids assume Anthropic is configured; override them -with the env vars above to match what your deployment proxies. +Default model ids assume Anthropic is configured; override them to match your +deployment. ## Run From 44514c37cee9edad8fd90a0d93e93cff1a45397a Mon Sep 17 00:00:00 2001 From: Thomas Kosiewski Date: Thu, 24 Sep 2026 09:57:30 +0000 Subject: [PATCH 04/10] docs(effect): tighten README Change-Id: I0f1c1c369b1b1cae3cb3d5112e6f3fe4bc8050cb Signed-off-by: Thomas Kosiewski --- packages/effect/README.md | 210 +++++++++++++++++++++----------------- 1 file changed, 119 insertions(+), 91 deletions(-) diff --git a/packages/effect/README.md b/packages/effect/README.md index f0f41580..f0c407db 100644 --- a/packages/effect/README.md +++ b/packages/effect/README.md @@ -1,49 +1,33 @@ # @coder/ai-sdk-effect > [!WARNING] -> **Experimental spike (Phase 1 of -> [coder/ai-sdk#144](https://github.com/coder/ai-sdk/issues/144)).** This -> package is `private: true`, is **not published to npm**, and is excluded from -> release-please. Its API will change without notice until the spike review -> concludes. - -An [Effect](https://effect.website) bridge for the Coder AI SDK: exposes -`@coder/ai-sdk-provider` (Coder AI Gateway) and `@coder/ai-sdk-sandbox` -(workspace sandboxes) through Effect-native abstractions — an `@effect/ai` -`LanguageModel` service, typed `AiError` failures, Effect Schema interop with -the Vercel AI SDK, and scoped `Layer`s for workspace lifecycle. - -## Pinned surface - -Effect's AI packages move fast, so the spike pins exact versions and codes -against their concrete API shapes: - -| Dependency | Version | Surface used | -| ------------------------ | -------- | ---------------------------------------------------------------------------------------------------------- | -| `effect` | `3.22.2` | `Effect`, `Layer`, `Stream`, `Schema`, `Context`, `Data` | -| `@effect/ai` | `0.37.0` | `LanguageModel.make` (`ProviderOptions` → encoded response parts), `AiError`, `Prompt`, `Response`, `Tool` | -| `@ai-sdk/provider` | `4.0.17` | `LanguageModelV4` spec types (same pin as `@coder/ai-sdk-provider`) | -| `@coder/ai-sdk-provider` | `0.4.20` | `createCoder`, `CoderProviderSettings` (published release, not `workspace:*` — see below) | -| `@coder/ai-sdk-sandbox` | `0.4.23` | `ensureCoderWorkspace`, `createCoderWorkspace`, `CoderTransport` | - -The spike depends on the _published_ `@coder/ai-sdk-*` releases rather than -`workspace:*` so that repo-wide `typecheck`/`test` need no cross-package build -ordering. Switching to `workspace:*` is part of the Phase 2 publishing -decision. - -## What exists (Phase 1) - -### `LanguageModel` over Coder AI Gateway - -`CoderLanguageModel.layer(modelId, source)` implements `@effect/ai`'s -`LanguageModel` service on top of the gateway. `source` is either -`CoderProviderSettings` or `{ provider }` for an existing `CoderProvider`, so -both auth modes (centralized and BYOK) work unchanged. `generateText`, -`generateObject`, and `streamText` are supported; structured outputs derive -their JSON schema from the Effect Schema you pass. Generation controls -(`maxOutputTokens`, `temperature`, `topP`, `topK`, penalties, `stopSequences`, -`seed`, `reasoning`) are set at construction time via an optional third -`GenerationOptions` argument and forwarded on every call. +> **Experimental spike: not published to npm.** Phase 1 of +> [coder/ai-sdk#144](https://github.com/coder/ai-sdk/issues/144). The package +> is `private: true` and excluded from release-please. Its API will change +> without notice until the spike review concludes. + +An [Effect](https://effect.website) bridge for `@coder/ai-sdk-provider` (Coder +AI Gateway) and `@coder/ai-sdk-sandbox` (workspace sandboxes): + +| Feature | API | +| ------------------------------------------------------------------------------------ | ------------------------------------------------------------------------ | +| [`@effect/ai` `LanguageModel` over AI Gateway](#languagemodel-over-coder-ai-gateway) | `CoderLanguageModel.layer`, `CoderLanguageModel.fromModel` | +| [Typed `AiError` failures](#typed-error-taxonomy) | `classifyError`, `isTransient` | +| [Effect Schema → Vercel AI SDK schemas](#effect-schema--vercel-ai-sdk-schemas) | `toAiSdkSchema` | +| [Scoped workspace `Layer`s](#scoped-sandbox-layers) | `acquireWorkspace` / `layerWorkspace`, `acquireSession` / `layerSession` | + +## `LanguageModel` over Coder AI Gateway + +`CoderLanguageModel.layer(modelId, source, generation?)` implements `@effect/ai`'s +`LanguageModel` service on top of the gateway. + +- **`source`:** `CoderProviderSettings`, or `{ provider }` for an existing + `CoderProvider`. Both auth modes (centralized and BYOK) work unchanged. +- **Supported calls:** `generateText`, `generateObject`, and `streamText`. + Structured outputs derive their JSON schema from the Effect Schema you pass. +- **`generation` (`GenerationOptions`):** `maxOutputTokens`, `temperature`, + `topP`, `topK`, penalties, `stopSequences`, `seed`, `reasoning`. Set at + construction time and forwarded on every call. ```ts import * as LanguageModel from "@effect/ai/LanguageModel"; @@ -68,19 +52,14 @@ program.pipe( ); ``` -The bridge core (`CoderLanguageModel.fromModel`) adapts any AI SDK -`LanguageModelV4`, which is also how the unit tests exercise it without HTTP. +`CoderLanguageModel.fromModel` (the bridge core) adapts any AI SDK +`LanguageModelV4`; unit tests use it to run without HTTP. -### Typed error taxonomy +## Typed error taxonomy -`@effect/ai`'s `AiError` is a **closed union**, so the bridge cannot add its -own error classes to the `LanguageModel` failure channel (a delta from the -tracking issue's sketch). Instead, failures are mapped losslessly into that -union (`HttpResponseError` keeps status, headers, and body) and `classifyError` -recovers the Coder-oriented taxonomy — `auth`, `rate-limit`, -`provider-unavailable`, `malformed-response`, `transport`, `unknown` — from -either an `AiError` or a raw AI SDK error. `isTransient` composes with -`Effect.retry`: +`classifyError` maps an `AiError` or a raw AI SDK error to one of: `auth`, +`rate-limit`, `provider-unavailable`, `malformed-response`, `transport`, +`unknown`. `isTransient` composes with `Effect.retry`: ```ts import * as LanguageModel from "@effect/ai/LanguageModel"; @@ -97,15 +76,21 @@ const resilient = LanguageModel.generateText({ prompt: "hello" }).pipe( ); ``` -### Effect Schema → Vercel AI SDK schemas +
+Why there are no Coder-specific error classes -AI SDK v7 accepts Standard Schema V1 values, but deriving the wire JSON schema -requires the optional `~standard.jsonSchema` converter, which -`Schema.standardSchemaV1` does not emit — passing a bare Effect standard -schema throws `Standard schema vendor 'effect' does not support JSON Schema -conversion`. `toAiSdkSchema` bridges the gap: the JSON schema is derived from -the Effect Schema (same derivation `@effect/ai` uses) and validation decodes -through it, so `tool()` inputs arrive fully decoded and typed: +`@effect/ai`'s `AiError` is a **closed union**, so the bridge cannot add its +own error classes to the `LanguageModel` failure channel (a delta from the +tracking issue's sketch). Failures are mapped losslessly into that union +(`HttpResponseError` keeps status, headers, and body), and `classifyError` +recovers the Coder-oriented taxonomy. + +
+ +## Effect Schema → Vercel AI SDK schemas + +`toAiSdkSchema` makes an Effect Schema usable in AI SDK APIs such as `tool()`, +with inputs fully decoded and typed: ```ts import * as Schema from "effect/Schema"; @@ -119,14 +104,32 @@ const getWeather = tool({ }); ``` -### Scoped sandbox `Layer`s +
+Why a bare Effect standard schema fails + +AI SDK v7 accepts Standard Schema V1 values, but deriving the wire JSON schema +requires the optional `~standard.jsonSchema` converter. `Schema.standardSchemaV1` +does not emit it, so passing a bare Effect standard schema throws +`Standard schema vendor 'effect' does not support JSON Schema conversion`. + +`toAiSdkSchema` derives the JSON schema from the Effect Schema (the same +derivation `@effect/ai` uses), and validation decodes through it. + +
+ +## Scoped sandbox `Layer`s `acquireWorkspace` / `layerWorkspace` wrap `ensureCoderWorkspace` in -`Effect.acquireRelease`: the workspace is provisioned (get-or-create, start, -agent-readiness wait) when the scope opens and torn down when it closes — -including when the fiber is interrupted after acquisition. The teardown policy -(`delete-if-created` by default) never touches a workspace the acquisition -merely attached to. `acquireSession` / `layerSession` do the same for +`Effect.acquireRelease`: + +- **Scope opens:** the workspace is provisioned (get-or-create, start, + agent-readiness wait). +- **Scope closes:** it is torn down, including when the fiber is interrupted + after acquisition. +- **Teardown policy:** `delete-if-created` by default. It never touches a + workspace the acquisition merely attached to. + +`acquireSession` / `layerSession` do the same for `createCoderWorkspace(...).createSession()`. ```ts @@ -151,42 +154,67 @@ program.pipe( ## Spike caveats -Honest limitations, chosen to keep the core small and correct. Unsupported -_inputs_ fail loudly with `MalformedInput`; response parts with no `@effect/ai` -equivalent are dropped: - -- **Provider-defined tools** and the **`oneOf` tool-choice mode** are not - expressible in `LanguageModelV4` call options → `MalformedInput`. -- Response parts of type `custom`, `reasoning-file`, and - `tool-approval-request`, and file payloads that are not raw data (URL / - provider-reference / inline-text), are dropped. -- `Prompt` provider options (per-part metadata) are not forwarded, and - generation controls are fixed at model construction — a per-call override - channel (an Effect config service, as `@effect/ai`'s own providers use) is - Phase 2. -- Workspace acquisition is uninterruptible (standard `acquireRelease` - semantics); a slow `ensureCoderWorkspace` cannot be cancelled mid-flight. - A workspace created by an acquisition that then fails (e.g. readiness - timeout) is rolled back best-effort per the teardown policy. -- Telemetry: the `ProviderOptions.span` is not wired into request headers. - -## Phase 2 (not in this package yet) +Unsupported _inputs_ fail loudly with `MalformedInput`; response parts with no +`@effect/ai` equivalent are dropped. + +| Limitation | Behavior | +| -------------------------------------------------------------------------------------------------------------------------------------------------------- | ---------------------------------------------------------- | +| Provider-defined tools; the `oneOf` tool-choice mode (not expressible in `LanguageModelV4` call options) | `MalformedInput` | +| Response parts of type `custom`, `reasoning-file`, `tool-approval-request`; file payloads that are not raw data (URL / provider-reference / inline-text) | Dropped | +| `Prompt` provider options (per-part metadata) | Not forwarded | +| Generation controls | Fixed at model construction (per-call override is Phase 2) | +| `ProviderOptions.span` (telemetry) | Not wired into request headers | + +Workspace acquisition is uninterruptible (standard `acquireRelease` +semantics): a slow `ensureCoderWorkspace` cannot be cancelled mid-flight. If an +acquisition creates a workspace and then fails (e.g. readiness timeout), the +workspace is rolled back best-effort per the teardown policy. + +
+Phase 2 (not in this package yet) - `LanguageModel` over `CoderAgent`/chatd (`TurnTranslator` → `Effect.Stream`, fiber interruption → `agent.interrupt()`). - Retryable error tagging aligned with the agent package's `CoderStreamError.isRetryable`. +- A per-call override channel for generation controls (an Effect config + service, as `@effect/ai`'s own providers use). - Publishing decision: versioning, `peerDependency` policy on `effect`/`@effect/ai`, release-please wiring, `workspace:*` deps. +
+ +## Pinned surface + +Effect's AI packages move fast, so the spike pins exact versions and codes +against their concrete API shapes. + +
+Pinned versions and the API surface used + +| Dependency | Version | Surface used | +| ------------------------ | -------- | ---------------------------------------------------------------------------------------------------------- | +| `effect` | `3.22.2` | `Effect`, `Layer`, `Stream`, `Schema`, `Context`, `Data` | +| `@effect/ai` | `0.37.0` | `LanguageModel.make` (`ProviderOptions` → encoded response parts), `AiError`, `Prompt`, `Response`, `Tool` | +| `@ai-sdk/provider` | `4.0.17` | `LanguageModelV4` spec types (same pin as `@coder/ai-sdk-provider`) | +| `@coder/ai-sdk-provider` | `0.4.20` | `createCoder`, `CoderProviderSettings` (published release, not `workspace:*`) | +| `@coder/ai-sdk-sandbox` | `0.4.23` | `ensureCoderWorkspace`, `createCoderWorkspace`, `CoderTransport` | + +The spike depends on the _published_ `@coder/ai-sdk-*` releases rather than +`workspace:*`, so repo-wide `typecheck`/`test` need no cross-package build +ordering. Switching to `workspace:*` is part of the Phase 2 publishing +decision. + +
+ ## Examples Runnable against a real deployment (see each file's header): -- [`examples/01-generate.ts`](./examples/01-generate.ts) — text generation. -- [`examples/02-structured.ts`](./examples/02-structured.ts) — structured +- [`examples/01-generate.ts`](./examples/01-generate.ts): text generation. +- [`examples/02-structured.ts`](./examples/02-structured.ts): structured output via Effect Schema. -- [`examples/03-sandbox.ts`](./examples/03-sandbox.ts) — scoped workspace +- [`examples/03-sandbox.ts`](./examples/03-sandbox.ts): scoped workspace acquisition. ## License From 5ef13e40b95c1b33c60800c2143a699d679e198b Mon Sep 17 00:00:00 2001 From: Thomas Kosiewski Date: Thu, 24 Sep 2026 09:57:30 +0000 Subject: [PATCH 05/10] docs: tighten release-please-ai README Change-Id: Ib15397d81356a03cb3552404db0d7d19cd769982 Signed-off-by: Thomas Kosiewski --- packages/release-please-ai/README.md | 108 +++++++++++++++------------ 1 file changed, 59 insertions(+), 49 deletions(-) diff --git a/packages/release-please-ai/README.md b/packages/release-please-ai/README.md index 3f38c15d..7766ca0d 100644 --- a/packages/release-please-ai/README.md +++ b/packages/release-please-ai/README.md @@ -1,20 +1,21 @@ # @coder/release-please-ai -AI-written release notes, generated **inside** [release-please](https://github.com/googleapis/release-please). +AI-written release notes, generated **inside** +[release-please](https://github.com/googleapis/release-please). For each +component release, Claude adds a plain-language summary and a few highlights on +top of release-please's structured changelog. -It plugs into release-please as a custom changelog generator: for every component -release, it keeps release-please's accurate, structured changelog and adds a short -editorial intro — a plain-language summary plus a few highlights — written by Claude. -release-please still owns the `CHANGELOG.md`, the release pull request, and the GitHub -Release; this package only enriches the notes it produces. +release-please still owns `CHANGELOG.md`, the release PR, and the GitHub +Release. -> Currently `private` (unpublished). The generator is repo-agnostic — designed to be published and reused across repositories. +> Currently `private` (unpublished), but repo-agnostic: designed to be +> published and reused across repositories. ## What you get -For a release whose commits are `feat(agent): add streaming responses (#42)`, -`fix(agent): stop dropping the final token (#43)`, `feat(agent)!: require Node 22 (#44)`, -release-please writes: +For a release with commits `feat(agent): add streaming responses (#42)`, +`fix(agent): stop dropping the final token (#43)`, and +`feat(agent)!: require Node 22 (#44)`, release-please writes: ```markdown ## [0.2.0](https://github.com/coder/ai-sdk/compare/agent-v0.1.0...agent-v0.2.0) (2026-06-18) @@ -39,41 +40,51 @@ Node 22, which is a breaking change. … ``` -Everything from the version header down is release-please's standard output; only the -summary paragraph and **Highlights** list are AI-written. The model only writes prose — -the version header, section grouping, PR/commit links, and breaking-change list all come -from release-please, so there is nothing for it to get factually wrong. +Only the summary paragraph and **Highlights** are AI-written. Headers, +sections, links, and the breaking-change list are release-please's standard +output, so there is nothing for the model to get factually wrong. + +**A release is never blocked on the model.** If it can't run (no API key, a +network/API error, or nothing noteworthy in the release), the generator returns +release-please's standard notes unchanged. ## How it works -1. `release-please-config.json` sets `"changelog-type": "ai"`. -2. The CLI registers the generator (`registerChangelogNotes("ai", …)`) and then runs - release-please. release-please only exposes its built-in `default`/`github` changelog - types through its config and action; selecting a _custom_ type requires registering it - in code first, so this package drives release-please's library API directly instead of - the off-the-shelf action. The rest of the release pipeline (versioning, tags, the - release PR, publishing) is unchanged. -3. For each component, release-please hands the generator the parsed, **already - path-scoped** commits for that package. The generator: - - delegates to release-please's own default changelog renderer for all structure, then - - asks Claude for a `summary` + `highlights` and splices that block in just below the - version header. - -If the model can't run — no API key, a network/API error, or simply nothing -noteworthy in the release — the generator returns release-please's standard notes -unchanged. **A release is never blocked on the model.** +1. `release-please-config.json` sets `"changelog-type": "ai"`. Editors may flag + the value (the published schema lists only `default`/`github`); that is + cosmetic, as it resolves at runtime once the generator is registered. +2. The CLI registers this custom changelog generator + (`registerChangelogNotes("ai", …)`), then runs release-please through its + library API. +3. For each component, the generator receives the parsed, **already + path-scoped** commits. It renders the structure with release-please's + default renderer, asks Claude for a `summary` + `highlights`, and splices + them in just below the version header. + +Versioning, tags, the release PR, and publishing are unchanged. + +
+Why this drives release-please's library API instead of the action + +release-please only exposes its built-in `default`/`github` changelog types +through its config and action. Selecting a _custom_ type requires registering +it in code first, so this package drives the library API directly instead of +the off-the-shelf action. + +
## Usage ### In CI (this repo) -Wired into `.github/workflows/release-please.yml`. The release job installs the workspace -and runs the CLI; it emits the same step outputs the stock release-please action does -(`--release_created`, `--tag_name`, `--version`), so downstream publish jobs need no -changes. +The release job in `.github/workflows/release-please.yml` installs the +workspace and runs the CLI. -**Required:** add an `ANTHROPIC_API_KEY` repository secret. Without it, releases still -succeed — they just use release-please's default notes. +- **Required:** an `ANTHROPIC_API_KEY` repository secret. Without it, releases + still succeed with default notes. +- Step outputs match the stock release-please action + (`--release_created`, `--tag_name`, `--version`), so downstream publish + jobs need no changes. ### As a library @@ -97,7 +108,7 @@ GITHUB_TOKEN=… GITHUB_REPOSITORY=coder/ai-sdk ANTHROPIC_API_KEY=… \ ## Configuration -The CLI is configured entirely through the environment: +The CLI is configured entirely through environment variables: | Variable | Required | Default | Purpose | | ------------------------------ | ------------ | ------------------------------- | --------------------------------------------- | @@ -109,20 +120,16 @@ The CLI is configured entirely through the environment: | `RELEASE_PLEASE_CONFIG_FILE` | no | `release-please-config.json` | Path to the config. | | `RELEASE_PLEASE_MANIFEST_FILE` | no | `.release-please-manifest.json` | Path to the manifest. | -The headings and which commit types are shown/hidden come from `changelog-sections` in -`release-please-config.json` — the same source release-please uses — so the AI notes and -the changelog always stay in sync. - -> The published release-please JSON schema only lists `default` and `github` for -> `changelog-type`, so editors may flag `"ai"`. That warning is cosmetic; the value is -> resolved at runtime once the generator is registered. +Section headings and visible commit types come from `changelog-sections` in +`release-please-config.json`, so the AI notes and the changelog always stay in +sync. ## Behavior & cost -release-please rebuilds a release's notes whenever it opens or updates that component's -release PR — i.e. on pushes to `main` that change a releasable package — so the notes stay -current while the PR is open. Generation is commit-driven (no diffs), so it's cheap: -roughly a few cents per run on Opus 4.8. +- **Regeneration:** whenever release-please opens or updates a component's + release PR (pushes to `main` that change a releasable package), so notes stay + current while the PR is open. +- **Cost:** commit-driven (no diffs), roughly a few cents per run on Opus 4.8. ## Development @@ -132,7 +139,8 @@ pnpm --filter @coder/release-please-ai test # unit tests; hermetic, model pnpm --filter @coder/release-please-ai test:e2e # live model call; skipped unless ANTHROPIC_API_KEY is set ``` -Source layout: +
+Source layout | File | Responsibility | | ------------------------ | ------------------------------------------------------------------------- | @@ -142,3 +150,5 @@ Source layout: | `src/prompt.ts` | System prompt and commit-to-prompt formatting. | | `src/sections.ts` | Section definitions and visible-commit filtering. | | `src/index.ts` | Public library exports. | + +
From c3adcc0ef49cc41484f723b35b6f407352c7177a Mon Sep 17 00:00:00 2001 From: Thomas Kosiewski Date: Thu, 24 Sep 2026 10:06:07 +0000 Subject: [PATCH 06/10] docs(agent): tighten README and move deep dives to docs/ Rewrite packages/agent/README.md for skimming: result-first sections, tables for options/errors/return values, and internals in
. Move the transport-event reference to docs/observability.md and the structured_output tool pattern to docs/structured-output.md, keeping short summaries under the same headings. Replace the Workspaces & quota and Durable workflows sections with summaries that link to their new docs files (written separately). Signed-off-by: Thomas Kosiewski --- packages/agent/README.md | 1945 +++++----------------- packages/agent/docs/observability.md | 130 ++ packages/agent/docs/structured-output.md | 134 ++ 3 files changed, 714 insertions(+), 1495 deletions(-) create mode 100644 packages/agent/docs/observability.md create mode 100644 packages/agent/docs/structured-output.md diff --git a/packages/agent/README.md b/packages/agent/README.md index 89bc4377..357f775d 100644 --- a/packages/agent/README.md +++ b/packages/agent/README.md @@ -4,42 +4,32 @@ [![npm](https://img.shields.io/npm/v/@coder/ai-sdk-agent.svg)](https://www.npmjs.com/package/@coder/ai-sdk-agent) [![License](https://img.shields.io/badge/license-Apache--2.0-blue.svg)](./LICENSE) -A **Vercel AI SDK–compliant agent backed by Coder Agents** — Coder's server‑side -agent runtime. Call `new CoderAgent(...)` and get back an object that implements the -AI SDK's [`Agent`](https://ai-sdk.dev/docs/reference/ai-sdk-core/agent) interface -(`generate()` / `stream()`). Script it, stream from it, and attach your own tools — +A **Vercel AI SDK agent backed by Coder Agents**, Coder's server‑side agent +runtime. `new CoderAgent(...)` implements the AI SDK's +[`Agent`](https://ai-sdk.dev/docs/reference/ai-sdk-core/agent) interface +(`generate()` / `stream()`): script it, stream from it, and attach your own tools exactly like the SDK's own `ToolLoopAgent`. +Coder Agents runs the complete agent loop **server‑side**: the multi‑step tool +loop, built‑in tools, MCP, sub‑agents, multi‑provider model routing, and +automatic context compaction. This package bridges it to the AI SDK's +client‑side loop without re‑implementing the loop. + > Status: targets Coder's stable chat API (`/api/v2/chats`), available since > Coder 2.37.0 (September 1, 2026). This package remains pre‑1.0. > See [API compatibility](#api-compatibility) for older deployments. -## Why - -**Coder Agents** runs a complete agent loop **server‑side** — the multi‑step tool -loop, built‑in tools, MCP, sub‑agents, multi‑provider model routing, and automatic -context compaction. The Vercel AI SDK runs its loop **client‑side**. This package -bridges the two, so a Coder agent looks and feels like a native AI SDK agent without -re‑implementing the loop. - ## Agent vs. provider — which package? -Two packages, two jobs: +| | `@coder/ai-sdk-agent` (this package) | [`@coder/ai-sdk-provider`](../provider) | +| ----------------- | --------------------------------------------------------------------------------------------------------------------------------------------------- | ---------------------------------------------------------------------------------------------------------------------- | +| What runs | Coder's server‑side agent: tool loop, built‑in tools, MCP servers, workspace‑scoped file/shell tools, sub‑agents, and compaction, on the deployment | Plain model calls through Coder's AI Gateway. A normal AI SDK provider: `generateText`, `streamText`, `generateObject` | +| Server state | Each `CoderAgent` is one server chat ("session") and may provision a workspace | No chat, no workspace; natively cancelable | +| Structured output | Through a tool call ([Structured output](#structured-output)) | `generateObject`, schema‑constrained | +| Reach for it when | You need **server‑side tools, MCP, or a workspace** | You just need **a model** (plan / extract / summarize / classify) | -- **`@coder/ai-sdk-agent` (this package)** — Coder's **server‑side agent**: the - multi‑step tool loop, built‑in tools, MCP servers, workspace‑scoped file/shell - tools, sub‑agents, and compaction all run on the deployment. Each `CoderAgent` - is one server chat ("session") and may provision a workspace. Reach for it when - you need **server‑side tools, MCP, or a workspace**. -- **[`@coder/ai-sdk-provider`](../provider)** — **plain model calls** through - Coder's AI Gateway. A normal AI SDK provider: `generateText`, `streamText`, and - **`generateObject` for schema‑constrained structured output**. No chat, no - workspace, natively cancelable. Reach for it when you just need **a model** - (plan / extract / summarize / classify) with no server‑side tools. - -Rule of thumb: **need server‑side tools, MCP, or a workspace → Agent; need a model -→ provider.** They compose — a multi‑step pipeline often uses the provider for its -pure text/JSON steps and the Agent only for the steps that touch tools. +They compose: a multi‑step pipeline often uses the provider for its pure +text/JSON steps and the Agent only for the steps that touch tools. ## Install @@ -47,9 +37,9 @@ pure text/JSON steps and the Agent only for the steps that touch tools. pnpm add @coder/ai-sdk-agent ai@^7 zod ``` -Requires Node ≥ 22 and `ai` v7 — the constructors throw an actionable error when -another `ai` major is detected (the guard fails open when the installed version -can't be resolved), instead of failing cryptically mid‑generation. +Requires Node ≥ 22 and `ai` v7. The constructors throw an actionable error when +another `ai` major is detected, instead of failing cryptically mid‑generation. +The guard fails open when the installed version can't be resolved. ## Quick start @@ -81,14 +71,15 @@ const result = await agent.stream({ prompt: "Write a haiku about Coder." }); for await (const delta of result.textStream) process.stdout.write(delta); ``` -`generate()` returns a real AI SDK `GenerateTextResult`; `stream()` returns a real -`StreamTextResult` (so `.textStream`, `.fullStream`, `.toUIMessageStream()`, `.steps`, -`.usage`, etc. all work). Because `CoderAgent` _is_ an `Agent`, it composes with the rest -of the AI SDK. +`generate()` returns a real AI SDK `GenerateTextResult`; `stream()` returns a +real `StreamTextResult` (`.textStream`, `.fullStream`, `.toUIMessageStream()`, +`.steps`, `.usage`, …). Because `CoderAgent` _is_ an `Agent`, it composes with +the rest of the AI SDK. ## Examples -Runnable scripts live in [`examples/`](./examples) (run against a real deployment via `tsx`): +Runnable scripts in [`examples/`](./examples) run against a real deployment via +`tsx`: ```bash export CODER_URL=https://dogfood.cdr.dev @@ -102,34 +93,41 @@ pnpm example:file # attach a file to a chat (optional: pass a path) pnpm example:structured # typed structured output via the structured_output tool ``` -Each example creates a new chat and archives it when done — it never touches workspaces. See -[`examples/README.md`](./examples/README.md) for details. +Each example creates a new chat and archives it when done; none touches +workspaces. Details: [`examples/README.md`](./examples/README.md). ## Custom tools -Tools you pass are registered with Coder Agents as **client‑executed** ("dynamic") tools. -When the model calls one, the run pauses on the server; the AI SDK runs your tool's -`execute`, this package submits the result back, and the run resumes. This is the standard -AI SDK tool loop — your `execute` runs in your process. +Tools you pass are registered with Coder Agents as **client‑executed** +("dynamic") tools; your `execute` runs in your process. This is the standard AI +SDK tool loop: + +1. The model calls your tool, and the run pauses on the server. +2. The AI SDK runs your tool's `execute`. +3. This package submits the result back, and the run resumes. -- Give tools an `execute` for scripting use (the loop runs to completion automatically). -- Coder's own server‑side tools (file editing, shell, MCP, …) still run on the server and - appear in the transcript as `providerExecuted` tool calls/results — you observe them, you - don't execute them. +- Give tools an `execute` for scripting use; the loop runs to completion + automatically. +- Coder's own server‑side tools (file editing, shell, MCP, …) still run on the + server. They appear in the transcript as `providerExecuted` tool + calls/results: you observe them, you don't execute them. -Migration note: since v0.2.1 server‑executed tools stream with `dynamic: true` -(they aren't in your `ToolSet`, and the AI SDK only accepts unknown tool names on -dynamic calls). In UI message streams they therefore surface as `dynamic-tool` -parts rather than `tool-{name}` parts — key off `toolName`, not `part.type`, when -rendering them. +
+Since v0.2.1: server tools stream as dynamic-tool parts + +Server‑executed tools stream with `dynamic: true`: they aren't in your `ToolSet`, +and the AI SDK only accepts unknown tool names on dynamic calls. In UI message +streams they therefore surface as `dynamic-tool` parts rather than `tool-{name}` +parts. Key off `toolName`, not `part.type`, when rendering them. + +
## Files -There are two distinct ways to get a file to the agent, depending on whether the model should -**read** it or **operate on** it. +Pick by whether the model should **read** the file or **operate on** it. -**Chat attachments** — content for the model to read (a PDF, image, CSV…). Drop a native AI SDK -`file` part into a message and it's uploaded transparently: +**Chat attachments** carry content for the model to read (a PDF, image, CSV…). +Drop a native AI SDK `file` part into a message and it's uploaded transparently: ```ts import { readFile } from "node:fs/promises"; @@ -152,10 +150,13 @@ await agent.generate({ }); ``` -Or upload once and reuse across turns with `attach()` — which also accepts a `Blob`/`File` or -stream (use `fs.openAsBlob` to avoid reading the whole file into memory): +To upload once and reuse across turns, use `attach()`. It also accepts a +`Blob`/`File` or stream; `fs.openAsBlob` avoids reading the whole file into +memory: ```ts +import { openAsBlob } from "node:fs"; + const file = await agent.attach({ content: await openAsBlob("report.pdf"), mediaType: "application/pdf", @@ -173,14 +174,17 @@ await agent.generate({ }); ``` -Attachments are capped at **10 MiB** and restricted to a narrow media‑type allowlist -(`application/pdf`, `application/json`, `text/{plain,markdown,csv}`, `image/{png,jpeg,gif,webp}`). -Oversized or unsupported files throw a clear error up front. +Attachments are capped at **10 MiB** and restricted to this media‑type +allowlist: `application/pdf`, `application/json`, `text/{plain,markdown,csv}`, +`image/{png,jpeg,gif,webp}`. Oversized or unsupported files throw a clear error +up front. + +**Workspace files** are material for the agent to operate on: a zip of assets, a +dataset, a binary — anything outside the allowlist or over the cap. Write it onto +the workspace filesystem and let the agent's tools take over. -**Workspace files** — material for the agent to operate on (a zip of assets, a dataset, a -binary — anything outside the allowlist or over the cap). Write it onto the workspace filesystem -and let the agent's tools take over. This needs a `workspaceFiles` adapter (the agent core stays -dependency‑free; whoever holds a workspace connection supplies a few‑line adapter): +This needs a `workspaceFiles` adapter. The agent core stays dependency‑free, so +whoever holds a workspace connection supplies a few‑line adapter: ```ts const agent = new CoderAgent({ /* … */ workspaceId: ws.id, workspaceFiles }); @@ -193,14 +197,13 @@ const { path } = await agent.uploadToWorkspace({ ## Auth -Pass a Coder **API token** or **session token** as `token`; it is sent as the -`Coder-Session-Token` header (REST) and used to authenticate the streaming WebSocket. Create -a token with `coder tokens create`, or reuse your CLI session. +Pass a Coder **API token** or **session token** as `token`. It is sent as the +`Coder-Session-Token` header (REST) and authenticates the streaming WebSocket. +Create one with `coder tokens create`, or reuse your CLI session. -When `baseUrl`/`token` are not passed, they default from the `CODER_URL` and -`CODER_SESSION_TOKEN` environment variables (the same convention as -`@coder/ai-sdk-sandbox`'s transports); explicit settings win over the -environment. With the variables set, connection config disappears entirely: +`baseUrl`/`token` default from the `CODER_URL` and `CODER_SESSION_TOKEN` +environment variables, the same convention as `@coder/ai-sdk-sandbox`'s +transports. Explicit settings win over the environment: ```ts const agent = new CoderAgent({ organizationId }); // uses CODER_URL + CODER_SESSION_TOKEN @@ -218,59 +221,68 @@ const agent = new CoderAgent({ client, organizationId }); Coder 2.37.0 promoted chat routes to `/api/v2/chats` ([server routes](https://github.com/coder/coder/pull/28496), -[Go SDK](https://github.com/coder/coder/pull/28497)). The promoted routes also -remain under `/api/experimental/chats` during the compatibility window -(CODAGT-921); legacy removal is tracked in CODAGT-922. Coder 2.36.4 has no stable -chat mount. +[Go SDK](https://github.com/coder/coder/pull/28497)). Coder 2.36.4 has no stable +chat mount. During the compatibility window (CODAGT-921) the promoted routes also +remain under `/api/experimental/chats`; legacy removal is tracked in CODAGT-922. Each `CoderChatClient` selects its chat prefix from HTTP responses, not version -strings. Until selected, an actual REST request (including JSON requests and -`Blob` uploads) tries v2 first and retries once under the experimental prefix -only on 404. The first non-404 HTTP response -(including an error) locks that prefix for the client's lifetime; concurrent -unresolved calls may probe independently, with the first conclusive response -winning. Two 404s preserve the original v2 error without locking; non-HTTP -failures do not lock either. A locked prefix is never renegotiated, even on 404. -Before opening a per-chat `/stream` or global `/watch` WebSocket, an unresolved -client preflights with `GET /api/v2/chats` using the same selection rules and -cancellation signal. -`ReadableStream` uploads also preflight so their consumed bodies are never -replayed. Legacy model-config lookup remains independent of chat-prefix -selection (see [Configuration](#configuration)). +strings: v2 first, then the experimental prefix on a 404. The first non‑404 HTTP +response locks the prefix for the client's lifetime. + +
+Prefix selection rules + +- Until a prefix is selected, an actual REST request (including JSON requests + and `Blob` uploads) tries v2 first and retries once under the experimental + prefix only on 404. +- The first non‑404 HTTP response (including an error) locks that prefix for the + client's lifetime. Concurrent unresolved calls may probe independently; the + first conclusive response wins. +- Two 404s preserve the original v2 error without locking. Non‑HTTP failures do + not lock either. +- A locked prefix is never renegotiated, even on 404. +- Before opening a per‑chat `/stream` or global `/watch` WebSocket, an + unresolved client preflights with `GET /api/v2/chats`, using the same + selection rules and cancellation signal. +- `ReadableStream` uploads also preflight, so their consumed bodies are never + replayed. +- Legacy model‑config lookup remains independent of chat‑prefix selection (see + [Configuration](#configuration)). + +
## Sessions -One `CoderAgent` instance maps to one chat ("session") on the Coder server. The chat is -created on the first turn and reused for subsequent `generate()`/`stream()` calls (multi‑turn -conversation with server‑side history). `agent.chatId` is the current chat id. - -- `agent.resetSession()` — start a fresh chat on the next turn (reuse one instance for sequential turns; you don't need a new agent per turn). -- `agent.interrupt({ signal? })` — interrupt an in‑flight generation. -- `agent.archive({ signal? })` — archive the underlying chat (cleanup; see [Cleanup](#cleanup)). -- `agent.listModels()` — list the organization's model configs, so you don't have to guess the `model` hint. -- Resume a prior chat: `new CoderAgent({ …, chatId: "…" })` — optionally with - `lastSeenMessageId` (the persisted resume cursor, read from - `agent.lastSeenMessageId`) to skip the resumed turn's pre‑prompt history - probe. See [Durable workflows](#durable-workflows-persist-resume-recover) - for the full resumption how‑to. - -Interrupting is asynchronous on the server: `interrupt()` resolves as soon as the -interrupt is acknowledged, and the run keeps winding down for a few seconds -afterwards. The client‑level `client.interruptChat(chatId, { wait: true })` sends -`?wait=true` to ask the server to hold the response until the run has stopped — -current Coder servers ignore the unknown parameter and still return immediately, -so confirm completion via the event stream (e.g. [`watchChats`](#watching-chats)) -rather than relying on it. - -A single instance is **single‑flight** — don't run concurrent generations against it. For concurrency, use one instance per session (and see [Workspaces & quota](#workspaces--quota)). +One `CoderAgent` instance maps to one chat ("session") on the Coder server. The +chat is created on the first turn and reused by later `generate()`/`stream()` +calls, with history kept server‑side. `agent.chatId` is the current chat id. + +| Call | What it does | +| ------------------------------------ | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | +| `agent.resetSession()` | Start a fresh chat on the next turn. Reuse one instance for sequential turns; you don't need a new agent per turn. | +| `agent.interrupt({ signal? })` | Interrupt an in‑flight generation. | +| `agent.archive({ signal? })` | Archive the underlying chat ([Cleanup](#cleanup)). | +| `agent.listModels()` | List the organization's model configs, so you don't have to guess the `model` hint. | +| `new CoderAgent({ …, chatId: "…" })` | Resume a prior chat. Optionally pass `lastSeenMessageId` (the persisted resume cursor, read from `agent.lastSeenMessageId`) to skip the resumed turn's pre‑prompt history probe. Full how‑to: [Durable workflows](./docs/durable-workflows.md). | + +**One generation at a time.** A single instance is **single‑flight**: don't run +concurrent generations against it. For concurrency, use one instance per session +(and see [Workspaces & quota](./docs/workspaces-and-quota.md)). + +**Interrupting is asynchronous on the server.** `interrupt()` resolves as soon +as the interrupt is acknowledged; the run keeps winding down for a few seconds +afterwards. The client‑level `client.interruptChat(chatId, { wait: true })` +sends `?wait=true` to ask the server to hold the response until the run has +stopped, but current Coder servers ignore the unknown parameter and still return +immediately. Confirm completion via the event stream (e.g. +[`watchChats`](#watching-chats)) instead. ## Rehydrating chat history -Chat history lives on the server. To render an existing chat in a UI (e.g. after -a reload), fetch its messages with the `CoderChatClient` (`agent.client`, or one -you construct — see [Auth](#auth)) and convert them with -`chatMessagesToUIMessages` — the mapping mirrors what a live‑streamed transcript -of the same turn looks like: +To render an existing chat in a UI (e.g. after a reload), fetch its messages with +a `CoderChatClient` (`agent.client`, or one you construct — see [Auth](#auth)) +and convert them with `chatMessagesToUIMessages`. The result mirrors a +live‑streamed transcript of the same turn: ```ts import { chatMessagesToUIMessages } from "@coder/ai-sdk-agent"; @@ -280,22 +292,21 @@ const uiMessages = chatMessagesToUIMessages(messages); // e.g. in React: useChat({ messages: uiMessages }) ``` -The converter sorts by message id, so the endpoint's newest‑first default page -order (and any pagination order) is safe to pass straight in — `useChat` always -receives a chronological transcript. - -Tool calls become `dynamic-tool` parts with their results folded in, `source` -parts become `source-url` parts, and unknown part kinds are skipped silently, so -history written by newer Coder servers degrades gracefully. One caveat: history -does not record which tool names were client (`ToolSet`) tools, so _every_ tool -call rehydrates as `dynamic-tool` — live, client tools stream as statically -typed `tool-{name}` parts. Render tools by name (ai's -`isToolOrDynamicToolUIPart` and `getToolOrDynamicToolName`) rather than by -exact `part.type` and the difference disappears. Persisted `file` -parts carry only a `file_id` (no bytes, usually no URL), so pass a `fileUrl` -resolver to keep attachments visible — download the bytes with -`client.getChatFile(fileId)` and return a data:/object/proxy URL; parts that end -up without a URL are skipped: +- **Any page order works.** The converter sorts by message id, so the endpoint's + newest‑first default (and any pagination order) is safe to pass straight in. + `useChat` always receives a chronological transcript. +- **Mapping.** Tool calls become `dynamic-tool` parts with their results folded + in, and `source` parts become `source-url` parts. Unknown part kinds are skipped + silently, so history written by newer Coder servers degrades gracefully. +- **Every tool call rehydrates as `dynamic-tool`.** History doesn't record which + tool names were client (`ToolSet`) tools; live, client tools stream as + statically typed `tool-{name}` parts. Render tools by name (ai's + `isToolOrDynamicToolUIPart` and `getToolOrDynamicToolName`), not by exact + `part.type`, and the difference disappears. +- **Files need a `fileUrl` resolver.** Persisted `file` parts carry only a + `file_id` (no bytes, usually no URL). Download the bytes with + `client.getChatFile(fileId)` and return a data:/object/proxy URL. Parts left + without a URL are skipped: ```ts chatMessagesToUIMessages(messages, { @@ -306,8 +317,8 @@ chatMessagesToUIMessages(messages, { ## Watching chats `client.watchChats({ signal })` yields lifecycle events (status/title changes, -creation, deletion, …) for **every chat visible to the authenticated user** as an -async iterable, backed by the `/api/v2/chats/watch` WebSocket: +creation, deletion, …) for **every chat visible to the authenticated user**. It +is an async iterable backed by the `/api/v2/chats/watch` WebSocket: ```ts for await (const event of client.watchChats({ signal })) { @@ -315,196 +326,96 @@ for await (const event of client.watchChats({ signal })) { } ``` -After [prefix selection](#api-compatibility) (whose preflight errors propagate), -this is a long‑lived subscription: dropped connections are redialed automatically -with exponential backoff (1s doubling to a 30s cap, reset once an event arrives). -Aborting the signal or ending iteration tears down the reader. A 4xx upgrade -rejection throws a terminal `CoderApiError` — bad/expired token, or an older -Coder server without the endpoint (404). For custom plumbing (own client, -browser sockets), the standalone +- **Long‑lived.** After [prefix selection](#api-compatibility) (whose preflight + errors propagate), dropped connections are redialed automatically with + exponential backoff: 1s doubling to a 30s cap, reset once an event arrives. +- **Teardown.** Aborting the signal or ending iteration tears down the reader. +- **Terminal errors.** A 4xx upgrade rejection throws a terminal + `CoderApiError`: a bad/expired token, or an older Coder server without the + endpoint (404). + +
+Custom plumbing: watchChatEvents + +For custom plumbing (own client, browser sockets), the standalone `watchChatEvents({ baseUrl, token, signal, webSocketFactory })` export defaults -to the stable prefix without preflighting; use `CoderChatClient` for automatic +to the stable prefix without preflighting. Use `CoderChatClient` for automatic prefix selection. -## Observability +
-`onTransportEvent` receives typed transport events — HTTP exchanges, the -per‑chat stream's WebSocket lifecycle, and turn‑segment boundaries — so timing -and tracing need no `fetch`/`webSocketFactory` wrapping and no re‑parsing of -stream frames. Pass it on `CoderAgentSettings` (it reaches both the client and -the model), on `CoderChatClientOptions`, or on `CoderLanguageModelConfig`: - -```ts -import { CoderAgent, type CoderTransportEvent } from "@coder/ai-sdk-agent"; - -const events: CoderTransportEvent[] = []; -const agent = new CoderAgent({ - baseUrl, - token, - organizationId, - onTransportEvent: (ev) => events.push(ev), -}); +## Observability -await agent.generate({ prompt: "…" }); - -// e.g. attribute where the turn spent its time: -for (const ev of events) { - if (ev.type === "http:response") - console.log( - `${ev.op}: ${ev.method} ${ev.path} → ${ev.status} in ${ev.durationMs.toFixed(0)}ms`, - ); - if (ev.type === "ws:event" && ev.event.type === "action_required") - console.log(`tool calls arrived at +${ev.timestamp - events[0]!.timestamp}ms`); - if (ev.type === "segment:settle") - console.log(`segment ${ev.segment}: ${ev.status} in ${ev.durationMs.toFixed(0)}ms`); -} -``` +Pass `onTransportEvent` to receive typed events for HTTP exchanges, the per‑chat +stream's WebSocket lifecycle, and turn‑segment boundaries. You get timing and +tracing without wrapping `fetch`/`webSocketFactory` or re‑parsing stream frames. -`CoderTransportEvent` is a discriminated union on `type`. Every event carries -`timestamp` (`Date.now()` at observation — comparable to server‑side timestamps -such as a message's `created_at`, for delivery‑lag measurements): - -| event | when | payload (besides `timestamp`) | -| ---------------- | --------------------------------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | -| `http:request` | a REST request is sent | `id` (correlates the pair), `op` (the client operation, e.g. `"createChatMessage"`), `method`, `path` | -| `http:response` | response headers arrive (incl. non‑2xx, `ok:false`) | `id`, `op`, `method`, `path`, `status`, `ok`, `durationMs` | -| `http:error` | the fetch itself rejects (network failure, abort) | `id`, `op`, `method`, `path`, `message`, `durationMs` | -| `ws:dial` | a stream connection attempt starts | `chatId`, `reader` (identifies the `streamChatEvents` call), `attempt` (1‑based per reader, increments per redial), `url` | -| `ws:open` | the WebSocket handshake completes | `chatId`, `reader`, `attempt` | -| `ws:event` | a decoded stream event arrives | `chatId`, `reader`, `attempt`, `event` (the decoded `ChatStreamEvent`, by reference — don't mutate), `forwarded` (the reader's replay verdict) | -| `ws:close` | the connection ends (exactly one per dial) | `chatId`, `reader`, `attempt`, `code`/`reason` when the server/network closed it; absent when the reader closed it (settle, teardown, redial) | -| `ws:error` | a socket error or unparseable frame | `chatId`, `reader`, `attempt`, `message` | -| `ws:redial` | a dropped connection is about to be redialed | `chatId`, `reader`, `attempt` (the ended connection), `consecutiveFailures`, `maxConsecutiveFailures`, `backoffMs` | -| `segment:start` | a turn segment (one model round‑trip) starts | `segment` (1‑based per model instance), `chatId` (absent before the first turn creates the chat) | -| `segment:settle` | the segment ends (exactly one per start) | `segment`, `chatId`, `reader` (the reader that served the segment; absent if none was acquired), `durationMs`, and: `status` + `finishReason` on a clean settle, `error` (`{name, message}`, plus `status` if the run still settled terminally) on failure, neither on a teardown | - -Semantics worth knowing: - -- **Isolation** — exceptions thrown by the handler are swallowed; they can never - alter transport behavior or a turn's outcome. -- **Zero overhead** — without a handler, no event objects are allocated and no - extra socket listeners are registered. -- **No secrets** — events carry no headers and no tokens (auth travels in the - `Coder-Session-Token` header, which is deliberately excluded); `path`/`url` - never contain credentials. -- `http:*` events name their operation: `op` is the public `CoderChatClient` - method performing the exchange (`"createChat"`, `"createChatMessage"`, - `"getMessages"`, `"submitToolResults"`, … — the `CoderClientOperation` - union), so per‑operation classification never has to reverse‑engineer - `path`. `method`/`path` stay for generic consumers; `archiveChat` stamps its - own `op` even though it issues the same `PATCH` as `updateChat`. WebSocket - prefix-selection preflights emit the existing `http:*` event kinds with - `op: "streamEvents"` or `op: "watchChats"` — no new event kinds. -- `ws:event` fires at arrival: after a redial, chatd's replay of the - in‑progress episode is visible here (correlate with `reader`/`attempt`), - stamped with the reader's own replay verdict — `forwarded: false` exactly on - the duplicate deltas the reader suppresses from the turn, so subscribers - never re‑derive the episode filter. -- `forwarded` does **not** reflect snapshot dedup: repeated or revised - `message` snapshots are always `forwarded: true`, because reconciling them - is deliberately the consumer's job past the transport layer — - `TurnTranslator`'s per‑message ledger decides what a revision re‑emits, and - that disposition is not stamped on transport frames. Use `ws:event` for - span pairing and replay accounting; subscribers needing content fidelity - must consume model output (or `TurnTranslator`), not transport frames. -- Every `ws:*` event carries `reader` — a monotonic id for the - `streamChatEvents` call (reader) behind the connection, allocated from one - process‑wide counter so it stays unique across model and client instances. - `attempt` restarts at 1 per reader, so identify a connection as - `(chatId, reader, attempt)`, never `(chatId, attempt)` alone: a client‑tool - pause the caller abandons is closed fire‑and‑forget when the next turn dials - its replacement, and the superseded reader's late `ws:close` (or a raced‑in - frame) can emit after the new reader's `ws:dial` — the reader id is what - tells them apart. `segment:settle` names the reader that served the segment; - `segment:start` predates stream acquisition and carries none. -- A multi‑step turn that drives client tools emits one `segment:start`/ - `segment:settle` pair per round‑trip, all riding **one** `ws:dial`ed - connection — the stream is retained across `requires_action` pauses. A pause - settles with `status: "requires_action"`, `finishReason: "tool-calls"`; the - final settle carries the terminal status (`waiting`/`completed`/`error`). -- `ws:*` events cover the per‑chat `/stream` reader (turn transport). The - `watchChats` subscription is not instrumented. -- With a pre‑built `client` in `CoderAgentSettings`, HTTP/WS events come from - the hook given to **that client's** options; the agent‑level hook then only - receives `segment:*` events. +Handler exceptions are swallowed, there is zero overhead without a handler, and +events carry no headers or tokens. Example, event reference, and semantics: +[docs/observability.md](./docs/observability.md). ## Timeouts & cancellation -Pass an `abortSignal` to `generate()`/`stream()` to cancel a turn. Aborting -**interrupts the server‑side run** (not just the local socket), so the chat stops -generating and releases its resources instead of running on, orphaned. Tearing -down a `stream()` early (cancelling the stream) interrupts the run too. +**Cancel a turn** by passing an `abortSignal` to `generate()`/`stream()`. +Aborting **interrupts the server‑side run**, not just the local socket, so the +chat stops generating and releases its resources instead of running on, +orphaned. Tearing down a `stream()` early (cancelling the stream) interrupts the +run too. -For a hard ceiling, set `requestTimeoutMs`. If a segment runs longer (e.g. the -server is wedged, or a workspace can't be scheduled), the run is interrupted and -the call rejects with a retryable `CoderChatError` (`kind: "timeout"`) instead of -hanging: +**Bound each segment** with `requestTimeoutMs`. If a segment runs longer (e.g. +the server is wedged, or a workspace can't be scheduled), the run is interrupted +and the call rejects with a retryable `CoderChatError` (`kind: "timeout"`) +instead of hanging: ```ts const agent = new CoderAgent({ /* … */ requestTimeoutMs: 120_000 }); ``` -`requestTimeoutMs` bounds **each server segment** — one model round‑trip until it -settles or pauses for a client tool. A multi‑step `generate()` that drives client -tools runs several segments, so it bounds each one, not the whole call. To cap the -**total** wall‑clock of a multi‑step call, pass a deadline as the signal instead: +A segment is one model round‑trip, until it settles or pauses for a client tool. +A multi‑step `generate()` that drives client tools runs several segments, so +`requestTimeoutMs` bounds each one, not the whole call. + +**Cap total wall‑clock** of a multi‑step call with a deadline signal instead: ```ts await agent.generate({ prompt: "…", abortSignal: AbortSignal.timeout(120_000) }); ``` -If the event stream drops mid‑turn, the agent redials it automatically with -exponential backoff, replaying the turn's events from its starting cursor and -deduplicating them on receipt — the server keeps generating during the gap, so -a transient drop costs nothing and the run is **not** interrupted. Only when the stream cannot be re‑established (several -consecutive failed attempts, ~15s) is the server run interrupted and the call -rejected with a `CoderStreamError` — an AI SDK `APICallError`. When the failed -turn had just created its chat AND had no external effects a replay would -repeat — no `workspaceId`, no `mcpServerIds`, and no freshly uploaded inline -attachments (pre‑uploaded `fileId` references are fine) — the dead session is -discarded and the error is `isRetryable: true`, so `generate()` calls with -`maxRetries` set retry the whole turn on a fresh chat automatically. Otherwise -the error is `isRetryable: false` and retrying is the caller's deliberate -decision: on a chat with prior state (resumed sessions, later turns, -tool‑result segments) a re‑invocation would resubmit the same prompt as a new -user turn, workspace/MCP tools may already have executed side effects, and -inline attachments would upload again. (For `stream()`, a mid‑stream failure -surfaces on the stream itself, outside the SDK's retry wrapper — handle it in -your consumption loop.) -A non‑transient 4xx upgrade rejection (bad/expired token, -deleted chat) fails fast with a `CoderApiError` instead of retrying; 408/425/429 -consume the redial budget like any other transient failure. +**Stream drops heal themselves.** If the event stream drops mid‑turn, the agent +redials it automatically with exponential backoff, replays the turn's events from +its starting cursor, and deduplicates them on receipt. The server keeps +generating during the gap, so a transient drop costs nothing and the run is +**not** interrupted. + +Only when the stream cannot be re‑established (several consecutive failed +attempts, ~15s) is the server run interrupted and the call rejected with a +`CoderStreamError`, an AI SDK `APICallError`: + +| `isRetryable` | When | Then | +| ------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | --------------------------------------------------------------------------------------------------------------------------- | +| `true` | The failed turn had just created its chat **and** had no external effects a replay would repeat: no `workspaceId`, no `mcpServerIds`, and no freshly uploaded inline attachments (pre‑uploaded `fileId` references are fine) | The dead session is discarded. `generate()` calls with `maxRetries` set retry the whole turn on a fresh chat automatically. | +| `false` | Otherwise | Retrying is the caller's deliberate decision. | + +
+Why the other cases aren't retryable + +On a chat with prior state (resumed sessions, later turns, tool‑result +segments), a re‑invocation would resubmit the same prompt as a new user turn. +Workspace/MCP tools may already have executed side effects, and inline +attachments would upload again. + +
+ +- For `stream()`, a mid‑stream failure surfaces on the stream itself, outside + the SDK's retry wrapper. Handle it in your consumption loop. +- A non‑transient 4xx upgrade rejection (bad/expired token, deleted chat) fails + fast with a `CoderApiError` instead of retrying. 408/425/429 consume the redial + budget like any other transient failure. ## Cleanup -`archive()` soft‑hides the chat (it stays in listings as `archived: true`; there -is no hard delete yet). A freshly interrupted chat keeps winding down server‑side -for a few seconds, during which archiving 409s — `archive()` retries those 409s -(~1s apart, up to ~15s overall; tune with `settleDeadlineMs` / -`settleRetryDelayMs`) and rethrows the last one if the chat never settles. Any -other failure, including your own abort, rethrows immediately. - -`archive()` and `interrupt()` target the session's chat — or, after the session -was dropped (`resetSession()`, or the automatic discard after a fresh‑chat -stream failure — see [Handling errors](#handling-errors)), the **last‑known -chat id** (`agent.lastKnownChatId`), so a stranded chat is still cleaned up -instead of leaking. Generation never uses the last‑known id: a turn after a -drop creates a fresh chat as always. Every chat stranded by an _automatic_ -discard is also recorded on a ledger (`agent.strandedChatIds`, oldest first) — -with `maxRetries` several failed attempts can strand one chat each while only -the final attempt's error surfaces — and one `archive()` retires them all, -oldest first, alongside its primary target. Both methods report what they -acted on instead of silently no‑oping: `archive()` resolves -`{ archived: true, chatId, archivedChatIds }` (each archived id is cleared as -a cleanup target) or `{ archived: false }` when no chat exists at all; -`interrupt()` resolves `{ interrupted: true, chatId }` / -`{ interrupted: false }` the same way. Deliberate abandonment is different: -`resetSession()` does **not** add to the ledger (you may want that chat kept), -so after a manual reset the old chat is targetable only until a new chat -supersedes `lastKnownChatId`. - -To make cleanup ride scope exit instead of a `finally` you have to remember, the -agent is an **async disposable**: +The agent is an **async disposable**, so cleanup can ride scope exit instead of a +`finally` you have to remember: ```ts await using agent = new CoderAgent({/* … */}); @@ -512,34 +423,73 @@ const { text } = await agent.generate({ prompt: "…" }); // agent.interrupt() + agent.archive() run automatically when the scope exits. ``` -Disposal interrupts any in‑flight run, then archives. It is **bounded and never -throws** (~15s overall, best‑effort): disposal errors are swallowed so they can't -mask the scope's own error. Call `archive()` directly when you need guaranteed -cleanup. - -In a request handler that returns before a fire‑and‑forget `archive()` settles, the -archive can be abandoned — `await using` (or an awaited `archive()` in `finally`) -avoids accumulating live chats. +- Disposal interrupts any in‑flight run, then archives. +- Disposal is **bounded and never throws** (~15s overall, best‑effort). Its + errors are swallowed so they can't mask the scope's own error. Call + `archive()` directly when you need guaranteed cleanup. +- In a request handler that returns before a fire‑and‑forget `archive()` + settles, the archive can be abandoned. `await using` (or an awaited + `archive()` in `finally`) avoids accumulating live chats. + +**What `archive()` does:** + +- It soft‑hides the chat: the chat stays in listings as `archived: true`. There + is no hard delete yet. +- A freshly interrupted chat keeps winding down server‑side for a few seconds, + and archiving 409s meanwhile. `archive()` retries those 409s (~1s apart, up to + ~15s overall; tune with `settleDeadlineMs` / `settleRetryDelayMs`) and + rethrows the last one if the chat never settles. +- Any other failure, including your own abort, rethrows immediately. + +**Return values.** Both methods report what they acted on instead of silently +no‑oping: + +| Method | Acted on a chat | No chat exists at all | +| ------------- | ----------------------------------------------------------------------------------------------- | ------------------------ | +| `archive()` | `{ archived: true, chatId, archivedChatIds }` (each archived id is cleared as a cleanup target) | `{ archived: false }` | +| `interrupt()` | `{ interrupted: true, chatId }` | `{ interrupted: false }` | + +
+Which chat archive() and interrupt() target after a session is dropped + +- They target the session's chat. After the session was dropped — + `resetSession()`, or the automatic discard after a fresh‑chat stream failure + (see [Handling errors](#handling-errors)) — they target the **last‑known chat + id** (`agent.lastKnownChatId`), so a stranded chat is still cleaned up instead + of leaking. +- Generation never uses the last‑known id: a turn after a drop creates a fresh + chat as always. +- Every chat stranded by an _automatic_ discard is also recorded on a ledger, + `agent.strandedChatIds` (oldest first). With `maxRetries`, several failed + attempts can strand one chat each while only the final attempt's error + surfaces. One `archive()` retires them all, oldest first, alongside its + primary target. +- Deliberate abandonment is different: `resetSession()` does **not** add to the + ledger (you may want that chat kept). After a manual reset, the old chat is + targetable only until a new chat supersedes `lastKnownChatId`. + +
## Handling errors -All errors extend `CoderAgentError`, except `CoderStreamError` (below). Two -carry structured detail you can branch on: - -- **`CoderApiError`** — an HTTP request failed. Fields: `status`, `method`, `path`, `detail`. -- **`CoderChatError`** — a turn ended in an error, timed out, or lost its stream. Fields: `kind`, `retryable`, `statusCode`, `provider`. -- **`CoderStreamError`** — the event stream dropped and could not be re‑established - within its redial budget. Extends the AI SDK's `APICallError` (not - `CoderAgentError`), so `generate()`'s `maxRetries` machinery recognizes it. - `isRetryable` is `true` only when the failed turn created its chat (the dead - session is discarded, so a retry starts fresh) and had no external effects to - repeat (workspace/MCP tooling, fresh attachment uploads); otherwise it is - `false` — a retry would resubmit the prompt as a new user turn and could - duplicate those effects. A failure mid‑`stream()` surfaces on the stream, - outside the retry wrapper. The last transport failure is in `cause`, and - `chatId` names the chat the failed turn had created or attached to (absent - when it failed before a chat existed) — after the fresh‑chat discard the - chat still exists server‑side, and `archive()` keeps targeting it via +All errors extend `CoderAgentError`, except `CoderStreamError`. + +| Error | Extends | Thrown when | Fields | +| ------------------ | --------------------- | --------------------------------------------------------------------------------- | --------------------------------------------- | +| `CoderApiError` | `CoderAgentError` | An HTTP request failed | `status`, `method`, `path`, `detail` | +| `CoderChatError` | `CoderAgentError` | A turn ended in an error, timed out, or lost its stream | `kind`, `retryable`, `statusCode`, `provider` | +| `CoderStreamError` | AI SDK `APICallError` | The event stream dropped and could not be re‑established within its redial budget | `isRetryable`, `cause`, `chatId` | + +`CoderStreamError` extends `APICallError` so `generate()`'s `maxRetries` +machinery recognizes it: + +- `isRetryable` is `true` only when the failed turn created its chat and had no + external effects to repeat (workspace/MCP tooling, fresh attachment uploads). + Full rules: [Timeouts & cancellation](#timeouts--cancellation). +- `cause` holds the last transport failure. +- `chatId` names the chat the failed turn had created or attached to (absent + when it failed before a chat existed). After the fresh‑chat discard the chat + still exists server‑side, and `archive()` keeps targeting it via `agent.lastKnownChatId` ([Cleanup](#cleanup)). ```ts @@ -558,37 +508,47 @@ try { } ``` -`maxRetries` defaults to `0`: this agent owns server‑side chat state, so an +**`maxRetries` defaults to `0`.** This agent owns server‑side chat state, so an SDK‑level retry could duplicate a turn. Prefer catching `retryable` errors and retrying the whole step deliberately. ## Usage & cost -Results carry normalized token usage in `usage`. A chatd turn runs several -model steps server‑side (one per server tool round), each reporting its own -usage — the SDK **sums every step**, and the AI SDK adds up the steps of a -turn that paused for client tools, so `result.usage` reflects what the whole -turn actually consumed. `inputTokens` is the full prompt size: Coder -normalizes the wire `input_tokens` to the _uncached_ count (cache reads/writes -are separate fields), so the SDK adds them back into the total and exposes the -split via `inputTokenDetails` (`noCacheTokens`, `cacheReadTokens`, -`cacheWriteTokens`). - -The snake_case wire usage lives **per step** at `result.steps[i].usage.raw` -(the AI SDK does not carry `raw` onto the summed `result.usage`) for fields -the normalized shape has no slot for (`context_limit`, cost, runtime, and any -newer wire fields, which pass through newest‑value‑wins). `raw` keeps the wire -convention (`input_tokens` = uncached only) with counters summed over that -step's server‑side model steps and `context_limit` from the newest one — so -don't divide `raw`'s summed counters by `context_limit` to estimate context -fullness; they are turn consumption, not a prompt‑size snapshot. When the -server reports them, `total_cost_micros` (micro‑USD) and `total_runtime_ms` -are mirrored the same way under each step's `providerMetadata.coder` -(`result.providerMetadata` reflects only the final step — sum -`result.steps[*].providerMetadata.coder` for whole‑turn cost when client tools -ran). Both are **absence‑tolerant mirrors**: on servers that don't send them -(cost is otherwise only on the aggregate cost endpoints, -`/api/v2/chats/cost/*`), nothing is emitted. +`result.usage` reflects what the whole turn actually consumed: + +- A chatd turn runs several model steps server‑side (one per server tool round), + each reporting its own usage. The SDK **sums every step**, and the AI SDK adds + up the steps of a turn that paused for client tools. +- `inputTokens` is the full prompt size. Coder normalizes the wire + `input_tokens` to the _uncached_ count (cache reads/writes are separate + fields), so the SDK adds them back into the total and exposes the split via + `inputTokenDetails` (`noCacheTokens`, `cacheReadTokens`, `cacheWriteTokens`). + +**Cost and runtime.** When the server reports them, `total_cost_micros` +(micro‑USD) and `total_runtime_ms` are mirrored under each step's +`providerMetadata.coder`. + +- `result.providerMetadata` reflects only the final step. Sum + `result.steps[*].providerMetadata.coder` for whole‑turn cost when client tools + ran. +- Both are absence‑tolerant mirrors: on servers that don't send them, nothing is + emitted. Cost is otherwise only on the aggregate cost endpoints + (`/api/v2/chats/cost/*`). + +
+Raw wire usage per step (steps[i].usage.raw) + +- The snake_case wire usage lives **per step** at `result.steps[i].usage.raw`. + The AI SDK does not carry `raw` onto the summed `result.usage`. +- Use it for fields the normalized shape has no slot for: `context_limit`, cost, + runtime, and any newer wire fields (which pass through newest‑value‑wins). +- `raw` keeps the wire convention (`input_tokens` = uncached only). Its counters + are summed over that step's server‑side model steps, with `context_limit` from + the newest one. +- So don't divide `raw`'s summed counters by `context_limit` to estimate context + fullness. They are turn consumption, not a prompt‑size snapshot. + +
Forward usage to a UI via message metadata: @@ -604,123 +564,34 @@ return result.toUIMessageStream({ ## Sources -Model configs with web search enabled emit `source` parts. These flow through to -`result.sources` and, in UI message streams, `source-url` parts (pass -`sendSources: true` to `toUIMessageStream` — the AI SDK omits them by default). +Model configs with web search enabled emit `source` parts. They flow through to +`result.sources` and, in UI message streams, to `source-url` parts. Pass +`sendSources: true` to `toUIMessageStream`; the AI SDK omits them by default. Earlier releases dropped them. ## Structured output Coder Agents has no server‑side `response_format`, so `CoderAgent` cannot -constrain what the model **says** to a JSON schema — a `responseFormat` / -`experimental_output` request emits a warning and is best‑effort at most. Pick -by what the step needs: +constrain what the model **says** to a JSON schema. A `responseFormat` / +`experimental_output` request emits a warning and is best‑effort at most. - **Pure text‑in / JSON‑out, no server‑side tools** → use - **[`@coder/ai-sdk-provider`](../provider)** with `generateObject` / + [`@coder/ai-sdk-provider`](../provider) with `generateObject` / `Output.object` (schema‑constrained; requires AI Gateway on the deployment). - **The answer must come out of an agent run** (server‑side tools, MCP, a - workspace) → use the **`structured_output` tool pattern** below. What the - model _says_ isn't schema‑constrained, but what it passes **into a tool** is - typed — so have it submit its answer by _calling a tool_ whose `inputSchema` - is your Zod schema. The answer arrives as the tool call's typed `input`; no - fishing JSON out of prose. - -```ts -import { stepCountIs, tool } from "ai"; -import { z } from "zod"; - -const Answer = z.object({ severity: z.enum(["critical", "major", "minor"]), summary: z.string() }); - -const agent = new CoderAgent({ - /* … */ - instructions: "… Submit your final answer by calling the structured_output tool exactly once.", - tools: { - structured_output: tool({ - description: - "Submit your final structured answer as JSON. Call this exactly once, when your work is complete.", - inputSchema: Answer, // your schema IS the tool's input schema - // Ack instead of stopping the turn: the model finishes naturally and can - // wind down anything it still has running (dev servers, watchers, …). - execute: async () => - "Output received. Wind down and end your turn. Do not call structured_output again.", - }), - }, - stopWhen: stepCountIs(6), // happy path is 2 steps: file + ack, wind down -}); - -const result = await agent.generate({ prompt: "…" }); -// toolCalls only holds the LAST step's calls — scan all steps. Take the last call -// that VALIDATES: a schema-invalid re-file must not shadow a valid answer (rule 2). -const filed = result.steps - .flatMap((s) => s.toolCalls) - .filter((c) => c.toolName === "structured_output"); -let answer: z.infer | undefined; -for (const call of filed.reverse()) { - const parsed = Answer.safeParse(call.input); - if (parsed.success) { - answer = parsed.data; // typed: { severity: "critical" | "major" | "minor"; summary: string } - break; - } -} -if (answer === undefined) - throw new Error("no valid structured_output call — nudge once on an idle chat (rule 3)"); -``` + workspace) → have the model submit its answer by _calling a tool_ whose + `inputSchema` is your Zod schema. The answer arrives as the tool call's typed + `input`. -Rules that keep it robust — each guards against a failure mode observed live: - -1. **Don't force `toolChoice`, don't stop on the call.** `toolChoice` is - construction‑time and applies to _every_ segment, so after the ack it would - force the tool again and again up to the step ceiling (and it blocks any - other tools the step needs). A `hasToolCall` stop is worse: the server only - receives a client tool result as a side effect of the _next_ loop segment, - so ending the loop on the call strands the chat in `requires_action` — - follow‑up messages queue forever and `archive()` 409s. Instructions plus the - tool's own description are enough; models file unprompted most of the time. -2. **Validate client‑side.** The schema is not enforced server‑side — - `schema.safeParse` on the tool input is the real gate. (Schema‑invalid calls - that the AI SDK catches in‑loop are automatically answered with a - `tool-error` result the model retries against.) -3. **Nudge at most once, and only an idle chat.** If the turn ends in prose - (`finishReason: "stop"`) without a valid call, send one typed re‑prompt - ("Call the structured_output tool now …"), then fail into your normal error - handling. Never re‑prompt a chat that isn't idle — the message would queue - behind whatever the server is still doing. -4. **Settle a turn that stopped on a tool call.** If the loop stops on a - tool‑call step — e.g. your `stopWhen` ceiling lands exactly on the - `structured_output` call (`finishReason: "tool-calls"`) — the tool results - ran locally but never reached the server. Guard on `agent.chatId` (it is - `undefined` until the first turn creates the chat), then submit the - stranded step's (`result.steps.at(-1)`) locally‑executed client outcomes - directly via - `agent.client.submitToolResults(chatId, { results: [{ tool_call_id, output, is_error }] }, AbortSignal.timeout(8_000))` - before touching the chat again, or it strands as in rule 1. Read the - outcomes off the step's **content parts**: a `tool-result` part is a - success, a `tool-error` part (the tool's `execute` threw) must be submitted - with `is_error: true` — mirroring what the resume path would have sent. If - a pending call has no local outcome (or the submit fails), end the stranded - turn with `agent.client.interruptChat(chatId, AbortSignal.timeout(8_000))` - instead. **Bound every one of these recovery requests with an - `AbortSignal`** — they target a server that may already be stalled, and the - bare `agent.interrupt()` / `agent.archive()` helpers carry no timeout. A - settled chat resumes its wind‑down server‑side for a few seconds, so retry - a 409ing archive (`agent.client.archiveChat(chatId, signal)`, per‑attempt - bound) under a short deadline instead of giving up. - -[`examples/06-structured-output.ts`](./examples/06-structured-output.ts) packages -all four rules into a small copyable helper — `structuredOutput(schema)` returns -`agentOpts` to spread into the constructor plus a typed `ask(agent, prompt)` -that runs the settle + one‑nudge ladder and returns a `z.infer`. -Compose additional client tools through the helper — -`structuredOutput(schema, { tools: { myTool } })` merges them into one ToolSet — -rather than passing `tools:` to the constructor next to the spread, where the -later key silently clobbers the other map. +The tool pattern needs four rules to stay robust (validate client‑side, never +stop on the call, and more). Guide: [docs/structured-output.md](./docs/structured-output.md). +Copyable helper: [`examples/06-structured-output.ts`](./examples/06-structured-output.ts). ## Workspace previews -When the agent is bound to a workspace (the `workspaceId` setting), you can -resolve — and share — the browser URL where a port on that workspace is served, -e.g. the dev server the agent just started: +When the agent is bound to a workspace (the `workspaceId` setting), resolve — and +share — the browser URL where a workspace port is served, e.g. the dev server the +agent just started: ```ts const { url } = await agent.getPreview({ port: 3000 }); @@ -730,185 +601,57 @@ const shared = await agent.sharePreview({ port: 3000, shareLevel: "authenticated // shared.url is now reachable by any logged-in user; shared.shareLevel is the level in effect ``` -Both are built on the stable v2 workspace APIs (workspace lookup + the wildcard -apps host; `sharePreview` adds a port‑share upsert), so they work against old -Coder servers — no experimental endpoints. - -- `getPreview({ port, agentName?, protocol?, signal? })` composes the subdomain - URL. The URL honors the port's current share level — private to the workspace - owner unless shared. `agentName` is optional when the workspace has exactly one - agent (with several, the error lists the candidates); `protocol: "https"` means - the app speaks TLS _inside the workspace_ (it adds the `s` label suffix, - `3000s--…`) and does not affect the browser scheme. -- `sharePreview({ port, shareLevel?, … })` additionally upserts the port's share - level (re‑invoking updates it in place) and returns the level in effect. - `shareLevel` is `"authenticated"` (any logged‑in user; the default), - `"organization"` (members of the workspace's organization; requires a newer - Coder server), or `"public"` (no auth at all — mind what the port serves). +| Method | What it does | +| ------------------------------------------------------ | ------------------------------------------------------------------------------------------------------------------------ | +| `getPreview({ port, agentName?, protocol?, signal? })` | Composes the subdomain URL. The URL honors the port's current share level: private to the workspace owner unless shared. | +| `sharePreview({ port, shareLevel?, … })` | Additionally upserts the port's share level (re‑invoking updates it in place) and returns the level in effect. | + +- Ports below 1000 are rejected up front. Coder subdomain URLs only encode 4–5 + digit ports, so `80--agent--…` would be parsed as an app named "80" and never + resolve; serve the preview on a higher port. +- `agentName` is optional when the workspace has exactly one agent. With + several, the error lists the candidates. +- `protocol: "https"` means the app speaks TLS _inside the workspace_. It adds + the `s` label suffix (`3000s--…`) and does not affect the browser scheme. +- `shareLevel` values: + + | `shareLevel` | Reachable by | + | --------------------------- | ---------------------------------------------------------------------- | + | `"authenticated"` (default) | Any logged‑in user | + | `"organization"` | Members of the workspace's organization; requires a newer Coder server | + | `"public"` | Anyone — no auth at all; mind what the port serves | + Reverting to owner‑only means deleting the share; `"owner"` is not accepted on upsert. -- Clear failures instead of broken URLs: a deployment without a wildcard access - URL (`--wildcard-access-url`) yields an explanatory error, and a server that - predates port sharing (< Coder v2.9) yields a 404 `CoderApiError` saying so. - Ports below 1000 are rejected up front for the same reason — Coder subdomain - URLs only encode 4–5 digit ports, so `80--agent--…` would be parsed as an app - named "80" and never resolve; serve the preview on a higher port. -The preview helpers call non‑chat endpoints, so they need `baseUrl` + `token` -credentials — pass them alongside `client` if you construct one yourself (or -let them default from `CODER_URL`/`CODER_SESSION_TOKEN`). +**Credentials.** The preview helpers call non‑chat endpoints, so they need +`baseUrl` + `token`. Pass them alongside `client` if you construct one yourself +(or let them default from `CODER_URL`/`CODER_SESSION_TOKEN`). + +
+Compatibility and failure cases (wildcard access URL, Coder < v2.9) + +Both helpers are built on the stable v2 workspace APIs (workspace lookup + the +wildcard apps host; `sharePreview` adds a port‑share upsert), so they work +against old Coder servers — no experimental endpoints. + +They fail clearly instead of returning broken URLs: + +- A deployment without a wildcard access URL (`--wildcard-access-url`) yields an + explanatory error. +- A server that predates port sharing (< Coder v2.9) yields a 404 + `CoderApiError` saying so. + +
## Workspaces & quota -A `CoderAgent` is one server‑side chat, and — depending on its configuration and -the deployment — a chat may be backed by a **Coder workspace** that runs its -tools. Workspaces are the scarce resource: a deployment budgets how many an -account may run at once, so **N agents running concurrently can need N -schedulable workspaces.** Past that bound, a turn can sit unscheduled and never -settle. This section is the operational guide for running fleets of agents: -how the binding works, how to size concurrency, what to clean up, and how to -diagnose a chat that is stuck. - -### How a chat binds to a workspace - -- **One chat, at most one workspace, fixed at creation.** `workspaceId` is sent - as `workspace_id` when the chat is created; nothing can rebind it afterwards — - message and update requests carry no workspace field. To move work to another - workspace, start a new agent/chat. -- **This SDK never provisions workspaces.** A `workspaceId` you pass must be an - existing workspace (provision one with - [`@coder/ai-sdk-sandbox`](../sandbox)'s `ensureCoderWorkspace`, the CLI, or - the v2 API). A chat created _without_ `workspaceId` can still come back - workspace‑backed — deployments may assign one server‑side; the SDK reads the - created chat's `workspace_id` and treats the chat as workspace‑backed from - then on (which matters for retry ownership, below). -- **Chat cleanup does not release the workspace.** `archive()` soft‑hides the - chat only; the workspace keeps running until template autostop or an explicit - stop — and stopping releases only stop‑scoped quota; persistent resources - (disks, volumes) keep consuming their cost until the workspace is deleted. - -### Sizing a fleet - -The structural rule: **workspaces that must be running concurrently ≤ -schedulable workspaces.** With one workspace per chat — the common fleet shape, -and what a deployment that auto‑assigns workspaces produces — that means -concurrent chats ≤ schedulable workspaces. Chats explicitly bound to a shared -`workspaceId` count that workspace once, so their concurrency is not -quota‑bound — at the price of sharing one filesystem and tool environment, -which is only acceptable within a single tenant / trust boundary. -What counts as "schedulable" is a deployment property, not an SDK knob — -whichever of these binds first: - -- **Workspace quota (premium deployments).** Templates declare per‑resource - costs; a user's budget is the sum of their groups' quota allowances, enforced - when a workspace build starts or stops. A start that would exceed the budget - **fails the build** (error code `INSUFFICIENT_QUOTA`, "insufficient quota"), - so the turn never gets its workspace — see - [resource quotas](https://coder.com/docs/admin/users/quotas). Note that a - _stopped_ workspace typically still consumes its persistent resources' cost, - so a fleet that only ever stops (never deletes) scratch workspaces converges - on a full budget. -- **Infrastructure.** Without quotas there is no per‑user workspace limit by - default ([workspace lifecycle](https://coder.com/docs/user-guides/workspace-lifecycle)) — - the bound is provisioner throughput and cluster capacity, and exceeding it - looks like slow or failing builds rather than a crisp quota error. - -Practical sizing: - -- Read headroom before fanning out: - `GET /api/v2/organizations/{org}/members/{user}/workspace-quota` returns - `{ "credits_consumed": …, "budget": … }`. Quota is denominated in credits, - not slots: admit another workspace only while `budget − credits_consumed` - covers _that workspace's_ cost (the sum of its template's `daily_cost` - declarations). "Free slots = headroom ÷ cost" only holds for a homogeneous - fleet on one template; with mixed templates, size against each planned - workspace's own cost. -- Keep fan‑out width within that headroom and queue the rest client‑side — an - unschedulable turn does not queue usefully on the server (see - [Preventing stuck turns](#preventing-stuck-turns)). -- Reuse one bound workspace across sequential turns and sessions instead of - provisioning per request — the workspace is the expensive part, the chat is - cheap. Reuse only **within one tenant / trust boundary**: workspace‑bound - agents have file and shell tools, so a reused filesystem carries one - session's artifacts (and secrets) into the next — provision per tenant, or - securely reset a workspace before reassigning it. -- Steps that don't need server‑side tools belong on the - [provider](../provider) — it never touches a workspace. - -### Autostop & cleanup - -Two lifetimes to manage, separately: - -- **Chats** — `archive()` / `await using` every agent ([Cleanup](#cleanup)), or - finished chats keep holding server resources. -- **Workspaces** — rely on template‑level scheduling rather than manual - hygiene: - - **Autostop TTL.** Give fleet templates a default TTL long enough to survive - a normal session (including idle gaps between turns), short enough that a - leaked workspace stops burning running‑cost within hours — without - autostop, a leaked workspace pins its full quota until someone notices. - Note that stopping only releases the quota of resources that go away on - stop; persistent resources (disks, volumes) keep consuming their - `daily_cost`, so a scratch fleet that only ever stops still converges on a - full budget — pair the TTL with dormancy auto‑deletion or explicit - deletion. With [`@coder/ai-sdk-sandbox`](../sandbox), `stopAfter: "8h"` - sets the TTL (`ttl_ms`) at creation. - - **Activity bump** (default 1 h) extends a running workspace's deadline when - Coder detects sessions — check - [what counts as activity](https://coder.com/docs/user-guides/workspace-scheduling) - before assuming an agent's server‑side tool use keeps its workspace alive. - - **Dormancy / failure cleanup** reap abandoned and repeatedly‑failing - workspaces automatically — see - [template scheduling](https://coder.com/docs/admin/templates/managing-templates/schedule). - -### Preventing stuck turns - -The signature failure mode of an over‑committed fleet: the chat is created, the -stream opens, and then — nothing. There is **no distinct "quota exceeded" error -kind on the chat stream**; a chat whose workspace can't be scheduled surfaces -either a generic turn error or, worse, a chat that sits in a non‑terminal -status indefinitely. Defend in this order: - -1. **Set `requestTimeoutMs` — always, in fleets.** It is unbounded by default. - On expiry the call rejects with a `CoderChatError` (`kind: "timeout"`, - `retryable: true`) so your dispatcher gets its slot back, and a server‑side - interrupt is fired **best‑effort** — fire‑and‑forget, and unreachable when - the timeout lands before chat creation returned an id — so the run usually - stops, but is not guaranteed released ([Timeouts](#timeouts--cancellation)). - Pair it with reconciliation: periodically sweep for non‑terminal chats older - than your budget and interrupt/archive them. -2. **Cap total wall‑clock** with `abortSignal: AbortSignal.timeout(…)` — - `requestTimeoutMs` bounds each segment, not a whole multi‑tool call. -3. **Own the retries.** Workspace‑backed turns are **never auto‑retried**: the - SDK marks a stream‑loss error `isRetryable` only when the failed turn - created its chat _and_ had no workspace, no MCP servers, and no fresh - uploads ([Timeouts](#timeouts--cancellation)). Your dispatcher owns the - retry decision — and should re‑check quota headroom first, or it re‑queues - into the same wall. -4. **Watch the fleet.** [`watchChats`](#watching-chats) yields status changes - for every chat **visible to the authenticated user** — with chats spread - across per‑tenant credentials, run one watcher per identity (or watch with - a credential that can see them all). Alert on chats sitting in a - non‑terminal status (e.g. `pending`) longer than your `requestTimeoutMs`. - Events alone aren't a complete monitor: reconnects resubscribe fresh (no - cursor or replay), so a chat already stuck before the watcher started — or - one that transitioned during a gap — never emits an event to start your - timer from. Seed and periodically reconcile against a chat listing - (`GET /api/v2/chats`) instead of trusting the event stream alone. - -### Troubleshooting: unschedulable & stuck chats - -Wire [`onTransportEvent`](#observability) into fleet telemetry — the event -sequence pinpoints _where_ a turn died: - -| symptom (transport events) | likely cause | fix | -| --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | -| `segment:start` and `ws:open` fired, initial status events, then silence — no `message_part`, no `segment:settle` | if the chat's status is `pending` (or its workspace build is `pending`/`failed`): the workspace can't be scheduled — quota exhausted, no free provisioner, failing template. If the status is `running` with a healthy build, it may just be a slow model/tool step — silence alone doesn't prove scheduling | check quota headroom (`workspace-quota` endpoint) and the workspace's build — a quota failure logs `INSUFFICIENT_QUOTA`; reclaim enough _credits_ for the planned build (stopping helps only for stop‑scoped costs — delete workspaces to reclaim persistent costs, or raise the group allowance) and set `requestTimeoutMs` so this fails loudly next time | -| `segment:settle` carries `error: { name: "CoderChatError", message: "…requestTimeoutMs budget…" }` | the per‑segment bound expired — wedged server, slow model, or an unschedulable workspace | inspect the workspace via the v2 API/UI: a `pending`/`failed` build means the row above; `running` means the turn was genuinely slow — raise `requestTimeoutMs` for long tool work | -| repeated `ws:redial` with `consecutiveFailures` climbing toward `maxConsecutiveFailures` (5; backoff 1 s → 2 s → 4 s → 8 s, ≈15 s of redialing without forward progress), then a `CoderStreamError` | the network path to the deployment is failing — not workspace scheduling (the server keeps generating through short gaps) | fix connectivity; mind retry ownership above — on workspace‑backed chats the error is `isRetryable: false`, so the replay decision is yours | -| `segment:settle` with `status: "error"` and an `error` payload | the turn failed server‑side — a provider/model error, a tool failure, or a scheduling/build failure that terminated the turn instead of leaving it pending | the settle event's `error` carries only `{ name, message }` — branch on `kind` / `retryable` / `statusCode` by catching the thrown `CoderChatError`: at the `generate()` call site, or — for `stream()` — around stream consumption, since mid‑stream failures surface on the stream, not from `await agent.stream()` ([Handling errors](#handling-errors)); check the workspace build state to rule scheduling in or out | -| turn settled `status: "requires_action"`, follow‑up messages queue forever | the loop ended on an unanswered client tool call | submit the stranded results or interrupt — see rule 4 under [Structured output](#structured-output); if a crash left the pause behind, reconcile effects first ([Make client tools crash-safe](#make-client-tools-crash-safe)) | -| `archive()` keeps returning 409 and rethrows after ~15 s | the chat never settled server‑side — usually a stuck run still holding its workspace | `interrupt()` with a bounded signal, then re‑archive; if the run stays wedged, stop the workspace itself | +A chat may be backed by a **Coder workspace** that runs its tools, and +workspaces are the scarce resource: **N agents running concurrently can need N +schedulable workspaces**. Past that bound, a turn can sit unscheduled and never +settle. Read [docs/workspaces-and-quota.md](./docs/workspaces-and-quota.md) +before running fleets: how chats bind to workspaces, fleet sizing, autostop and +cleanup, preventing stuck turns, and troubleshooting. ## Configuration @@ -935,27 +678,37 @@ sequence pinpoints _where_ a turn died: | `chatId` | resume an existing chat | | `lastSeenMessageId` | resume cursor for `chatId` (persist `agent.lastSeenMessageId`) — skips the resumed turn's pre‑prompt history probe | -The `model` hint resolves against the organization's model configs -(`GET /api/v2/organizations/{organizationId}/chats/models`, with the provider -type joined from the response's provider descriptors) in order: a config UUID -is used as‑is, then an exact `provider:model` match, an exact model id, a -display‑name substring (case‑insensitive), and finally a model‑id substring. -On older deployments where the organization‑scoped route does not exist yet -(404), resolution falls back once to the legacy deployment‑wide -`/api/experimental/chats/model-configs` listing, independently of chat-prefix -selection. Partial payloads from older/newer servers are tolerated (entries -match on the fields they carry), and an unresolvable hint falls back to the -server's default model instead of failing. Use -`agent.listModels()` to see what's available. +**Model hint resolution.** The `model` hint resolves against the organization's +model configs, first match wins: + +1. a config UUID, used as‑is +2. an exact `provider:model` match +3. an exact model id +4. a display‑name substring (case‑insensitive) +5. a model‑id substring + +An unresolvable hint falls back to the server's default model instead of +failing. Use `agent.listModels()` to see what's available. + +
+Where model configs come from, and older deployments + +- Configs come from + `GET /api/v2/organizations/{organizationId}/chats/models`, with the provider + type joined from the response's provider descriptors. +- On older deployments where the organization‑scoped route does not exist yet + (404), resolution falls back once to the legacy deployment‑wide + `/api/experimental/chats/model-configs` listing, independently of chat‑prefix + selection. +- Partial payloads from older/newer servers are tolerated: entries match on the + fields they carry. + +
### Reasoning effort Set `reasoningEffort` on `CoderAgentSettings` (or `CoderLanguageModelConfig` -when using the model directly). The global scale is `none`, `minimal`, `low`, -`medium`, `high`, `xhigh`, `max`. The exported `ReasoningEffort` type also accepts -other strings for forward compatibility. Model configs returned by -`agent.listModels()` / `client.listModelConfigs(organizationId)` expose the -optional `reasoning_efforts` list of selectable values for each model. +when using the model directly): ```ts import { CoderAgent } from "@coder/ai-sdk-agent"; @@ -970,14 +723,27 @@ console.log(chat.last_reasoning_effort); // string, null, or absent on the wire await agent.archive(); ``` -The setting is independent of the `model` hint. It is sent as `reasoning_effort` -on chat creation and every user-message submission, including submissions to -resumed or busy chats that queue the message. When unset, the request key is -omitted entirely. Tool-result submissions and WebSocket reconnects do not post -user messages and do not send this field. There is no per-call reasoning-effort -override; configure it on the agent/model instance. Older Coder servers that do -not recognize `reasoning_effort` ignore the unknown JSON field, so setting it is -safe but has no effect there. +- **Values.** The global scale is `none`, `minimal`, `low`, `medium`, `high`, + `xhigh`, `max`. The exported `ReasoningEffort` type also accepts other strings + for forward compatibility. +- **Per‑model options.** Model configs returned by `agent.listModels()` / + `client.listModelConfigs(organizationId)` expose the optional + `reasoning_efforts` list of selectable values for each model. +- **Independent of the `model` hint.** +- **No per‑call override.** Configure it on the agent/model instance. + +
+When reasoning_effort is sent, and older servers + +- It is sent as `reasoning_effort` on chat creation and every + user‑message submission, including submissions to resumed or busy chats that + queue the message. When unset, the request key is omitted entirely. + Tool‑result submissions and WebSocket reconnects do not post user messages and + do not send this field. +- Older Coder servers that do not recognize `reasoning_effort` ignore the + unknown JSON field, so setting it is safe but has no effect there. + +
## How it works @@ -989,846 +755,34 @@ CoderAgent (implements ai.Agent) └─ Coder Agents ← runs the agent loop SERVER-side ``` -- One `doStream` call advances the chat until it **settles** (`waiting`/`completed`) or - **pauses** for a client tool (`requires_action`). The SDK loop and the server‑side loop - mesh at the client‑tool boundary, so there's no double loop. -- Streaming text is emitted from `message_part` deltas; every `message` snapshot is then - reconciled against a per‑message emitted‑content ledger, so nothing double‑counts: a - trailing snapshot after deltas is a no‑op, a fast snapshot‑only turn emits in full, a - message that commits while the stream is redialing yields exactly its missing tail, and a - revision that appends to an earlier message yields the appended suffix (rewrites that - can't be expressed as deltas are safely suppressed). - -## Durable workflows: persist, resume, recover - -How to run **one agent session across process boundaries** — queue jobs, -durable‑workflow steps (Vercel Workflow, step functions, Temporal, …), cron -ticks — and survive the crashes, stream drops, and timeouts in between. The -mechanics this how‑to leans on are specified in [Sessions](#sessions), -[Timeouts & cancellation](#timeouts--cancellation), -[Handling errors](#handling-errors), and [Observability](#observability). +One `doStream` call advances the chat until it **settles** (`waiting`/`completed`) +or **pauses** for a client tool (`requires_action`). The SDK loop and the +server‑side loop mesh at the client‑tool boundary, so there's no double loop. -Two facts make the pattern work: +
+How streamed text avoids double‑counting -- **All chat state lives on the Coder server.** Messages, tool activity, the - run itself — none of it is in your process. The only durable thing a - workflow has to carry between steps is **`agent.chatId`: a string** — plus, - optionally, **`agent.lastSeenMessageId`: the resume cursor**, which saves the - resumed turn one serial round‑trip (below). -- **`CoderAgent` can't ride a `fetch`‑shim durability layer** (it talks REST + - WebSocket through its own client), so each turn runs **inside** a durable - step — and the checkpointed chat id is the thread between steps. +Streaming text is emitted from `message_part` deltas. Every `message` snapshot is +then reconciled against a per‑message emitted‑content ledger, so nothing +double‑counts: -Running example: a pipeline whose steps each run as their own job — possibly -on another machine, hours apart, retried after failures. +- a trailing snapshot after deltas is a no‑op; +- a fast snapshot‑only turn emits in full; +- a message that commits while the stream is redialing yields exactly its + missing tail; +- a revision that appends to an earlier message yields the appended suffix; +- rewrites that can't be expressed as deltas are safely suppressed. -### Shape each step: one turn, then checkpoint +
-```ts -import { CoderAgent, type CoderTransportEvent } from "@coder/ai-sdk-agent"; - -interface WorkflowCheckpoint { - chatId: string; - /** The resume cursor — pairs with chatId, never persisted without it. */ - lastSeenMessageId?: number; -} - -// Any durable KV your engine gives you: step state, a job row, a DB table. -declare const checkpoints: { - get(workflowId: string): Promise; - set(workflowId: string, checkpoint: WorkflowCheckpoint): Promise; -}; - -export async function runTurn(workflowId: string, prompt: string): Promise { - const checkpoint = await checkpoints.get(workflowId); // undefined on the first step → the turn creates the chat - const agent = new CoderAgent({ - baseUrl: process.env.CODER_URL!, - token: process.env.CODER_SESSION_TOKEN!, // read per step — never checkpoint or log it - organizationId: process.env.CODER_ORG_ID!, - chatId: checkpoint?.chatId, - // The previous step's cursor lets this step skip its pre-prompt history - // probe; absent (first resume, older checkpoints), the turn seeds it from - // the chat's newest message instead — one extra GET, same result. - lastSeenMessageId: checkpoint?.lastSeenMessageId, - requestTimeoutMs: 300_000, // always bound workflow steps — see below - onTransportEvent: observe, // per-step telemetry — see below - }); - - try { - const { text } = await agent.generate({ - prompt, - // Total wall-clock for the step: requestTimeoutMs bounds each segment, - // not a whole multi-segment (client-tool) turn — see "Bound every step". - abortSignal: AbortSignal.timeout(600_000), - }); - return text; - } finally { - // The durable resume handle — written even when the turn failed: a first - // step that fails AFTER creating its chat (timeout, stream loss) must - // still persist the id, or the retried step orphans a live chat and its - // partial effects. Exception: after an exhausted stream failure on a - // chat this very call created, the SDK has already discarded the dead - // session (agent.chatId is undefined) and nothing is written — the thrown - // CoderStreamError's chatId still names the stranded chat, and - // agent.archive() retires it via agent.lastKnownChatId — see the - // CoderStreamError notes below for what retrying means then. - if (agent.chatId) { - const next: WorkflowCheckpoint = { chatId: agent.chatId }; - // The cursor is optional sugar: persist it when a turn established one - // (a turn that failed before streaming may not have), skip it otherwise. - if (agent.lastSeenMessageId !== undefined) { - next.lastSeenMessageId = agent.lastSeenMessageId; - } - await checkpoints.set(workflowId, next); - } - } -} -``` - -The shape matters: - -- **One turn per step, `generate()` not `stream()`.** The checkpointed unit is - a finished result — and a mid‑`stream()` failure surfaces on the stream, - outside `generate()`'s error contract, which complicates step retry logic. -- **Persist the id, not the instance** (and never the token — read it from - each step's environment). -- **Checkpoint in `finally`, not only on success**, so failed-but-alive chats - stay reachable. What remains is the hard-crash window between chat creation - and the checkpoint write (below). -- **No `await using` here.** The chat outlives the step; disposal archives it - and archiving ends resumability (below). - -### Resume in the next step — any process, any machine - -```ts -// Step 1 — a queue job on machine A: -await runTurn("wf-1042", "Investigate the failing nightly build and propose a fix."); - -// Step 2 — hours later, on machine B: same chat, full server-side history. -await runTurn("wf-1042", "Apply the fix you proposed and summarize what changed."); - -// Final step — cleanup belongs to the workflow, not to each step (see below). -await archiveWorkflowChat("wf-1042"); -``` - -What resuming does (and doesn't do): - -- **Construction is free.** `new CoderAgent({ …, chatId })` performs no I/O — - a bad or archived id fails the _turn_, not the constructor. (The one - exception is local validation of `lastSeenMessageId`: a cursor that isn't a - positive integer, or arrives without its `chatId`, throws immediately.) -- **The first resumed turn seeds its cursor** from the chat's newest message - (one single‑message history probe), so it streams only its own events — - it never re‑absorbs earlier turns' content or usage. Supplying the - checkpointed `lastSeenMessageId` skips that probe: the previous step already - knew the cursor, so the resumed prompt goes out without any pre‑prompt - round‑trip. -- **Re‑supply process‑local configuration.** Client `tools` live in your - process, not on the server, so each step that expects tool calls must pass - them again. The workspace binding is the opposite: fixed at creation, - carried by the chat, unchangeable on resume. -- **Sessions stay single‑flight across the whole system**, not just within one - process: run steps strictly sequentially per chat. A second job posting to - the same chat queues behind the live run and burns its own - `requestTimeoutMs` waiting — it attributes nothing of the live run's output - to itself (its window opens only when its own queued prompt materializes), - and if it times out first, it withdraws its queued prompt rather than - leaving it to run unattended. The live run's turn is guarded symmetrically: - the server starts the queued prompt's run **without ever emitting a - terminal settle for the finished one**, so the SDK settles the finishing - turn the moment the promoted prompt appears on its stream — nothing of the - next turn's output or usage leaks into it. That settle reports - `finishReason: "stop"` as a documented approximation: the wire doesn't - distinguish a run that completed into a promotion from one interrupted by - `busy_behavior: "interrupt"`, so an interrupted turn also settles `"stop"` - (with whatever partial output it committed) rather than surfacing an - interruption. - -### Let the SDK absorb drops — and handle what it re‑throws - -Self‑healed, invisible to the step (the run is **not** interrupted): - -- **Transient stream drops.** The server keeps generating through the gap; the - SDK redials with backoff and replays what it missed — committed messages - past the turn's cursor plus the in‑progress message's deltas — deduplicating - on receipt. A message that committed _while the connection was down_ arrives - as exactly its missing tail. Drops cost latency, never content, and never - duplicate content. -- **Client‑tool pauses ride one connection.** A multi‑segment turn keeps its - stream across `requires_action` pauses, and a drop while your tool executes - redials in the background. -- **A lost tool‑call event.** If a turn pauses for client tools but the pause's - tool‑call event doesn't arrive within ~2 s, the SDK recovers the pending - calls from committed history over REST and the turn continues. - -Surfaced to the step once self‑healing is exhausted: - -```ts -import { CoderApiError, CoderChatError, CoderStreamError } from "@coder/ai-sdk-agent"; - -try { - await runTurn(workflowId, prompt); -} catch (err) { - if (err instanceof CoderChatError && err.retryable) { - // Timeout or transient turn failure. The run was interrupted (best-effort) - // and the chat survives — but a re-run resubmits the prompt as a new user - // turn: retry only steps designed to tolerate that (below). - } else if (err instanceof CoderStreamError) { - // Redial budget exhausted (~15s without forward progress). On a resumed - // chat this is never auto-retried — the retry decision is yours (below). - } else if (err instanceof CoderApiError) { - // Branch on err.status: back off and retry 408/425/429/5xx; fail the workflow - // on the rest (expired token, archived/deleted chat, …). - } - throw err; -} -``` - -- **`CoderChatError` with `retryable: true`** — the segment's - `requestTimeoutMs` expired (`kind: "timeout"`) or the turn failed - transiently (`kind: "stream_closed"`, an upstream 5xx). `retryable` - classifies the _failure_ as transient — it does not make a re‑run free. - Unlike the guarded `CoderStreamError` path below, the session is **not** - discarded: the retried step reloads the checkpointed chat id (the `finally` - checkpoint above is what makes this hold even when the _first_ step fails - after creating its chat) and resubmits the same prompt as a new user turn, - with the aborted attempt's partial output in history and any tool effects - that ran before the failure not undone. The same idempotency judgment as - below applies — retry steps designed to tolerate re‑submission; reconcile - first when they aren't. And don't resubmit _immediately_: the expiry's - server interrupt is asynchronous and best‑effort, so the timed‑out run may - still be committing — run the same interrupt‑and‑wait‑for‑terminal‑status - recovery as after a crash (below) before the retry, or its trailing output - is absorbed into the retried turn. A timed‑out run can also have _finished_ - before that interrupt landed — but because the interrupt was already fired, - the recovery's [result check](#recover-the-result-before-resubmitting) - must treat the attempt as cut short (pin `cutShort` there) rather than - trusting its own 409. -- **`CoderStreamError`** (an AI SDK `APICallError`) — the stream could not be - re‑established. `isRetryable: true` only when the failed turn had just - created its chat _and_ had no external effects a replay would repeat (no - workspace, no MCP servers, no fresh inline uploads); the SDK then discards - the dead session so a retry — `maxRetries` or a re‑run step — starts clean - on a fresh chat. The discard happens for _every_ chat the failed turn - itself created — **including the effectful, `isRetryable: false` case**: - `agent.chatId` is `undefined` and the `finally` checkpoint writes nothing — - but the stranded chat is not nameless: the error's **`chatId`** field names - it, and `interrupt()`/`archive()` keep targeting it via - `agent.lastKnownChatId` ([Cleanup](#cleanup)), so the step's error path can - retire the orphan directly — or checkpoint `err.chatId` for the - reconciliation sweep (below) when its workspace/MCP effects need - reconciling first. A retried step then starts a fresh chat that has _not_ - seen the orphan's effects: gate that retry on the same idempotency judgment - as any re‑submission. **On a resumed chat it is always `isRetryable: false`:** - re‑running the step resubmits the same prompt as a _new user turn_, with the - failed attempt's partial output still in history (usually fine — the model - sees its own aborted attempt), and any workspace/MCP tool effects that ran - before the drop are not undone. That judgment call belongs to your workflow, - which is exactly why it isn't automatic. -- **`CoderApiError`** — an HTTP request failed; every non‑2xx wraps in this - error, so branch on `status` before deciding. Rate limiting and transient - server failures (408, 425, 429, 5xx) deserve backoff and a step retry (with - the same re‑submission caveat as above). Auth failures and archived/deleted - chats (401/403/404, the archived‑chat 400) hit the same wall on every - attempt — fail the workflow. - -### Recover after a crash - -A hard crash (OOM kill, host loss) runs no `finally` blocks. Reason from the -checkpoint: - -- **Crashed with an empty checkpoint** — usually nothing exists server‑side - and the retried step just creates a fresh chat. But an empty checkpoint is - not proof of nothing: a crash or `requestTimeoutMs` expiry while the create - request was _in flight_ can leave a chat the server committed but whose id - never arrived — unreachable even by the SDK's own interrupt. Treat "no - checkpoint" as _up to one unacknowledged chat per attempt may exist_ — not - as a clean slate, and automatic step retries can stack several. The - reconciliation sweep below must therefore retire _every_ matching orphan - (and its workspace/MCP effects, if the deployment auto‑attached any), not - stop at the first. -- **Crashed after creation, before the checkpoint** — the chat is orphaned: - its run keeps generating until it settles on its own — or, if the crash hit - one of your client tools, never settles: it stays paused in - `requires_action`, holding its workspace, invisible to the retried step - (which has no id to interrupt). Either way the retried step starts a fresh - chat, so orphans are found only by the reconciliation sweep from - [Preventing stuck turns](#preventing-stuck-turns) — that sweep is not - optional in this pattern, and it needs a **wider filter** here: the fleet - sweep watches for _non‑terminal_ chats stuck past their budget, but an - orphaned first turn may settle terminally on its own and then sit as a - live, unarchived chat forever. Sweep every chat no checkpoint accounts - for — regardless of status — and interrupt/archive it. (To shrink the window, checkpoint eagerly from the - first `ws:dial` transport event — it carries the chat id as soon as the chat - exists — but then the checkpoint's lifecycle is yours too: the retried step - resumes a chat holding a dead half‑turn, and whenever a `CoderStreamError` - killed a chat this step created — retryable or not: the SDK discards every - such session, but only in memory — you must clear the eagerly checkpointed - id yourself before any retry, or "starts fresh after a discard" silently - turns into resuming the dead chat.) -- **Crashed after the checkpoint** — the chat is resumable, but don't just - resubmit. The crashed attempt's run may still be live server‑side, and a - resubmission then queues behind it: nothing the old run commits is - attributed to the resumed turn (its window opens only at its own queued - prompt's materialization), but the step burns its `requestTimeoutMs` - waiting for the old run to finish first — and a timeout withdraws the - queued prompt again. A crash _inside one of your client tools_ is worse: the - chat is paused in `requires_action`, where new messages wait forever — and - if the tool's external effect committed before the crash, that pause is the - only pending record of it, so reconcile the tool ledger **before** any - interrupt ([Make client tools crash-safe](#make-client-tools-crash-safe)). - Both cases have one remedy — stop what the crash left behind and **wait - until the chat actually settles** before deciding: a client‑tool pause is - reconciled, never interrupted away; anything else is interrupted: - `interrupt()` resolves on acknowledgment while the run keeps winding down - (and committing) for a few more seconds, so poll the chat's status to a - terminal one first. On a chat with no live run the interrupt rejects with a - 409 `CoderApiError`; that's the good case — ignore it: - - ```ts - // Both shared with "Make client tools crash-safe" (below): the ledger - // reconciliation resolves to its `unstarted` — true only when it fell back - // to interrupting — and journals each action write-ahead, so a recovery - // pass that dies mid-recovery is visible — and classifiable — to the next. - declare function reconcileClientTools(): Promise; - // Journal entries are per-ATTEMPT, so the chat id alone cannot scope them: - // a verdict from attempt N must not classify attempt N+1 (double crash), - // and a later step's recovery on the same chat must not inherit this - // one's. Every entry pins the step marker and an attempt count at write - // time; a consult applies an entry only while both still match, so each - // resubmission retires every earlier verdict by itself — there is no - // clear step to forget. - type JournalEntry = { - // "submitting" is written ahead of every send on a checkpointed chat; - // "resumed" / "cut-short" are the ledger reconcile's verdict (both in - // the next sections). - state: "submitting" | "resumed" | "cut-short"; - marker: string; // the step-scoped marker ("Recover the result…", below) - attempt: number; // markerAttempts() at write time - at: number; // write time — bounds the in-flight-send wait below - }; - declare const journal: { - get(chatId: string): Promise; - set(chatId: string, entry: JournalEntry): Promise; - }; - // This step's attempts committed so far: user messages in the converted - // transcript carrying the marker. Steps run sequentially, so they sit - // contiguous at the tail of history — page newest-first (as the recovery - // scan below already does) and stop at the first user message without it. - declare function markerAttempts(): Promise; - - // Bounded: a stalled deployment must fail the step, not hang it. - const deadline = AbortSignal.timeout(15_000); - // Whether an interrupt stopped a live run drives the resubmit decision - // below: a 409 means there was nothing to stop. - let cutShort = true; - let { status } = await agent.client.getChat(agent.chatId!, deadline); - // Recompute what the entry pinned at write time; a mismatch on either - // means it classifies an earlier attempt (or another step's) — dead, skip. - const entry = await journal.get(agent.chatId!); - const attempts = await markerAttempts(); - const journaled = entry?.marker === marker && entry.attempt === attempts ? entry : undefined; - if (status === "requires_action") { - // The crash hit a client tool mid-execution: this pause is the only - // pending record of its committed effects. Never interrupt it away — - // reconcile the ledger instead (below): it submits recorded results - // (reviving the run — nothing was cut short) or interrupts itself only - // when nothing committed. - cutShort = await reconcileClientTools(); - } else if (journaled && journaled.state !== "submitting") { - // A previous recovery pass acted on this attempt, then died before the - // step's result was recorded. Its journaled state is the verdict: - // "resumed" — it revived the run by submitting recorded results; - // interrupting now would cut down a run doing the step's work, and the - // resubmitted prompt would mint NEW tool-call ids, sidestepping the - // idempotency keys. Let it finish; nothing was cut short. - // "cut-short" — it went on to interrupt (an unstarted sibling in the - // batch); the attempt is truncated, so the check below must resubmit. - cutShort = journaled.state === "cut-short"; - if (cutShort) { - // The journal records write-ahead INTENT, not a landed interrupt: the - // pass may have died in between, leaving the revived run working. - // Re-drive the interrupt to make the intent true — a 409 means the - // chat already settled; the cut-short verdict stands either way. - await agent.interrupt({ signal: deadline }).catch((err) => { - if (!(err instanceof CoderApiError && err.status === 409)) throw err; - }); - } - } else { - await agent.interrupt({ signal: deadline }).catch((err) => { - // A 409 proves nothing is stoppable NOW — that means "never cut short" - // only if the probe saw no interrupt already in flight. A probed - // "interrupting" is an earlier interrupt (a timeout's best-effort one, - // a sweeper) already truncating the attempt; this 409 just says it - // landed. Keep cutShort = true then. - if (err instanceof CoderApiError && err.status === 409) cutShort = status === "interrupting"; - else throw err; - }); - } - // One thing can still be in flight here: the crashed attempt's own SEND. A - // matching "submitting" entry — its attempt count has not grown — means the - // process died with `createChatMessage` possibly en route, and the server - // can commit it AFTER every read above (the 409 only said nothing was - // running THEN). Sleep out what remains of the entry's commit window — - // requestTimeoutMs past the write is the conservative horizon, and a - // restart has usually consumed it already — so a late-landing turn starts - // before the settle poll below and the next section's negative history - // read is past the window. Then recount: a send that never landed OVER AN - // EXISTING attempt pins cutShort — the step only ever plans a resubmission - // over a cut-short, errored, or unsubmitted attempt, so the write-ahead - // that superseded a "cut-short" verdict still carries its consequence, and - // the scan below cannot mistake the old attempt's truncated tail for a - // finished result. - if (journaled?.state === "submitting") { - await new Promise((r) => setTimeout(r, Math.max(0, journaled.at + 300_000 - Date.now()))); - if ((await markerAttempts()) === journaled.attempt && journaled.attempt > 0) cutShort = true; - } - // The 15 s interrupt bound must not cap what follows: a reconciled pause - // revives a full model/tool continuation. Budget the settle poll — and the - // history reads after it — on the turn scale, like the step itself. - const settle = AbortSignal.timeout(600_000); - do { - ({ status } = await agent.client.getChat(agent.chatId!, settle)); - if (status === "waiting" || status === "completed" || status === "error") break; - // With no accepted interrupt racing to clear it, requires_action is a - // STABLE pause: a run resumed by submitted tool results called a NEW - // client tool. Exit and reconcile the ledger again (below) instead of - // polling out the deadline. - if (status === "requires_action" && !cutShort) break; - await new Promise((r) => setTimeout(r, 1_000)); // acknowledged ≠ settled — wait it out - } while (true); - ``` - - Settled is necessary, not sufficient: a terminal status doesn't say whether - the crashed attempt already _finished the step's work_ — check history - before resubmitting - ([Recover the result before resubmitting](#recover-the-result-before-resubmitting)). - -### Make client tools crash-safe - -Interrupting un‑strands a `requires_action` pause — but if the crashed -process died _between_ a non‑idempotent client tool committing its external -effect (a payment, a deploy, an email) and that result reaching the server, -the pause is the **only pending record of the execution**. Interrupt it away -and the resubmitted prompt is free to call the tool again — and duplicate the -effect. Close the window with a tool‑invocation ledger in the same durable -store as the `chatId` checkpoint, keyed by **chat id + `toolCallId`** — the -server assigns the call id and commits it to history, so both sides of the -reconciliation survive the crash. The chat id must be part of the key (and -of the idempotency key handed to the external system): call ids are only -unique _within_ a chat, and a store shared across workflows must never let -one chat's `done` entry answer — or deduplicate — another chat's call. In -multi‑tenant stores, fold the tenant in too: - -```ts -import { tool } from "ai"; -import { z } from "zod"; -import { CoderAgent } from "@coder/ai-sdk-agent"; - -type LedgerEntry = { state: "committing" } | { state: "done"; output: string }; -declare const ledger: { - get(key: string): Promise; - set(key: string, entry: LedgerEntry): Promise; -}; -declare const payments: { - charge(amountCents: number, opts: { idempotencyKey: string }): Promise<{ receiptId: string }>; -}; - -const ledgerKey = (chatId: string, toolCallId: string) => `${chatId}/${toolCallId}`; -const ChargeArgs = z.object({ amountCents: z.number().int() }); - -// A factory taking a getter, not an id: the chat may not exist until the -// turn creates it, so the id must be read lazily — it is set before any -// tool runs. -const chargeCard = (chatId: () => string) => - tool({ - description: "Charge the customer's card.", - inputSchema: ChargeArgs, // shared with recovery's re-drive dispatch below - execute: async ({ amountCents }, { toolCallId }) => { - const key = ledgerKey(chatId(), toolCallId); - // Write-ahead, BEFORE the effect: a crash between the two writes leaves - // "committing" (ambiguous, but resolvable below); no entry at all proves - // the effect never started. - await ledger.set(key, { state: "committing" }); - // Hand the external system the SAME key as its idempotency key, so - // re-driving this exact call can never double-charge. - const { receiptId } = await payments.charge(amountCents, { idempotencyKey: key }); - await ledger.set(key, { state: "done", output: `charged: receipt ${receiptId}` }); - return `charged: receipt ${receiptId}`; - }, - }); - -// Wire-up: a direct self-reference in the constructor call is circular for -// the type checker — defer it through a box assigned right after. -let self: { chatId?: string } | undefined; -const agent = new CoderAgent({ - /* …base options as in runTurn… */ - tools: { charge_card: chargeCard(() => self!.chatId!) }, -}); -self = agent; -``` - -On crash recovery, when the checkpointed chat is paused in `requires_action`, -reconcile _before_ any interrupt. The pending calls are in committed history — -the same derivation the SDK itself uses to recover a lost `action_required` -event — and the ledger holds the verdict on each: - -```ts -// The step's configured ToolSet names: history does not record which calls -// were client tools (see [Rehydrating chat history]), so — like the SDK's -// own derivation — restrict to the names this step registers; a server/MCP -// call must never reach the ledger as "unstarted". (The recovery scan's -// final-text cut reuses this set — next section.) -const clientTools = new Set(["charge_card"]); - -// Pending = the last assistant message's client (non-provider-executed, -// ToolSet-named) tool calls, minus ids answered by a later message's tool -// result. -const { messages } = await agent.client.getMessages(agent.chatId!, { limit: 200 }, deadline); -const ordered = [...messages].sort((a, b) => a.id - b.id); -const lastAssistant = ordered.findLast((m) => m.role === "assistant"); -const handled = new Set( - ordered - .filter((m) => m.id > (lastAssistant?.id ?? 0)) - .flatMap((m) => m.content ?? []) - .filter((p) => p.type === "tool-result") - .map((p) => p.tool_call_id), -); -const pending = (lastAssistant?.content ?? []).filter( - (p) => - p.type === "tool-call" && - !p.provider_executed && - clientTools.has(p.tool_name!) && - !handled.has(p.tool_call_id), -); - -const results: { tool_call_id: string; output: unknown }[] = []; -let unstarted = false; -for (const call of pending) { - const key = ledgerKey(agent.chatId!, call.tool_call_id!); - const entry = await ledger.get(key); - if (entry?.state === "done") { - // Effect committed, result stranded: submit the record — the pause is - // answered and nothing re-executes. - results.push({ tool_call_id: call.tool_call_id!, output: entry.output }); - } else if (entry) { - // "committing" is ambiguous — the crash hit between the write-ahead and - // "done". Safe to re-drive: the SAME idempotency key makes a committed - // effect replay as a no-op, and the call's args are committed in history. - // Dispatch by tool name — every ledgered tool needs its own re-drive - // path, its args validated exactly as the live loop would have. - if (call.tool_name !== "charge_card") - throw new Error(`no re-drive path for tool ${call.tool_name}`); - const { amountCents } = ChargeArgs.parse(call.args); - const { receiptId } = await payments.charge(amountCents, { idempotencyKey: key }); - const output = `charged: receipt ${receiptId}`; - await ledger.set(key, { state: "done", output }); - results.push({ tool_call_id: call.tool_call_id!, output }); - } else { - unstarted = true; // the tool never ran — interrupting loses nothing - } -} -if (results.length > 0 || unstarted) { - // ONE write-ahead with the batch's FINAL verdict, before any network - // call — a mixed batch journals "cut-short" before even its submission, - // so no crash point can leave a stale "resumed" masking the pending - // interrupt. Every crash then either re-reconciles the stable pause or - // re-drives the journaled intent. Scoped to THIS attempt: a resubmission - // grows the marker count and retires the verdict on its own. - await journal.set(agent.chatId!, { - state: unstarted ? "cut-short" : "resumed", - marker, - attempt: await markerAttempts(), - at: Date.now(), - }); -} -if (results.length > 0) await agent.client.submitToolResults(agent.chatId!, { results }, deadline); -if (unstarted) await agent.client.interruptChat(agent.chatId!, deadline); -// As `reconcileClientTools()` in the settle-wait snippet above: resolve to -// `unstarted` — cutShort is true only when this fell back to interrupting. -``` - -- **Only interrupt when the ledger shows no committed effect.** Submitted - results are committed to chat history, so even when a mixed batch (one call - done, a sibling never started) still ends in an interrupt, the effect's - recorded outcome survives for the resubmitted turn to see. -- **Submitting answers the pause; it does not revive your tool loop.** Once - every pending call is answered, the turn resumes **server‑side** with no - process streaming it. That is why the settle‑wait's reconcile branch never - interrupts a pause — nothing stranded is left to stop, and an interrupt - now would cut down the very run the submission revived — and why the - resolved `unstarted` leaves **`cutShort = false`** after a submission: - nothing was cut short, and the next section's `!cutShort` check must - classify the resumed run's finish as a completed attempt, not resubmit - it. Give the poll a turn‑scale deadline rather than the 15 s interrupt - bound; it exits on `requires_action` (stable here, with no accepted - interrupt clearing it), which means the resumed run called a _new_ tool — - reconcile again; a call with no ledger entry is safe to interrupt. -- **The finished run's output lands only in history** — recover it like any - completed‑but‑unrecorded attempt (next section) instead of resubmitting. - -### Recover the result before resubmitting - -The settle‑wait resolves overlap with a still‑live run, but not the last -crash window: **the server finished the turn, and the process died before the -step's return value was recorded.** History now holds a perfectly good -result; a retried step that blindly resubmits runs the whole turn again — -duplicated output, repeated workspace/MCP/tool effects. This is the -at‑least‑once boundary every durable engine has around external effects, and -the chat itself is the record that narrows it to one honest window (below). -Make attempts matchable first — the server stores the prompt verbatim, so -embed a step‑scoped marker — and journal every send before it leaves the -process: - -```ts -const marker = `[${workflowId}/step-2]`; // stable across this step's retries -// Write-ahead, BEFORE the send — first attempt and recovery resubmit alike. -// A crash before this write proves no send ever started; a crash after it -// leaves a dated entry whose commit window the settle-wait sleeps out. On a -// resubmission this write supersedes the "cut-short" verdict it acts on — -// safely: an unlanded send over an existing attempt makes the settle-wait -// pin cutShort right back. A first turn has no chat id to journal under: its -// send-crash window belongs to the empty-checkpoint sweep -// ("Recover after a crash"). -if (agent.chatId) { - await journal.set(agent.chatId, { - state: "submitting", - marker, - attempt: await markerAttempts(), - at: Date.now(), - }); -} -const { text } = await agent.generate({ prompt: `${marker} ${prompt}` }); -``` - -Then, after the settle‑wait, three states are distinguishable — and only one -of them resubmits: - -```ts -import { chatMessagesToUIMessages } from "@coder/ai-sdk-agent"; - -// A long turn can commit more than a page of messages after the prompt — -// page newest-first (the endpoint's default order) until the turn's user -// message is on hand; "never submitted" must never be concluded from a -// truncated read, or this recovery resubmits exactly the turn it was meant -// to deduplicate. -const fetched = []; -let page = await agent.client.getMessages(agent.chatId!, { limit: 200 }, settle); -fetched.push(...page.messages); -while (!fetched.some((m) => m.role === "user") && page.has_more) { - const oldestId = Math.min(...fetched.map((m) => m.id)); - page = await agent.client.getMessages(agent.chatId!, { before_id: oldestId, limit: 200 }, settle); - fetched.push(...page.messages); -} -const transcript = chatMessagesToUIMessages(fetched); // chronological -const lastUser = transcript.findLast((m) => m.role === "user"); -const submitted = lastUser?.parts.some((p) => p.type === "text" && p.text.includes(marker)); - -if (submitted && !cutShort && (status === "waiting" || status === "completed")) { - // The attempt finished; only the recording was lost. Recover what - // `generate().text` would have returned — the FINAL step's text. Only a - // CLIENT tool ends an AI SDK step, and narration lands on both sides of a - // server-side call, so cut after the last part named in `clientTools` - // (shared with the reconcile above) and join the text that follows. - const parts = transcript - .slice(transcript.indexOf(lastUser!) + 1) - .filter((m) => m.role === "assistant") - .flatMap((m) => m.parts); - return parts - .slice(parts.findLastIndex((p) => p.type === "dynamic-tool" && clientTools.has(p.toolName)) + 1) - .filter((p) => p.type === "text") - .map((p) => p.text) - .join(""); -} -// Resubmit only what never finished: the prompt never reached the server -// (!submitted — sound only behind the write-ahead; see below), the turn -// failed (status "error" — normal retry judgment applies), or the interrupt -// cut a live run short (cutShort — the model sees its own aborted attempt in -// history and continues from it). One exit is neither: a requires_action -// settle goes back to ledger reconciliation (previous section) — never to -// recovery or resubmission. -``` - -- **`cutShort` is the completed/interrupted discriminator — but a 409 only - proves nothing is running _now_.** An interrupted attempt leaves the same - marker and a truncated answer in history, so history alone can't tell - "finished" from "stopped"; what can is whether any interrupt stopped the - run. The inference "409 ⇒ the attempt finished" is therefore sound only - when recovery's interrupt is the _first_ one that could have: true after a - hard crash, false after a `requestTimeoutMs` expiry — the expiry already - fired the SDK's own best‑effort interrupt, which may have stopped the run - before recovery ever looked. Carry the reason recovery ran: when it follows - a known timeout or abort — or a matching journaled `"cut-short"`, or an - unlanded `"submitting"` over an existing attempt (the settle‑wait already - pins both) — pin `cutShort` to `true` and let only the hard‑crash path - trust the 409. -- **`!submitted` races the crashed attempt's own send.** A crash kills the - process, not necessarily its in‑flight `createChatMessage` — the server can - commit that request _after_ an interrupt 409'd and a history read saw no - marker, and a resubmit on that evidence queues the turn twice, duplicating - its workspace/MCP/tool effects. The write‑ahead is what makes the negative - read honest: no matching `"submitting"` entry proves no send ever started, - and a matching one held recovery at the settle‑wait until the entry's - commit horizon passed. That narrows the window; only server‑enforced - submission idempotency could close it, and `POST /chats/{id}/messages` - offers none today. A commit that outlasts even the horizon still lands - under the same marker, so the duplicate is at least visible — in history, - and to the next consult's attempt count. -- **The cut lands at the last _configured client‑tool_ boundary.** History - does not mark which calls were client tools - ([Rehydrating chat history](#rehydrating-chat-history)), so the cut - classifies by name against the reconcile's `clientTools` set — narration - before a trailing _server‑side_ call survives, matching what - `generate().text` aggregates. What name‑matching cannot see: a call - recorded under a name the step no longer configures (a tool renamed - between deploys) reads as server‑side and leaks earlier‑step narration - into the result, and a server tool sharing a configured name still cuts - final‑step narration away. -- **Steps with structured results recover the call's input, not the text.** - A [structured output](#structured-output) step's answer is the - `structured_output` call's typed input, which rehydrates as a - `dynamic-tool` part on the recovered turn - ([Rehydrating chat history](#rehydrating-chat-history)) — and - `structured_output` is one of the step's configured client tools, so the - text join above cuts at it and returns only the ack prose that follows. - Recover the filed call instead — this scan replaces the text join - _inside_ the recovery branch above, reusing its `parts` — validated - client‑side exactly like the live path (rule 2 there — the schema is the - real gate): - - ```ts - // Scan backward to the last call that VALIDATES, as in the live path — a - // schema-invalid re-file must not shadow a valid answer. - const filed = parts - .filter((p) => p.type === "dynamic-tool") - .filter((p) => p.toolName === "structured_output"); - let answer: z.infer | undefined; - for (const call of filed.reverse()) { - const parsed = Answer.safeParse(call.input); - if (parsed.success) { - answer = parsed.data; - break; - } - } - // recovered ⇔ answer !== undefined - ``` - -### Watch turn health from inside the step - -[Transport events](#observability) classify how each segment ended without -parsing errors or stream frames — emit them as step metrics/traces: - -```ts -declare const metrics: { - ok(segment: number, status: string, ms: number): void; - fail(segment: number, error: string, detail: string): void; - teardown(segment: number): void; -}; - -function observe(ev: CoderTransportEvent): void { - if (ev.type === "ws:redial") { - // Early warning: the stream dropped and is reconnecting. The server keeps - // generating — nothing is lost yet; the turn fails if this hits the cap. - console.warn( - `redial ${ev.consecutiveFailures}/${ev.maxConsecutiveFailures}, next in ${ev.backoffMs}ms`, - ); - } else if (ev.type === "segment:settle") { - if (ev.finishReason) metrics.ok(ev.segment, ev.status!, ev.durationMs); - else if (ev.error) metrics.fail(ev.segment, ev.error.name, ev.error.message); - else metrics.teardown(ev.segment); - } -} -``` - -`segment:settle` fires exactly once per `segment:start` and separates the -three endings a workflow cares about: - -- **Clean settle** — `status` + `finishReason` present. `finishReason: -"tool-calls"` (status `requires_action`) is a healthy mid‑turn pause for - your client tools; terminal statuses end the turn. -- **Failure** — `error: { name, message }` present (plus `status` when the - server run itself still settled terminally): server‑side turn errors, - transport failure after redials were exhausted, `requestTimeoutMs` expiry - (`error.name: "CoderChatError"`), and caller aborts — which settle as - failures carrying the abort reason's name, never as teardowns: - `"TimeoutError"` for an `AbortSignal.timeout(…)` deadline (the running - example), `"AbortError"` for an explicit abort. -- **Teardown** — neither: the stream consumer itself cancelled the turn - (`stream()`'s `ReadableStream.cancel()` with no abort of its own) — rare in - `generate()`‑shaped steps. - -The hook is observability‑only — handler exceptions are swallowed and can't -disturb the turn, and an unset hook costs nothing. For monitoring _across_ -steps (chats stuck before a watcher started, fleet‑level sweeps), see -[Preventing stuck turns](#preventing-stuck-turns). - -### Bound every step - -`requestTimeoutMs` arms **one timer per segment** covering everything the -segment does: the REST phase (chat creation, message/tool‑result submission, -uploads), the stream — and, the part that matters here, **any redial backoff -inside the segment**. A segment that spends 12 s reconnecting has 12 s less -budget; redials never reset the clock, so a flaky network can't stretch a step -past its bound. On expiry the call rejects with `CoderChatError` -(`kind: "timeout"`, `retryable: true`) and the server run is interrupted -best‑effort. - -Two workflow‑specific notes: - -- A turn that times out (or crashes) before its chat id arrived leaves nothing - to _resume_ — the checkpoint stays empty and the retried step starts fresh — - but possibly an unacknowledged chat to sweep (see - [Recover after a crash](#recover-after-a-crash)). -- `requestTimeoutMs` bounds each _segment_; a turn driving client tools runs - several. Cap a step's total wall‑clock with - `abortSignal: AbortSignal.timeout(…)` — - [Timeouts & cancellation](#timeouts--cancellation). - -### Archive at the end — and only at the end - -**An archived chat cannot be resumed:** the server rejects new messages on it -(400, `Cannot send messages to an archived chat`). The per‑request habits from -[Cleanup](#cleanup) — `await using`, `archive()` in a `finally` — would -destroy the session from inside an intermediate step, so here the _workflow_ -owns the chat's end of life: archive once, in the final step, and in the -workflow's failure/compensation handler so abandoned runs don't accumulate -live chats. - -```ts -export async function archiveWorkflowChat(workflowId: string): Promise { - const checkpoint = await checkpoints.get(workflowId); - if (!checkpoint) return; // the workflow died before its first turn created a chat - const agent = new CoderAgent({ - baseUrl: process.env.CODER_URL!, - token: process.env.CODER_SESSION_TOKEN!, - organizationId: process.env.CODER_ORG_ID!, - chatId: checkpoint.chatId, - }); - // A crashed step may have left a run live — stop it first, bounded. On an - // already-settled chat the interrupt rejects with a 409; ignore it. - await agent.interrupt({ signal: AbortSignal.timeout(15_000) }).catch(() => {}); - await agent.archive(); // retries 409s while the interrupted run winds down (~15s) -} -``` +## Durable workflows: persist, resume, recover -### Checklist - -- One turn per durable step; `generate()`, not `stream()`. -- Persist `agent.chatId` (a string) — never the instance, never the token. -- Bound every step: `requestTimeoutMs` per segment, an abort deadline for - total wall‑clock. -- Let redials self‑heal; own every retry decision — a re‑run step resubmits - its prompt as a new user turn. -- Before resubmitting after a crash: reconcile the tool ledger before any - interrupt, recover a finished attempt's result from history, and trust - "never submitted" only past the journaled send's commit window. -- Keep concurrent steps' fan‑out within workspace quota — - [Workspaces & quota](#workspaces--quota). -- Steps that don't need server‑side tools (plan / extract / synthesize) are - cheaper and natively structured through - [`@coder/ai-sdk-provider`](../provider) + `generateObject` — no chat, no - workspace, no cleanup. -- Archive in the final step / failure handler — never per step. +Run **one agent session across process boundaries** — queue jobs, +durable‑workflow steps (Vercel Workflow, step functions, Temporal, …), cron +ticks — and survive crashes, stream drops, and timeouts in between. All chat +state lives on the Coder server, so a workflow carries only `agent.chatId` (plus, +optionally, the `agent.lastSeenMessageId` resume cursor) between steps, and +each turn runs inside a durable step. Guide: [docs/durable-workflows.md](./docs/durable-workflows.md). ## Testing @@ -1841,7 +795,8 @@ pnpm check # format check + lint + typecheck (CI gate) pnpm build ``` -End‑to‑end tests run against a live Coder deployment and are opt‑in via env: +End‑to‑end tests run against a live Coder deployment and are opt‑in via env. The +suite creates **new chats only** (no workspaces) and archives them afterward: ```bash CODER_URL=https://dogfood.cdr.dev \ @@ -1849,15 +804,15 @@ CODER_SESSION_TOKEN=$(coder tokens create --name e2e) \ pnpm test:e2e ``` -The e2e suite creates **new chats only** (no workspaces) and archives them afterward. - ## Limitations -- Older deployments use a temporary experimental-route fallback; it is not a - guarantee that every chat feature is available (see [API compatibility](#api-compatibility)). -- Designed for Node (WebSocket via `ws`); a browser build can inject a `webSocketFactory`. -- A v7 `@ai-sdk/harness` adapter (the conceptually exact fit) is a future direction once that - experimental API stabilizes. +- Older deployments use a temporary experimental‑route fallback; it is not a + guarantee that every chat feature is available (see + [API compatibility](#api-compatibility)). +- Designed for Node (WebSocket via `ws`); a browser build can inject a + `webSocketFactory`. +- A v7 `@ai-sdk/harness` adapter (the conceptually exact fit) is a future + direction once that experimental API stabilizes. ## License diff --git a/packages/agent/docs/observability.md b/packages/agent/docs/observability.md new file mode 100644 index 00000000..9fbc1def --- /dev/null +++ b/packages/agent/docs/observability.md @@ -0,0 +1,130 @@ +# Observability: transport events + +`onTransportEvent` receives typed transport events: HTTP exchanges, the per‑chat +stream's WebSocket lifecycle, and turn‑segment boundaries. Use it for timing and +tracing without wrapping `fetch`/`webSocketFactory` or re‑parsing stream frames. + +Pass it on any of: + +- `CoderAgentSettings` — reaches both the client and the model. +- `CoderChatClientOptions` +- `CoderLanguageModelConfig` + +## Example: where did the turn spend its time? + +```ts +import { CoderAgent, type CoderTransportEvent } from "@coder/ai-sdk-agent"; + +const events: CoderTransportEvent[] = []; +const agent = new CoderAgent({ + baseUrl, + token, + organizationId, + onTransportEvent: (ev) => events.push(ev), +}); + +await agent.generate({ prompt: "…" }); + +for (const ev of events) { + if (ev.type === "http:response") + console.log( + `${ev.op}: ${ev.method} ${ev.path} → ${ev.status} in ${ev.durationMs.toFixed(0)}ms`, + ); + if (ev.type === "ws:event" && ev.event.type === "action_required") + console.log(`tool calls arrived at +${ev.timestamp - events[0]!.timestamp}ms`); + if (ev.type === "segment:settle") + console.log(`segment ${ev.segment}: ${ev.status} in ${ev.durationMs.toFixed(0)}ms`); +} +``` + +## Event reference + +`CoderTransportEvent` is a discriminated union on `type`. Every event carries +`timestamp`: `Date.now()` at observation. It is comparable to server‑side +timestamps such as a message's `created_at`, so you can measure delivery lag. + +| event | when | payload (besides `timestamp`) | +| ---------------- | --------------------------------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | +| `http:request` | a REST request is sent | `id` (correlates the pair), `op` (the client operation, e.g. `"createChatMessage"`), `method`, `path` | +| `http:response` | response headers arrive (incl. non‑2xx, `ok:false`) | `id`, `op`, `method`, `path`, `status`, `ok`, `durationMs` | +| `http:error` | the fetch itself rejects (network failure, abort) | `id`, `op`, `method`, `path`, `message`, `durationMs` | +| `ws:dial` | a stream connection attempt starts | `chatId`, `reader` (identifies the `streamChatEvents` call), `attempt` (1‑based per reader, increments per redial), `url` | +| `ws:open` | the WebSocket handshake completes | `chatId`, `reader`, `attempt` | +| `ws:event` | a decoded stream event arrives | `chatId`, `reader`, `attempt`, `event` (the decoded `ChatStreamEvent`, by reference — don't mutate), `forwarded` (the reader's replay verdict) | +| `ws:close` | the connection ends (exactly one per dial) | `chatId`, `reader`, `attempt`, `code`/`reason` when the server/network closed it; absent when the reader closed it (settle, teardown, redial) | +| `ws:error` | a socket error or unparseable frame | `chatId`, `reader`, `attempt`, `message` | +| `ws:redial` | a dropped connection is about to be redialed | `chatId`, `reader`, `attempt` (the ended connection), `consecutiveFailures`, `maxConsecutiveFailures`, `backoffMs` | +| `segment:start` | a turn segment (one model round‑trip) starts | `segment` (1‑based per model instance), `chatId` (absent before the first turn creates the chat) | +| `segment:settle` | the segment ends (exactly one per start) | `segment`, `chatId`, `reader` (the reader that served the segment; absent if none was acquired), `durationMs`, and: `status` + `finishReason` on a clean settle, `error` (`{name, message}`, plus `status` if the run still settled terminally) on failure, neither on a teardown | + +## Guarantees + +- **Isolation** — exceptions thrown by the handler are swallowed. They can never + alter transport behavior or a turn's outcome. +- **Zero overhead** — without a handler, no event objects are allocated and no + extra socket listeners are registered. +- **No secrets** — events carry no headers and no tokens. Auth travels in the + `Coder-Session-Token` header, which is deliberately excluded. `path`/`url` + never contain credentials. + +## Semantics + +### HTTP operation names + +- `op` is the public `CoderChatClient` method performing the exchange + (`"createChat"`, `"createChatMessage"`, `"getMessages"`, + `"submitToolResults"`, … — the `CoderClientOperation` union). Classify per + operation without reverse‑engineering `path`. +- `method`/`path` stay for generic consumers. +- `archiveChat` stamps its own `op`, even though it issues the same `PATCH` as + `updateChat`. +- WebSocket prefix‑selection preflights emit the existing `http:*` event kinds + with `op: "streamEvents"` or `op: "watchChats"`. There are no new event kinds. + +### Replay and `forwarded` + +- `ws:event` fires at arrival. After a redial, chatd's replay of the + in‑progress episode is visible here; correlate with `reader`/`attempt`. +- Each `ws:event` is stamped with the reader's own replay verdict: + `forwarded: false` exactly on the duplicate deltas the reader suppresses from + the turn. Subscribers never re‑derive the episode filter. +- `forwarded` does **not** reflect snapshot dedup. Repeated or revised + `message` snapshots are always `forwarded: true`: reconciling them is + deliberately the consumer's job past the transport layer. `TurnTranslator`'s + per‑message ledger decides what a revision re‑emits, and that disposition is + not stamped on transport frames. +- Use `ws:event` for span pairing and replay accounting. If you need content + fidelity, consume model output (or `TurnTranslator`), not transport frames. + +### Identifying a connection + +Identify a connection as `(chatId, reader, attempt)`, never `(chatId, attempt)` +alone. + +- `reader` is a monotonic id for the `streamChatEvents` call behind the + connection. It comes from one process‑wide counter, so it stays unique across + model and client instances. Every `ws:*` event carries it. +- `attempt` restarts at 1 per reader. +- Why it matters: a client‑tool pause the caller abandons is closed + fire‑and‑forget when the next turn dials its replacement. The superseded + reader's late `ws:close` (or a raced‑in frame) can emit after the new reader's + `ws:dial`. The reader id tells them apart. +- `segment:settle` names the reader that served the segment. `segment:start` + predates stream acquisition and carries none. + +### Multi‑step turns + +A multi‑step turn that drives client tools emits one +`segment:start`/`segment:settle` pair per round‑trip, all riding **one** +`ws:dial`ed connection: the stream is retained across `requires_action` pauses. + +- A pause settles with `status: "requires_action"`, `finishReason: "tool-calls"`. +- The final settle carries the terminal status (`waiting`/`completed`/`error`). + +### Coverage + +- `ws:*` events cover the per‑chat `/stream` reader (turn transport). The + `watchChats` subscription is not instrumented. +- With a pre‑built `client` in `CoderAgentSettings`, HTTP/WS events come from the + hook given to **that client's** options. The agent‑level hook then only + receives `segment:*` events. diff --git a/packages/agent/docs/structured-output.md b/packages/agent/docs/structured-output.md new file mode 100644 index 00000000..8b4f7c45 --- /dev/null +++ b/packages/agent/docs/structured-output.md @@ -0,0 +1,134 @@ +# Structured output from an agent run + +Coder Agents has no server‑side `response_format`. `CoderAgent` cannot constrain +what the model **says** to a JSON schema: a `responseFormat` / +`experimental_output` request emits a warning and is best‑effort at most. + +Pick by what the step needs: + +| The step… | Use | +| ------------------------------------------------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------- | +| is pure text‑in / JSON‑out, with no server‑side tools | [`@coder/ai-sdk-provider`](../../provider) with `generateObject` / `Output.object` — schema‑constrained; requires AI Gateway on the deployment | +| must produce its answer from an agent run (server‑side tools, MCP, a workspace) | the **`structured_output` tool pattern** below | + +## The `structured_output` tool pattern + +What the model _says_ isn't schema‑constrained, but what it passes **into a +tool** is typed. So have it submit its answer by calling a tool whose +`inputSchema` is your Zod schema. The answer arrives as the tool call's typed +`input` — no fishing JSON out of prose. + +```ts +import { CoderAgent } from "@coder/ai-sdk-agent"; +import { stepCountIs, tool } from "ai"; +import { z } from "zod"; + +const Answer = z.object({ severity: z.enum(["critical", "major", "minor"]), summary: z.string() }); + +const agent = new CoderAgent({ + /* … */ + instructions: "… Submit your final answer by calling the structured_output tool exactly once.", + tools: { + structured_output: tool({ + description: + "Submit your final structured answer as JSON. Call this exactly once, when your work is complete.", + inputSchema: Answer, // your schema IS the tool's input schema + // Ack instead of stopping the turn: the model finishes naturally and can + // wind down anything it still has running (dev servers, watchers, …). + execute: async () => + "Output received. Wind down and end your turn. Do not call structured_output again.", + }), + }, + stopWhen: stepCountIs(6), // happy path is 2 steps: file + ack, wind down +}); + +const result = await agent.generate({ prompt: "…" }); +// toolCalls only holds the LAST step's calls — scan all steps. Take the last call +// that VALIDATES: a schema-invalid re-file must not shadow a valid answer (rule 2). +const filed = result.steps + .flatMap((s) => s.toolCalls) + .filter((c) => c.toolName === "structured_output"); +let answer: z.infer | undefined; +for (const call of filed.reverse()) { + const parsed = Answer.safeParse(call.input); + if (parsed.success) { + answer = parsed.data; // typed: { severity: "critical" | "major" | "minor"; summary: string } + break; + } +} +if (answer === undefined) + throw new Error("no valid structured_output call — nudge once on an idle chat (rule 3)"); +``` + +## Rules that keep it robust + +Each rule guards against a failure mode observed live. + +### 1. Don't force `toolChoice`, and don't stop on the call + +- **`toolChoice`** is construction‑time and applies to _every_ segment. After the + ack it would force the tool again and again, up to the step ceiling. It also + blocks any other tools the step needs. +- **A `hasToolCall` stop is worse.** The server only receives a client tool + result as a side effect of the _next_ loop segment. Ending the loop on the + call strands the chat in `requires_action`: follow‑up messages queue forever + and `archive()` 409s. + +Instructions plus the tool's own description are enough; models file unprompted +most of the time. + +### 2. Validate client‑side + +The schema is not enforced server‑side. `schema.safeParse` on the tool input is +the real gate. Schema‑invalid calls that the AI SDK catches in‑loop are +automatically answered with a `tool-error` result the model retries against. + +### 3. Nudge at most once, and only an idle chat + +If the turn ends in prose (`finishReason: "stop"`) without a valid call, send one +typed re‑prompt ("Call the structured_output tool now …"), then fail into your +normal error handling. + +Never re‑prompt a chat that isn't idle: the message would queue behind whatever +the server is still doing. + +### 4. Settle a turn that stopped on a tool call + +If the loop stops on a tool‑call step — e.g. your `stopWhen` ceiling lands +exactly on the `structured_output` call (`finishReason: "tool-calls"`) — the tool +results ran locally but never reached the server. Settle the chat before touching +it again, or it strands as in rule 1: + +1. Guard on `agent.chatId`. It is `undefined` until the first turn creates the + chat. +2. Read the stranded step's (`result.steps.at(-1)`) locally‑executed client + outcomes off its **content parts**: + - a `tool-result` part is a success; + - a `tool-error` part (the tool's `execute` threw) must be submitted with + `is_error: true` — mirroring what the resume path would have sent. +3. Submit them directly: + `agent.client.submitToolResults(chatId, { results: [{ tool_call_id, output, is_error }] }, AbortSignal.timeout(8_000))`. +4. If a pending call has no local outcome (or the submit fails), end the + stranded turn instead with + `agent.client.interruptChat(chatId, AbortSignal.timeout(8_000))`. + +**Bound every one of these recovery requests with an `AbortSignal`.** They +target a server that may already be stalled, and the bare `agent.interrupt()` / +`agent.archive()` helpers carry no timeout. + +A settled chat resumes its wind‑down server‑side for a few seconds. Retry a +409ing archive (`agent.client.archiveChat(chatId, signal)`, per‑attempt bound) +under a short deadline instead of giving up. + +## Copyable helper + +[`examples/06-structured-output.ts`](../examples/06-structured-output.ts) +packages all four rules into a small helper: + +- `structuredOutput(schema)` returns `agentOpts` to spread into the constructor, + plus a typed `ask(agent, prompt)` that runs the settle + one‑nudge ladder and + returns a `z.infer`. +- Compose additional client tools through the helper: + `structuredOutput(schema, { tools: { myTool } })` merges them into one ToolSet. + Don't pass `tools:` to the constructor next to the spread — the later key + silently clobbers the other map. From d7fa382529fa349310ec47ab7311bdee8fb9aa0a Mon Sep 17 00:00:00 2001 From: Thomas Kosiewski Date: Thu, 24 Sep 2026 10:04:27 +0000 Subject: [PATCH 07/10] docs(agent): add durable-workflows and workspaces guides Extract the Workspaces & quota and Durable workflows README sections into standalone guides under packages/agent/docs/, restructured for skimming: tables for options and error types, edge cases and protocol internals moved into
blocks. Former ### headings are kept as ## with identical text so anchors resolve. Signed-off-by: Thomas Kosiewski --- packages/agent/docs/durable-workflows.md | 887 ++++++++++++++++++++ packages/agent/docs/workspaces-and-quota.md | 215 +++++ 2 files changed, 1102 insertions(+) create mode 100644 packages/agent/docs/durable-workflows.md create mode 100644 packages/agent/docs/workspaces-and-quota.md diff --git a/packages/agent/docs/durable-workflows.md b/packages/agent/docs/durable-workflows.md new file mode 100644 index 00000000..e0f45822 --- /dev/null +++ b/packages/agent/docs/durable-workflows.md @@ -0,0 +1,887 @@ +# Durable workflows: persist, resume, recover + +Run **one agent session across process boundaries** — queue jobs, +durable‑workflow steps (Vercel Workflow, step functions, Temporal, …), cron +ticks — and survive the crashes, stream drops, and timeouts in between. Read +this when a chat must outlive the process that started it. Back to the +[package README](../README.md); mechanics are in +[Sessions](../README.md#sessions), +[Timeouts & cancellation](../README.md#timeouts--cancellation), +[Handling errors](../README.md#handling-errors), and +[Observability](../README.md#observability). + +How it works: + +- **All chat state lives on the Coder server** — messages, tool activity, the + run itself. Between steps, carry only **`agent.chatId`** (a string) and, + optionally, **`agent.lastSeenMessageId`** (the resume cursor; saves one + serial round‑trip). +- **Each turn runs inside a durable step.** `CoderAgent` talks REST + + WebSocket through its own client, so it can't ride a `fetch`‑shim durability + layer. + +| Sections | Covers | +| ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------- | +| [Shape each step](#shape-each-step-one-turn-then-checkpoint) → [Archive](#archive-at-the-end--and-only-at-the-end) | The normal path. | +| [Let the SDK absorb drops](#let-the-sdk-absorb-drops--and-handle-what-it-rethrows) | Errors that reach your step. | +| [Recover after a crash](#recover-after-a-crash), [Make client tools crash-safe](#make-client-tools-crash-safe), [Recover the result](#recover-the-result-before-resubmitting) | Hard crashes. The three snippets share variables. | + +## Shape each step: one turn, then checkpoint + +```ts +import { CoderAgent, type CoderTransportEvent } from "@coder/ai-sdk-agent"; + +interface WorkflowCheckpoint { + chatId: string; + /** The resume cursor — pairs with chatId, never persisted without it. */ + lastSeenMessageId?: number; +} + +// Any durable KV your engine gives you: step state, a job row, a DB table. +declare const checkpoints: { + get(workflowId: string): Promise; + set(workflowId: string, checkpoint: WorkflowCheckpoint): Promise; +}; + +export async function runTurn(workflowId: string, prompt: string): Promise { + const checkpoint = await checkpoints.get(workflowId); // undefined on the first step → the turn creates the chat + const agent = new CoderAgent({ + baseUrl: process.env.CODER_URL!, + token: process.env.CODER_SESSION_TOKEN!, // read per step — never checkpoint or log it + organizationId: process.env.CODER_ORG_ID!, + chatId: checkpoint?.chatId, + lastSeenMessageId: checkpoint?.lastSeenMessageId, // optional: skips the pre-prompt history probe + requestTimeoutMs: 300_000, // per segment — see "Bound every step" + onTransportEvent: observe, // see "Watch turn health" + }); + + try { + const { text } = await agent.generate({ + prompt, + abortSignal: AbortSignal.timeout(600_000), // total wall-clock for the step + }); + return text; + } finally { + // Checkpoint even when the turn failed: failed-but-alive chats stay reachable. + if (agent.chatId) { + const next: WorkflowCheckpoint = { chatId: agent.chatId }; + // A turn that failed before streaming may have no cursor. + if (agent.lastSeenMessageId !== undefined) { + next.lastSeenMessageId = agent.lastSeenMessageId; + } + await checkpoints.set(workflowId, next); + } + } +} +``` + +| Rule | Why | +| --------------------------------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | +| **One turn per step; `generate()`, not `stream()`** | The checkpointed unit is a finished result. A mid‑`stream()` failure surfaces on the stream, outside `generate()`'s error contract, which complicates step retries. | +| **Persist the id, not the instance** | Never the token — read it from each step's environment. | +| **Checkpoint in `finally`** | A first step that fails _after_ creating its chat (timeout, stream loss) must still persist the id, or the retry orphans a live chat and its partial effects. Only the [hard‑crash window](#recover-after-a-crash) remains. | +| **No `await using`** | Disposal archives the chat, and [archiving ends resumability](#archive-at-the-end--and-only-at-the-end). | + +
When the finally checkpoint writes nothing + +After an exhausted stream failure on a chat this very call created, the SDK has +already discarded the dead session: `agent.chatId` is `undefined`, so nothing +is written. The thrown `CoderStreamError`'s `chatId` still names the stranded +chat, and `agent.archive()` retires it via `agent.lastKnownChatId`. See +[`CoderStreamError`](#let-the-sdk-absorb-drops--and-handle-what-it-rethrows) +for what retrying means then. + +
+ +## Resume in the next step — any process, any machine + +```ts +// Step 1 — a queue job on machine A: +await runTurn("wf-1042", "Investigate the failing nightly build and propose a fix."); + +// Step 2 — hours later, on machine B: same chat, full server-side history. +await runTurn("wf-1042", "Apply the fix you proposed and summarize what changed."); + +// Final step — cleanup belongs to the workflow, not to each step (see below). +await archiveWorkflowChat("wf-1042"); +``` + +- **Construction does no I/O.** A bad or archived `chatId` fails the _turn_, + not the constructor. Only `lastSeenMessageId` is validated locally: a cursor + that isn't a positive integer, or lacks its `chatId`, throws immediately. +- **A resumed turn streams only its own events** — never earlier turns' + content or usage. Without a checkpointed cursor (first resume, older + checkpoints), it seeds one from the chat's newest message: one extra + single‑message GET, same result. +- **Re‑supply client `tools` in every step that expects tool calls.** They live + in your process. The workspace binding is the opposite: fixed at creation, + unchangeable on resume. +- **Run steps strictly sequentially per chat.** Sessions are single‑flight + system‑wide, not just per process. + +
What happens if two jobs post to the same chat + +The second job queues behind the live run and burns its own `requestTimeoutMs` +waiting. It attributes nothing of the live run's output to itself — its window +opens only when its own queued prompt materializes. If it times out first, it +withdraws its queued prompt rather than leaving it to run unattended. + +The live run's turn is guarded symmetrically. The server starts the queued +prompt's run **without ever emitting a terminal settle for the finished one**, +so the SDK settles the finishing turn the moment the promoted prompt appears on +its stream; nothing of the next turn's output or usage leaks into it. + +That settle reports `finishReason: "stop"` as a documented approximation. The +wire doesn't distinguish a run that completed into a promotion from one +interrupted by `busy_behavior: "interrupt"`, so an interrupted turn also +settles `"stop"` (with whatever partial output it committed) rather than +surfacing an interruption. + +
+ +## Bound every step + +`requestTimeoutMs` arms **one timer per segment**, covering the REST phase +(chat creation, message/tool‑result submission, uploads), the stream, and +**any redial backoff inside the segment**. + +- **Redials never reset the clock.** 12 s spent reconnecting is 12 s less + budget, so a flaky network can't stretch a step past its bound. +- **On expiry** the call rejects with `CoderChatError` (`kind: "timeout"`, + `retryable: true`), and the server run is interrupted best‑effort. +- **Cap total wall‑clock with `abortSignal: AbortSignal.timeout(…)`.** A turn + driving client tools runs several segments + ([Timeouts & cancellation](../README.md#timeouts--cancellation)). +- **A turn that times out (or crashes) before its chat id arrived** leaves + nothing to _resume_ — the checkpoint stays empty and the retried step starts + fresh — but possibly an unacknowledged chat to sweep + ([Recover after a crash](#recover-after-a-crash)). + +## Watch turn health from inside the step + +Emit [transport events](../README.md#observability) as step metrics/traces; +they classify how each segment ended without parsing errors or stream frames: + +```ts +declare const metrics: { + ok(segment: number, status: string, ms: number): void; + fail(segment: number, error: string, detail: string): void; + teardown(segment: number): void; +}; + +function observe(ev: CoderTransportEvent): void { + if (ev.type === "ws:redial") { + // Early warning: the server keeps generating, nothing is lost yet; + // the turn fails if this hits the cap. + console.warn( + `redial ${ev.consecutiveFailures}/${ev.maxConsecutiveFailures}, next in ${ev.backoffMs}ms`, + ); + } else if (ev.type === "segment:settle") { + if (ev.finishReason) metrics.ok(ev.segment, ev.status!, ev.durationMs); + else if (ev.error) metrics.fail(ev.segment, ev.error.name, ev.error.message); + else metrics.teardown(ev.segment); + } +} +``` + +`segment:settle` fires exactly once per `segment:start`, with one of three +endings: + +| Ending | Fields | Means | +| ---------------- | -------------------------------------------------------------------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | +| **Clean settle** | `status` + `finishReason` | `finishReason: "tool-calls"` (status `requires_action`) is a healthy mid‑turn pause for your client tools; terminal statuses end the turn. | +| **Failure** | `error: { name, message }`, plus `status` when the server run still settled terminally | Server‑side turn errors, transport failure after exhausted redials, `requestTimeoutMs` expiry (`error.name: "CoderChatError"`), and caller aborts — named after the abort reason, never teardowns: `"TimeoutError"` for an `AbortSignal.timeout(…)` deadline, `"AbortError"` for an explicit abort. | +| **Teardown** | neither | The stream consumer itself cancelled the turn (`stream()`'s `ReadableStream.cancel()` with no abort of its own) — rare in `generate()`‑shaped steps. | + +Handler exceptions are swallowed and can't disturb the turn; an unset hook +costs nothing. For fleet‑level monitoring _across_ steps, see +[Preventing stuck turns](./workspaces-and-quota.md#preventing-stuck-turns). + +## Archive at the end — and only at the end + +**An archived chat cannot be resumed:** the server rejects new messages on it +(400, `Cannot send messages to an archived chat`). So skip the per‑request +[Cleanup](../README.md#cleanup) habits (`await using`, `archive()` in a +`finally`) in intermediate steps. + +The _workflow_ owns the chat's end of life: archive once, in the final step and +in the failure/compensation handler, so abandoned runs don't accumulate live +chats. + +```ts +export async function archiveWorkflowChat(workflowId: string): Promise { + const checkpoint = await checkpoints.get(workflowId); + if (!checkpoint) return; // the workflow died before its first turn created a chat + const agent = new CoderAgent({ + baseUrl: process.env.CODER_URL!, + token: process.env.CODER_SESSION_TOKEN!, + organizationId: process.env.CODER_ORG_ID!, + chatId: checkpoint.chatId, + }); + // A crashed step may have left a run live — stop it first, bounded. On an + // already-settled chat the interrupt rejects with a 409; ignore it. + await agent.interrupt({ signal: AbortSignal.timeout(15_000) }).catch(() => {}); + await agent.archive(); // retries 409s while the interrupted run winds down (~15s) +} +``` + +## Let the SDK absorb drops — and handle what it re‑throws + +These self‑heal, invisible to the step; the run is **not** interrupted: + +- **Transient stream drops.** The server keeps generating; the SDK redials + with backoff and replays what it missed. Drops cost latency, never content, + and never duplicate content. +- **Drops during a client‑tool pause.** The turn keeps its stream across + `requires_action` pauses; a drop while your tool executes redials in the + background. +- **A lost tool‑call event.** If the pause's tool‑call event doesn't arrive + within ~2 s, the SDK recovers the pending calls from committed history over + REST and the turn continues. + +
What a redial replays + +Committed messages past the turn's cursor plus the in‑progress message's +deltas, deduplicated on receipt. A message that committed _while the +connection was down_ arrives as exactly its missing tail. + +
+ +Once self‑healing is exhausted, the step sees one of three errors: + +```ts +import { CoderApiError, CoderChatError, CoderStreamError } from "@coder/ai-sdk-agent"; + +try { + await runTurn(workflowId, prompt); +} catch (err) { + if (err instanceof CoderChatError && err.retryable) { + // Timeout or transient turn failure; the chat survives. A re-run + // resubmits the prompt as a new user turn (below). + } else if (err instanceof CoderStreamError) { + // Redial budget exhausted (~15s without forward progress). On a resumed + // chat this is never auto-retried — the retry decision is yours (below). + } else if (err instanceof CoderApiError) { + // Branch on err.status: back off and retry 408/425/429/5xx; fail the workflow + // on the rest (expired token, archived/deleted chat, …). + } + throw err; +} +``` + +**Any re‑run resubmits the prompt as a new user turn.** Partial output stays in +history; tool effects that already ran are not undone. `retryable` means the +_failure_ is transient, not that a re‑run is free: retry only steps that +tolerate re‑submission, and reconcile first otherwise. + +| Error | Cause | Before retrying | +| --------------------------------------------- | ------------------------------------------------------------------------------------------------------------------ | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | +| `CoderChatError`, `retryable: true` | `requestTimeoutMs` expired (`kind: "timeout"`) or a transient turn failure (`kind: "stream_closed"`, upstream 5xx) | Chat survives. Run the [settle‑wait](#recover-after-a-crash); pin `cutShort = true` in the [result check](#recover-the-result-before-resubmitting). | +| `CoderStreamError` (an AI SDK `APICallError`) | The stream could not be re‑established | `isRetryable: true` only for a turn that just created its chat with no workspace, no MCP servers, and no fresh inline uploads (the SDK then discards the session). Always `false` on a resumed chat. | +| `CoderApiError` (every non‑2xx) | An HTTP request failed | Back off and retry 408, 425, 429, 5xx. Fail the workflow on 401/403/404 and the archived‑chat 400 — every attempt hits the same wall. | + +
CoderChatError: why wait, and why pin cutShort + +The session is **not** discarded. The retried step reloads the checkpointed +chat id (the `finally` checkpoint makes this hold even when the _first_ step +fails after creating its chat) and resubmits the same prompt as a new user +turn, with the aborted attempt's partial output in history and any tool +effects that ran before the failure not undone. + +Don't resubmit _immediately_: the expiry's server interrupt is asynchronous and +best‑effort, so the timed‑out run may still be committing. Run the same +interrupt‑and‑wait‑for‑terminal‑status recovery as after a crash before the +retry, or its trailing output is absorbed into the retried turn. + +A timed‑out run can also have _finished_ before that interrupt landed. Because +the interrupt was already fired, the result check must treat the attempt as +cut short (pin `cutShort`) rather than trust its own 409. + +
+ +
CoderStreamError: retry semantics and orphaned chats + +When `isRetryable: true`, a retry — `maxRetries` or a re‑run step — starts +clean on a fresh chat. + +**On a resumed chat it is always `isRetryable: false`.** Re‑running the step +resubmits the same prompt as a _new user turn_, with the failed attempt's +partial output still in history (usually fine — the model sees its own aborted +attempt), and any workspace/MCP tool effects that ran before the drop are not +undone. That judgment call belongs to your workflow, which is why it isn't +automatic. + +**The discard happens for _every_ chat the failed turn itself created — +including the effectful, `isRetryable: false` case.** `agent.chatId` is +`undefined` and the `finally` checkpoint writes nothing, but the stranded chat +is not nameless: the error's **`chatId`** field names it, and +`interrupt()`/`archive()` keep targeting it via `agent.lastKnownChatId` +([Cleanup](../README.md#cleanup)). The step's error path can retire the orphan +directly, or checkpoint `err.chatId` for the +[reconciliation sweep](#recover-after-a-crash) when its workspace/MCP effects +need reconciling first. A retried step then starts a fresh chat that has _not_ +seen the orphan's effects: gate that retry on the same idempotency judgment as +any re‑submission. + +
+ +## Recover after a crash + +A hard crash (OOM kill, host loss) runs no `finally` blocks. What to do depends +on where it hit: + +| Crashed… | Left behind server‑side | Do | +| ----------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------- | -------------------------------------------------------------------------------------- | +| **with an empty checkpoint** | Usually nothing — but _up to one unacknowledged chat per attempt may exist_; automatic retries can stack several. | Fresh chat, plus the sweep. | +| **after creation, before the checkpoint** | An orphan: generating until it settles — or, if the crash hit a client tool, paused in `requires_action` forever, holding its workspace. | Fresh chat (no id to interrupt), plus the sweep. | +| **after the checkpoint** | A resumable chat, possibly with a live run or a client‑tool pause. | Settle‑wait (below), then the [result check](#recover-the-result-before-resubmitting). | + +
Why an empty checkpoint is not a clean slate + +A crash or `requestTimeoutMs` expiry while the create request was _in flight_ +can leave a chat the server committed but whose id never arrived — unreachable +even by the SDK's own interrupt. + +
+ +**The reconciliation sweep is not optional in this pattern.** Start from the +sweep in +[Preventing stuck turns](./workspaces-and-quota.md#preventing-stuck-turns), +with two changes: + +- **Widen the filter:** sweep every chat no checkpoint accounts for — + regardless of status — and interrupt/archive it. An orphaned first turn may + settle terminally on its own and then sit live and unarchived forever; the + fleet sweep only catches _non‑terminal_ chats. +- **Retire _every_ matching orphan**, not just the first, including its + workspace/MCP effects if the deployment auto‑attached any. + +
Shrinking the window with an eager checkpoint (and what it costs) + +Checkpoint eagerly from the first `ws:dial` transport event — it carries the +chat id as soon as the chat exists. The checkpoint's lifecycle is then yours +too: + +- The retried step resumes a chat holding a dead half‑turn. +- Whenever a `CoderStreamError` killed a chat this step created — retryable or + not — the SDK discards the session, but only in memory. Clear the eagerly + checkpointed id yourself before any retry, or "starts fresh after a discard" + silently turns into resuming the dead chat. + +
+ +**After the checkpoint, don't just resubmit.** Stop what the crash left behind, +then **wait until the chat actually settles** before deciding: + +- **A client‑tool pause is reconciled, never interrupted away** — if the + tool's effect committed, the pause is its only pending record + ([Make client tools crash-safe](#make-client-tools-crash-safe)). +- **Anything else is interrupted, then polled to a terminal status:** + `interrupt()` resolves on acknowledgment, but the run keeps winding down + (and committing) for a few seconds. +- **A 409 `CoderApiError`** from the interrupt means no live run — the good + case. + +
What goes wrong if you resubmit straight away + +- The crashed attempt's run may still be live. A resubmission queues behind + it: nothing the old run commits is attributed to the resumed turn (its window + opens only at its own queued prompt's materialization), but the step burns + its `requestTimeoutMs` waiting for the old run to finish first — and a + timeout withdraws the queued prompt again. +- A crash inside one of your client tools leaves the chat paused in + `requires_action`, where new messages wait forever. + +
+ +```ts +// Defined in "Make client tools crash-safe": resolves to its `unstarted`, +// true only when it fell back to interrupting. +declare function reconcileClientTools(): Promise; +// A per-ATTEMPT journal: each entry pins the step marker and attempt count. +type JournalEntry = { + // "submitting": written ahead of every send on a checkpointed chat. + // "resumed" / "cut-short": the ledger reconcile's verdict. + state: "submitting" | "resumed" | "cut-short"; + marker: string; // the step-scoped marker ("Recover the result…", below) + attempt: number; // markerAttempts() at write time + at: number; // write time — bounds the in-flight-send wait below +}; +declare const journal: { + get(chatId: string): Promise; + set(chatId: string, entry: JournalEntry): Promise; +}; +// Count of this step's committed attempts (user messages carrying the marker). +declare function markerAttempts(): Promise; + +// Bounded: a stalled deployment must fail the step, not hang it. +const deadline = AbortSignal.timeout(15_000); +// Did an interrupt stop a live run? A 409 means there was nothing to stop. +let cutShort = true; +let { status } = await agent.client.getChat(agent.chatId!, deadline); +// Apply the entry only if it classifies THIS attempt of THIS step. +const entry = await journal.get(agent.chatId!); +const attempts = await markerAttempts(); +const journaled = entry?.marker === marker && entry.attempt === attempts ? entry : undefined; +if (status === "requires_action") { + // A client tool crashed mid-execution: reconcile, never interrupt. + cutShort = await reconcileClientTools(); +} else if (journaled && journaled.state !== "submitting") { + // A previous recovery pass acted on this attempt, then died. + cutShort = journaled.state === "cut-short"; + if (cutShort) { + // The journal records intent, not a landed interrupt: re-drive it. + // A 409 means the chat already settled; the verdict stands. + await agent.interrupt({ signal: deadline }).catch((err) => { + if (!(err instanceof CoderApiError && err.status === 409)) throw err; + }); + } +} else { + await agent.interrupt({ signal: deadline }).catch((err) => { + // A probed "interrupting" means an earlier interrupt already truncated + // the attempt; the 409 just says it landed — keep cutShort = true. + if (err instanceof CoderApiError && err.status === 409) cutShort = status === "interrupting"; + else throw err; + }); +} +// The crashed attempt's own send may still land: sleep out its commit +// window, then recount. +if (journaled?.state === "submitting") { + await new Promise((r) => setTimeout(r, Math.max(0, journaled.at + 300_000 - Date.now()))); + if ((await markerAttempts()) === journaled.attempt && journaled.attempt > 0) cutShort = true; +} +// Turn-scale budget: a reconciled pause revives a full model/tool continuation. +const settle = AbortSignal.timeout(600_000); +do { + ({ status } = await agent.client.getChat(agent.chatId!, settle)); + if (status === "waiting" || status === "completed" || status === "error") break; + // Stable pause: the resumed run called a NEW client tool — reconcile again. + if (status === "requires_action" && !cutShort) break; + await new Promise((r) => setTimeout(r, 1_000)); // acknowledged ≠ settled — wait it out +} while (true); +``` + +Settled is necessary, not sufficient: the crashed attempt may already have +_finished the step's work_. +[Check history before resubmitting](#recover-the-result-before-resubmitting). + +
How the settle‑wait decides, branch by branch + +**Journal scoping.** Entries are per attempt, so the chat id alone cannot scope +them: a verdict from attempt N must not classify attempt N+1 (double crash), +and a later step's recovery on the same chat must not inherit this one's. A +consult applies an entry only while both marker and attempt count still match, +so each resubmission retires every earlier verdict by itself — there is no +clear step to forget. The ledger reconciliation journals each action +write‑ahead, so a recovery pass that dies mid‑recovery is visible — and +classifiable — to the next. + +**`markerAttempts()`** counts user messages in the converted transcript +carrying the marker. Steps run sequentially, so they sit contiguous at the tail +of history: page newest‑first (as the result scan does) and stop at the first +user message without the marker. + +**`requires_action`.** The crash hit a client tool mid‑execution, and this +pause is the only pending record of its committed effects. Reconciliation +submits recorded results (reviving the run — nothing was cut short) or +interrupts itself only when nothing committed. + +**Journaled `"resumed"`.** The previous pass revived the run by submitting +recorded results. Interrupting now would cut down a run doing the step's work, +and the resubmitted prompt would mint _new_ tool‑call ids, sidestepping the +idempotency keys. Let it finish; nothing was cut short. + +**Journaled `"cut-short"`.** The previous pass went on to interrupt (an +unstarted sibling in the batch), so the attempt is truncated and the result +check must resubmit. The journal records write‑ahead _intent_, not a landed +interrupt — the pass may have died in between, leaving the revived run working +— so the interrupt is re‑driven to make the intent true. + +**A 409 on the fresh interrupt.** A 409 proves nothing is stoppable _now_. That +means "never cut short" only if the probe saw no interrupt already in flight. A +probed `"interrupting"` is an earlier interrupt (a timeout's best‑effort one, a +sweeper) already truncating the attempt. + +**A matching `"submitting"` entry** — its attempt count has not grown — means +the process died with `createChatMessage` possibly en route, and the server can +commit it _after_ every read above (the 409 only said nothing was running +then). The wait sleeps out what remains of the entry's commit window: +`requestTimeoutMs` past the write is the conservative horizon, and a restart +has usually consumed it already. A late‑landing turn then starts before the +settle poll, and the next section's negative history read is past the window. + +**An unlanded send over an existing attempt pins `cutShort`.** The step only +ever plans a resubmission over a cut‑short, errored, or unsubmitted attempt, so +the write‑ahead that superseded a `"cut-short"` verdict still carries its +consequence, and the result scan cannot mistake the old attempt's truncated +tail for a finished result. + +**Two budgets.** The 15 s interrupt bound must not cap what follows: a +reconciled pause revives a full model/tool continuation. The settle poll — and +the history reads after it — are budgeted on the turn scale, like the step +itself. + +**`requires_action` inside the poll.** With no accepted interrupt racing to +clear it, `requires_action` is a _stable_ pause: a run resumed by submitted +tool results called a new client tool. Exit and reconcile the ledger again +instead of polling out the deadline. + +
+ +## Make client tools crash-safe + +If the process dies _between_ a non‑idempotent client tool committing its +effect (a payment, a deploy, an email) and the result reaching the server, the +`requires_action` pause is the **only pending record of the execution**. +Interrupt it away, and the resubmitted prompt may call the tool again — +duplicating the effect. + +Fix: a **tool‑invocation ledger** in the same durable store as the `chatId` +checkpoint, keyed by **chat id + `toolCallId`**. The server assigns the call id +and commits it to history, so both sides survive the crash. + +
Why the chat id (and tenant) belong in the key + +Call ids are only unique _within_ a chat. Put the chat id in the ledger key and +in the idempotency key handed to the external system: a store shared across +workflows must never let one chat's `done` entry answer — or deduplicate — +another chat's call. In multi‑tenant stores, fold the tenant in too. + +
+ +```ts +import { tool } from "ai"; +import { z } from "zod"; +import { CoderAgent } from "@coder/ai-sdk-agent"; + +type LedgerEntry = { state: "committing" } | { state: "done"; output: string }; +declare const ledger: { + get(key: string): Promise; + set(key: string, entry: LedgerEntry): Promise; +}; +declare const payments: { + charge(amountCents: number, opts: { idempotencyKey: string }): Promise<{ receiptId: string }>; +}; + +const ledgerKey = (chatId: string, toolCallId: string) => `${chatId}/${toolCallId}`; +const ChargeArgs = z.object({ amountCents: z.number().int() }); + +// Takes a getter: the chat may not exist until the turn creates it, but its +// id is set before any tool runs. +const chargeCard = (chatId: () => string) => + tool({ + description: "Charge the customer's card.", + inputSchema: ChargeArgs, // shared with recovery's re-drive dispatch below + execute: async ({ amountCents }, { toolCallId }) => { + const key = ledgerKey(chatId(), toolCallId); + // Write-ahead, BEFORE the effect: no entry proves the effect never started. + await ledger.set(key, { state: "committing" }); + // The SAME key as the idempotency key: re-driving can never double-charge. + const { receiptId } = await payments.charge(amountCents, { idempotencyKey: key }); + await ledger.set(key, { state: "done", output: `charged: receipt ${receiptId}` }); + return `charged: receipt ${receiptId}`; + }, + }); + +// A direct self-reference in the constructor call is circular for the type +// checker — defer it through a box assigned right after. +let self: { chatId?: string } | undefined; +const agent = new CoderAgent({ + /* …base options as in runTurn… */ + tools: { charge_card: chargeCard(() => self!.chatId!) }, +}); +self = agent; +``` + +On recovery, reconcile a `requires_action` pause _before_ any interrupt. The +pending calls are in committed history; the ledger holds the verdict on each: + +| Ledger entry | Meaning | Action | +| ------------ | ------------------------------------------------------ | -------------------------------------------------------------------------------------------------------------------------------------- | +| `done` | Effect committed, result stranded. | Submit the recorded output; nothing re‑executes. | +| `committing` | Ambiguous: crashed between the write‑ahead and `done`. | Re‑drive with the _same_ idempotency key (a committed effect replays as a no‑op), args from history, validated as the live loop would. | +| none | The tool never ran. | Interrupting loses nothing. | + +This is the derivation the SDK uses to recover a lost `action_required` event: + +```ts +// History does not record which calls were client tools (see [Rehydrating +// chat history]), so restrict to the names this step registers: a server/MCP +// call must never reach the ledger as "unstarted". The result scan's +// final-text cut reuses this set (next section). +const clientTools = new Set(["charge_card"]); + +// Pending = the last assistant message's client (non-provider-executed, +// ToolSet-named) tool calls, minus ids answered by a later message's tool +// result. +const { messages } = await agent.client.getMessages(agent.chatId!, { limit: 200 }, deadline); +const ordered = [...messages].sort((a, b) => a.id - b.id); +const lastAssistant = ordered.findLast((m) => m.role === "assistant"); +const handled = new Set( + ordered + .filter((m) => m.id > (lastAssistant?.id ?? 0)) + .flatMap((m) => m.content ?? []) + .filter((p) => p.type === "tool-result") + .map((p) => p.tool_call_id), +); +const pending = (lastAssistant?.content ?? []).filter( + (p) => + p.type === "tool-call" && + !p.provider_executed && + clientTools.has(p.tool_name!) && + !handled.has(p.tool_call_id), +); + +const results: { tool_call_id: string; output: unknown }[] = []; +let unstarted = false; +for (const call of pending) { + const key = ledgerKey(agent.chatId!, call.tool_call_id!); + const entry = await ledger.get(key); + if (entry?.state === "done") { + results.push({ tool_call_id: call.tool_call_id!, output: entry.output }); + } else if (entry) { + // "committing": re-drive. Every ledgered tool needs its own path. + if (call.tool_name !== "charge_card") + throw new Error(`no re-drive path for tool ${call.tool_name}`); + const { amountCents } = ChargeArgs.parse(call.args); + const { receiptId } = await payments.charge(amountCents, { idempotencyKey: key }); + const output = `charged: receipt ${receiptId}`; + await ledger.set(key, { state: "done", output }); + results.push({ tool_call_id: call.tool_call_id!, output }); + } else { + unstarted = true; // the tool never ran — interrupting loses nothing + } +} +if (results.length > 0 || unstarted) { + // ONE write-ahead with the batch's FINAL verdict, before any network call. + await journal.set(agent.chatId!, { + state: unstarted ? "cut-short" : "resumed", + marker, + attempt: await markerAttempts(), + at: Date.now(), + }); +} +if (results.length > 0) await agent.client.submitToolResults(agent.chatId!, { results }, deadline); +if (unstarted) await agent.client.interruptChat(agent.chatId!, deadline); +// As `reconcileClientTools()` in the settle-wait snippet above: resolve to +// `unstarted` — cutShort is true only when this fell back to interrupting. +``` + +- **Only interrupt when the ledger shows no committed effect.** Submitted + results are committed to history, so even a mixed batch (one call done, a + sibling never started) that ends in an interrupt keeps the effect's recorded + outcome for the resubmitted turn. +- **Submitting answers the pause; it does not revive your tool loop.** The turn + resumes **server‑side** with no process streaming it. Its output lands only + in history — [recover it](#recover-the-result-before-resubmitting) instead + of resubmitting. + +
How reconciliation interacts with the settle‑wait + +- **Why the settle‑wait's reconcile branch never interrupts a pause:** nothing + stranded is left to stop, and an interrupt now would cut down the very run + the submission revived. +- **Why `unstarted` leaves `cutShort = false` after a submission with no + unstarted call:** nothing was cut short, and the next section's `!cutShort` + check must classify the resumed run's finish as a completed attempt, not + resubmit it. +- **The settle poll uses a turn‑scale deadline**, not the 15 s interrupt bound. + It exits on `requires_action` (stable here, with no accepted interrupt + clearing it), which means the resumed run called a _new_ tool — reconcile + again; a call with no ledger entry is safe to interrupt. +- **One batch verdict, journaled first.** A mixed batch journals `"cut-short"` + before even its submission, so no crash point can leave a stale `"resumed"` + masking the pending interrupt. Every crash then either re‑reconciles the + stable pause or re‑drives the journaled intent. The verdict is scoped to this + attempt: a resubmission grows the marker count and retires it on its own. +- **Tool write‑ahead.** A crash between the `committing` and `done` writes + leaves `committing` (ambiguous, but resolvable by re‑driving); no entry at + all proves the effect never started. + +
+ +## Recover the result before resubmitting + +The last crash window: **the server finished the turn, but the process died +before recording the step's result.** A blind resubmit runs the whole turn +again — duplicated output, repeated workspace/MCP/tool effects. + +**First, make attempts matchable.** The server stores the prompt verbatim: +embed a step‑scoped marker, and journal every send before it leaves the +process. + +```ts +const marker = `[${workflowId}/step-2]`; // stable across this step's retries +// Write-ahead, BEFORE the send — first attempt and recovery resubmit alike. +if (agent.chatId) { + await journal.set(agent.chatId, { + state: "submitting", + marker, + attempt: await markerAttempts(), + at: Date.now(), + }); +} +const { text } = await agent.generate({ prompt: `${marker} ${prompt}` }); +``` + +
What the send write‑ahead guarantees + +- A crash before this write proves no send ever started. A crash after it + leaves a dated entry whose commit window the settle‑wait sleeps out. +- On a resubmission this write supersedes the `"cut-short"` verdict it acts on + — safely: an unlanded send over an existing attempt makes the settle‑wait pin + `cutShort` right back. +- A first turn has no chat id to journal under. Its send‑crash window belongs + to the empty‑checkpoint sweep ([Recover after a crash](#recover-after-a-crash)). + +
+ +**Then, after the settle‑wait,** classify the attempt: + +| State | Condition | Action | +| ------------------------ | --------------------------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | +| Finished, recording lost | `submitted && !cutShort`, status `waiting` or `completed` | Return the recovered result. | +| Never finished | `!submitted`, status `"error"`, or `cutShort` | Resubmit. `!submitted` is sound only behind the write‑ahead; `"error"` gets normal retry judgment; after `cutShort` the model continues from its aborted attempt in history. | +| Paused | settled `requires_action` | Back to ledger reconciliation ([previous section](#make-client-tools-crash-safe)) — never recovery or resubmission. | + +```ts +import { chatMessagesToUIMessages } from "@coder/ai-sdk-agent"; + +// A long turn can commit more than a page after the prompt: page newest-first +// (the endpoint's default order) until the turn's user message is on hand. +// "Never submitted" must never be concluded from a truncated read. +const fetched = []; +let page = await agent.client.getMessages(agent.chatId!, { limit: 200 }, settle); +fetched.push(...page.messages); +while (!fetched.some((m) => m.role === "user") && page.has_more) { + const oldestId = Math.min(...fetched.map((m) => m.id)); + page = await agent.client.getMessages(agent.chatId!, { before_id: oldestId, limit: 200 }, settle); + fetched.push(...page.messages); +} +const transcript = chatMessagesToUIMessages(fetched); // chronological +const lastUser = transcript.findLast((m) => m.role === "user"); +const submitted = lastUser?.parts.some((p) => p.type === "text" && p.text.includes(marker)); + +if (submitted && !cutShort && (status === "waiting" || status === "completed")) { + // Recover what `generate().text` would have returned: the FINAL step's + // text, after the last configured client-tool part. + const parts = transcript + .slice(transcript.indexOf(lastUser!) + 1) + .filter((m) => m.role === "assistant") + .flatMap((m) => m.parts); + return parts + .slice(parts.findLastIndex((p) => p.type === "dynamic-tool" && clientTools.has(p.toolName)) + 1) + .filter((p) => p.type === "text") + .map((p) => p.text) + .join(""); +} +// Otherwise resubmit — unless the settle was requires_action (table above). +``` + +- **`cutShort` separates completed from interrupted; history can't** (both + leave the marker and an answer). A 409 only proves nothing is running _now_: + pin `cutShort = true` when recovery follows a known timeout or abort, and let + only the hard‑crash path trust the 409. The settle‑wait already pins it for a + matching journaled `"cut-short"` or an unlanded `"submitting"` over an + existing attempt. +- **`!submitted` races the crashed attempt's own send.** The write‑ahead + narrows that window; it cannot close it. +- **The text cut lands at the last _configured client‑tool_ boundary.** Only a + client tool ends an AI SDK step, so narration before a trailing + _server‑side_ call survives, matching what `generate().text` aggregates. + +
Why a 409 cannot prove the attempt finished after a timeout + +The inference "409 ⇒ the attempt finished" is sound only when recovery's +interrupt is the _first_ one that could have stopped the run: true after a hard +crash, false after a `requestTimeoutMs` expiry. The expiry already fired the +SDK's own best‑effort interrupt, which may have stopped the run before recovery +ever looked. So carry the reason recovery ran. + +
+ +
The remaining duplicate‑send window + +This is the at‑least‑once boundary every durable engine has around external +effects; the chat itself is the record that narrows it to one honest window. + +A crash kills the process, not necessarily its in‑flight `createChatMessage`. +The server can commit that request _after_ an interrupt 409'd and a history +read saw no marker, and a resubmit on that evidence queues the turn twice, +duplicating its workspace/MCP/tool effects. + +The write‑ahead makes the negative read honest: no matching `"submitting"` +entry proves no send ever started, and a matching one held recovery at the +settle‑wait until the entry's commit horizon passed. Only server‑enforced +submission idempotency could close the window, and +`POST /chats/{id}/messages` offers none today. A commit that outlasts even the +horizon still lands under the same marker, so the duplicate is at least +visible — in history, and to the next consult's attempt count. + +
+ +
Where name‑matching the client‑tool cut goes wrong + +Narration lands on both sides of a server‑side call, and history does not mark +which calls were client tools +([Rehydrating chat history](../README.md#rehydrating-chat-history)). So the +cut classifies by name against the reconcile's `clientTools` set. It cannot +see: + +- A call recorded under a name the step no longer configures (a tool renamed + between deploys) reads as server‑side and leaks earlier‑step narration into + the result. +- A server tool sharing a configured name still cuts final‑step narration + away. + +
+ +
Structured‑output steps: recover the call's input, not the text + +A [structured output](../README.md#structured-output) step's answer is the +`structured_output` call's typed input, which rehydrates as a `dynamic-tool` +part on the recovered turn +([Rehydrating chat history](../README.md#rehydrating-chat-history)). +`structured_output` is one of the step's configured client tools, so the text +join above cuts at it and returns only the ack prose that follows. Recover the +filed call instead: this scan replaces the text join _inside_ the recovery +branch above, reusing its `parts`, and validates client‑side exactly like the +live path (rule 2 there — the schema is the real gate): + +```ts +// Scan backward to the last call that VALIDATES, as in the live path — a +// schema-invalid re-file must not shadow a valid answer. +const filed = parts + .filter((p) => p.type === "dynamic-tool") + .filter((p) => p.toolName === "structured_output"); +let answer: z.infer | undefined; +for (const call of filed.reverse()) { + const parsed = Answer.safeParse(call.input); + if (parsed.success) { + answer = parsed.data; + break; + } +} +// recovered ⇔ answer !== undefined +``` + +
+ +## Checklist + +- One turn per durable step; `generate()`, not `stream()`. +- Persist `agent.chatId` (a string) — never the instance, never the token. +- Bound every step: `requestTimeoutMs` per segment, an abort deadline for total + wall‑clock. +- Let redials self‑heal; own every retry decision — a re‑run step resubmits its + prompt as a new user turn. +- Before resubmitting after a crash: reconcile the tool ledger before any + interrupt, recover a finished attempt's result from history, and trust "never + submitted" only past the journaled send's commit window. +- Keep concurrent steps' fan‑out within workspace quota — + [Workspaces & quota](./workspaces-and-quota.md). +- Steps that don't need server‑side tools (plan / extract / synthesize) are + cheaper and natively structured through + [`@coder/ai-sdk-provider`](../../provider) + `generateObject` — no chat, no + workspace, no cleanup. +- Archive in the final step / failure handler — never per step. diff --git a/packages/agent/docs/workspaces-and-quota.md b/packages/agent/docs/workspaces-and-quota.md new file mode 100644 index 00000000..ed1086c3 --- /dev/null +++ b/packages/agent/docs/workspaces-and-quota.md @@ -0,0 +1,215 @@ +# Workspaces & quota + +How to run fleets of agents: how a chat binds to a Coder workspace, how to size +concurrency, what to clean up, and how to diagnose a stuck chat. Read it before +running many `CoderAgent`s at once. Back to the [package README](../README.md). + +A chat may be backed by a **Coder workspace** that runs its tools, depending on +its configuration and the deployment. Workspaces are the scarce resource: a +deployment budgets how many an account may run at once, so **N agents running concurrently can need N schedulable workspaces.** Past that +bound, a turn can sit unscheduled and never settle. + +## How a chat binds to a workspace + +- **One chat, at most one workspace, fixed at creation.** `workspaceId` is sent + as `workspace_id` on chat creation; message and update requests carry no + workspace field. To move work to another workspace, start a new agent/chat. +- **This SDK never provisions workspaces.** Pass an existing one. Provision it + with [`@coder/ai-sdk-sandbox`](../../sandbox)'s `ensureCoderWorkspace`, the + CLI, or the v2 API. +- **A chat created without `workspaceId` can still be workspace‑backed.** + Deployments may assign one server‑side. The SDK reads the created chat's + `workspace_id` and treats the chat as workspace‑backed from then on (this + affects [retry ownership](#preventing-stuck-turns)). +- **Chat cleanup does not release the workspace.** `archive()` soft‑hides the + chat only. The workspace keeps running until template autostop or an + explicit stop. Stopping releases only stop‑scoped quota; persistent resources + (disks, volumes) keep consuming their cost until the workspace is deleted. + +## Sizing a fleet + +Rule: **workspaces that must be running concurrently ≤ schedulable +workspaces.** + +- **One workspace per chat** (the common shape, and what auto‑assigning + deployments produce): concurrent chats ≤ schedulable workspaces. +- **Chats bound to a shared `workspaceId`** count it once, so their concurrency + is not quota‑bound. They share one filesystem and tool environment — only + acceptable within a single tenant / trust boundary. + +"Schedulable" is a deployment property, not an SDK knob. Whichever binds first: + +| Limit | How it works | When exceeded | +| ----------------------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ---------------------------------------------------------------------------------------------------- | +| **Workspace quota** (premium deployments) | Templates declare per‑resource costs. A user's budget is the sum of their groups' quota allowances, enforced when a build starts or stops ([resource quotas](https://coder.com/docs/admin/users/quotas)). | The build **fails** (`INSUFFICIENT_QUOTA`, "insufficient quota"); the turn never gets its workspace. | +| **Infrastructure** | No per‑user workspace limit by default without quotas ([workspace lifecycle](https://coder.com/docs/user-guides/workspace-lifecycle)). The bound is provisioner throughput and cluster capacity. | Slow or failing builds, not a crisp quota error. | + +Practical sizing: + +1. **Read headroom before fanning out.** + `GET /api/v2/organizations/{org}/members/{user}/workspace-quota` returns + `{ "credits_consumed": …, "budget": … }`. Admit another workspace only while + `budget − credits_consumed` covers _that workspace's_ cost (the sum of its + template's `daily_cost` declarations). +
Credits are not slots + + "Free slots = headroom ÷ cost" only holds for a homogeneous fleet on one + template. With mixed templates, size against each planned workspace's own + cost. + +
+ +2. **Queue the rest client‑side.** An unschedulable turn does not queue + usefully on the server ([Preventing stuck turns](#preventing-stuck-turns)). +3. **Reuse one bound workspace** across sequential turns and sessions. The + workspace is the expensive part; the chat is cheap. Reuse only **within one + tenant / trust boundary**: agents have file and shell tools, so a reused + filesystem carries one session's artifacts (and secrets) into the next. + Provision per tenant, or securely reset a workspace before reassigning it. +4. **Send steps that don't need server‑side tools to the + [provider](../../provider)** — it never touches a workspace. + +## Autostop & cleanup + +Manage two lifetimes separately: + +- **Chats:** `archive()` / `await using` every agent + ([Cleanup](../README.md#cleanup)), or finished chats keep holding server + resources. +- **Workspaces:** rely on template‑level scheduling, not manual hygiene. + +| Template setting | What to do | +| ------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | +| **Autostop TTL** | Long enough to survive a normal session (including idle gaps between turns), short enough that a leaked workspace stops burning running‑cost within hours. Without autostop, a leaked workspace pins its full quota until someone notices. [Sandbox](../../sandbox): `stopAfter: "8h"` sets `ttl_ms` at creation. | +| **Activity bump** (default 1 h) | Extends a running workspace's deadline when Coder detects sessions. Check [what counts as activity](https://coder.com/docs/user-guides/workspace-scheduling) before assuming server‑side tool use keeps a workspace alive. | +| **Dormancy / failure cleanup** | Reaps abandoned and repeatedly‑failing workspaces automatically ([template scheduling](https://coder.com/docs/admin/templates/managing-templates/schedule)). | + +**Stopping is not enough.** A _stopped_ workspace typically still consumes its +persistent resources' `daily_cost` (disks, volumes), so a scratch fleet that +only ever stops converges on a full budget. Pair the TTL with dormancy +auto‑deletion or explicit deletion. + +## Preventing stuck turns + +The signature failure of an over‑committed fleet: the chat is created, the +stream opens, then nothing. There is **no distinct "quota exceeded" error kind +on the chat stream.** An unschedulable chat surfaces a generic turn error or +sits in a non‑terminal status indefinitely. + +Defend in this order: + +1. **Set `requestTimeoutMs` — always, in fleets.** It is unbounded by default. + On expiry the call rejects with a `CoderChatError` (`kind: "timeout"`, + `retryable: true`), so your dispatcher gets its slot back. Pair it with a + reconciliation sweep: periodically interrupt/archive non‑terminal chats + older than your budget. +
Why the run is not guaranteed released on timeout + + The timeout fires a server‑side interrupt **best‑effort**: fire‑and‑forget, + and unreachable when the timeout lands before chat creation returned an id. + The run usually stops, but is not guaranteed released + ([Timeouts](../README.md#timeouts--cancellation)). + +
+ +2. **Cap total wall‑clock** with `abortSignal: AbortSignal.timeout(…)`. + `requestTimeoutMs` bounds each segment, not a whole multi‑tool call. +3. **Own the retries.** Workspace‑backed turns are **never auto‑retried**: a + stream‑loss error is `isRetryable` only when the failed turn created its + chat _and_ had no workspace, no MCP servers, and no fresh uploads + ([Timeouts](../README.md#timeouts--cancellation)). Re‑check quota headroom + before retrying, or you re‑queue into the same wall. +4. **Watch the fleet.** [`watchChats`](../README.md#watching-chats) yields + status changes for every chat **visible to the authenticated user** — run + one watcher per identity for per‑tenant credentials, or use a credential + that sees them all. Alert on chats in a non‑terminal status (e.g. `pending`) + longer than your `requestTimeoutMs`. Seed and periodically reconcile against + `GET /api/v2/chats`; don't trust the event stream alone. +
Why events alone miss stuck chats + + Reconnects resubscribe fresh (no cursor or replay). A chat already stuck + before the watcher started — or one that transitioned during a gap — never + emits an event to start your timer from. + +
+ +## Troubleshooting: unschedulable & stuck chats + +Wire [`onTransportEvent`](../README.md#observability) into fleet telemetry; the +event sequence pinpoints _where_ a turn died. Expand the symptom you see. + +
segment:start and ws:open fired, initial status events, then silence + +No `message_part`, no `segment:settle`. + +- **Likely cause:** if the chat's status is `pending` (or its workspace build + is `pending`/`failed`), the workspace can't be scheduled — quota exhausted, + no free provisioner, failing template. If the status is `running` with a + healthy build, it may just be a slow model/tool step; silence alone doesn't + prove scheduling. +- **Fix:** check quota headroom (`workspace-quota` endpoint) and the + workspace's build — a quota failure logs `INSUFFICIENT_QUOTA`. Reclaim enough + _credits_ for the planned build: stopping helps only for stop‑scoped costs; + delete workspaces to reclaim persistent costs, or raise the group allowance. + Set `requestTimeoutMs` so this fails loudly next time. + +
+ +
segment:settle carries a CoderChatError mentioning the requestTimeoutMs budget + +The event looks like +`error: { name: "CoderChatError", message: "…requestTimeoutMs budget…" }`. + +- **Likely cause:** the per‑segment bound expired — wedged server, slow model, + or an unschedulable workspace. +- **Fix:** inspect the workspace via the v2 API/UI. A `pending`/`failed` build + means the previous symptom; `running` means the turn was genuinely slow — + raise `requestTimeoutMs` for long tool work. + +
+ +
Repeated ws:redial, then a CoderStreamError + +`consecutiveFailures` climbs toward `maxConsecutiveFailures` (5; backoff +1 s → 2 s → 4 s → 8 s, ≈15 s of redialing without forward progress). + +- **Likely cause:** the network path to the deployment is failing — not + workspace scheduling (the server keeps generating through short gaps). +- **Fix:** fix connectivity. Mind [retry ownership](#preventing-stuck-turns): + on workspace‑backed chats the error is `isRetryable: false`, so the replay + decision is yours. + +
+ +
segment:settle with status: "error" and an error payload + +- **Likely cause:** the turn failed server‑side — a provider/model error, a + tool failure, or a scheduling/build failure that terminated the turn instead + of leaving it pending. +- **Fix:** the settle event's `error` carries only `{ name, message }`. To + branch on `kind` / `retryable` / `statusCode`, catch the thrown + `CoderChatError`: at the `generate()` call site, or — for `stream()` — around + stream consumption, since mid‑stream failures surface on the stream, not from + `await agent.stream()` ([Handling errors](../README.md#handling-errors)). + Check the workspace build state to rule scheduling in or out. + +
+ +
Turn settled status: "requires_action"; follow‑up messages queue forever + +- **Likely cause:** the loop ended on an unanswered client tool call. +- **Fix:** submit the stranded results or interrupt — see rule 4 under + [Structured output](../README.md#structured-output). If a crash left the + pause behind, reconcile effects first + ([Make client tools crash-safe](./durable-workflows.md#make-client-tools-crash-safe)). + +
+ +
archive() keeps returning 409 and rethrows after ~15 s + +- **Likely cause:** the chat never settled server‑side — usually a stuck run + still holding its workspace. +- **Fix:** `interrupt()` with a bounded signal, then re‑archive. If the run + stays wedged, stop the workspace itself. + +
From 163fe06a8282d77fae837807642bf549644a405c Mon Sep 17 00:00:00 2001 From: Thomas Kosiewski Date: Thu, 24 Sep 2026 10:08:48 +0000 Subject: [PATCH 08/10] docs(agent): group README sections and add contents table Change-Id: I33368a8f75e4a0f5da1a315a00418d195d9e26c2 Signed-off-by: Thomas Kosiewski --- packages/agent/README.md | 667 ++++++++++++++++++++------------------- 1 file changed, 340 insertions(+), 327 deletions(-) diff --git a/packages/agent/README.md b/packages/agent/README.md index 357f775d..0e9c026c 100644 --- a/packages/agent/README.md +++ b/packages/agent/README.md @@ -19,12 +19,25 @@ client‑side loop without re‑implementing the loop. > Coder 2.37.0 (September 1, 2026). This package remains pre‑1.0. > See [API compatibility](#api-compatibility) for older deployments. +**Contents** + +| Get started | Build | Reference | Run in production | +| ------------------------------------------------------- | ----------------------------------------------------- | --------------------------------------- | -------------------------------------------------------------- | +| [Install](#install) | [Custom tools](#custom-tools) | [Auth](#auth) | [Timeouts & cancellation](#timeouts--cancellation) | +| [Quick start](#quick-start) | [Files](#files) | [Configuration](#configuration) | [Handling errors](#handling-errors) | +| [Examples](#examples) | [Structured output](#structured-output) | [API compatibility](#api-compatibility) | [Cleanup](#cleanup) | +| [Agent vs. provider](#agent-vs-provider--which-package) | [Sessions](#sessions) | [How it works](#how-it-works) | [Usage & cost](#usage--cost) | +| | [Rehydrating chat history](#rehydrating-chat-history) | [Testing](#testing) | [Observability](#observability) | +| | [Watching chats](#watching-chats) | [Limitations](#limitations) | [Workspaces & quota](#workspaces--quota) | +| | [Workspace previews](#workspace-previews) | | [Durable workflows](#durable-workflows-persist-resume-recover) | +| | [Sources](#sources) | | | + ## Agent vs. provider — which package? | | `@coder/ai-sdk-agent` (this package) | [`@coder/ai-sdk-provider`](../provider) | | ----------------- | --------------------------------------------------------------------------------------------------------------------------------------------------- | ---------------------------------------------------------------------------------------------------------------------- | | What runs | Coder's server‑side agent: tool loop, built‑in tools, MCP servers, workspace‑scoped file/shell tools, sub‑agents, and compaction, on the deployment | Plain model calls through Coder's AI Gateway. A normal AI SDK provider: `generateText`, `streamText`, `generateObject` | -| Server state | Each `CoderAgent` is one server chat ("session") and may provision a workspace | No chat, no workspace; natively cancelable | +| Server state | Each `CoderAgent` is one server chat ("session"), bound to at most one workspace | No chat, no workspace; natively cancelable | | Structured output | Through a tool call ([Structured output](#structured-output)) | `generateObject`, schema‑constrained | | Reach for it when | You need **server‑side tools, MCP, or a workspace** | You just need **a model** (plan / extract / summarize / classify) | @@ -195,61 +208,23 @@ const { path } = await agent.uploadToWorkspace({ // Then ask the agent to `unzip assets.zip` — uploadToWorkspace writes bytes as-is; it does not unpack. ``` -## Auth - -Pass a Coder **API token** or **session token** as `token`. It is sent as the -`Coder-Session-Token` header (REST) and authenticates the streaming WebSocket. -Create one with `coder tokens create`, or reuse your CLI session. - -`baseUrl`/`token` default from the `CODER_URL` and `CODER_SESSION_TOKEN` -environment variables, the same convention as `@coder/ai-sdk-sandbox`'s -transports. Explicit settings win over the environment: - -```ts -const agent = new CoderAgent({ organizationId }); // uses CODER_URL + CODER_SESSION_TOKEN -``` - -You can also pass a pre‑built client: - -```ts -import { CoderAgent, CoderChatClient } from "@coder/ai-sdk-agent"; -const client = new CoderChatClient({ baseUrl, token }); -const agent = new CoderAgent({ client, organizationId }); -``` - -## API compatibility - -Coder 2.37.0 promoted chat routes to `/api/v2/chats` -([server routes](https://github.com/coder/coder/pull/28496), -[Go SDK](https://github.com/coder/coder/pull/28497)). Coder 2.36.4 has no stable -chat mount. During the compatibility window (CODAGT-921) the promoted routes also -remain under `/api/experimental/chats`; legacy removal is tracked in CODAGT-922. - -Each `CoderChatClient` selects its chat prefix from HTTP responses, not version -strings: v2 first, then the experimental prefix on a 404. The first non‑404 HTTP -response locks the prefix for the client's lifetime. +## Structured output -
-Prefix selection rules +Coder Agents has no server‑side `response_format`, so `CoderAgent` cannot +constrain what the model **says** to a JSON schema. A `responseFormat` / +`experimental_output` request emits a warning and is best‑effort at most. -- Until a prefix is selected, an actual REST request (including JSON requests - and `Blob` uploads) tries v2 first and retries once under the experimental - prefix only on 404. -- The first non‑404 HTTP response (including an error) locks that prefix for the - client's lifetime. Concurrent unresolved calls may probe independently; the - first conclusive response wins. -- Two 404s preserve the original v2 error without locking. Non‑HTTP failures do - not lock either. -- A locked prefix is never renegotiated, even on 404. -- Before opening a per‑chat `/stream` or global `/watch` WebSocket, an - unresolved client preflights with `GET /api/v2/chats`, using the same - selection rules and cancellation signal. -- `ReadableStream` uploads also preflight, so their consumed bodies are never - replayed. -- Legacy model‑config lookup remains independent of chat‑prefix selection (see - [Configuration](#configuration)). +- **Pure text‑in / JSON‑out, no server‑side tools** → use + [`@coder/ai-sdk-provider`](../provider) with `generateObject` / + `Output.object` (schema‑constrained; requires AI Gateway on the deployment). +- **The answer must come out of an agent run** (server‑side tools, MCP, a + workspace) → have the model submit its answer by _calling a tool_ whose + `inputSchema` is your Zod schema. The answer arrives as the tool call's typed + `input`. -
+The tool pattern needs four rules to stay robust (validate client‑side, never +stop on the call, and more). Guide: [docs/structured-output.md](./docs/structured-output.md). +Copyable helper: [`examples/06-structured-output.ts`](./examples/06-structured-output.ts). ## Sessions @@ -344,131 +319,275 @@ prefix selection.
-## Observability +## Workspace previews -Pass `onTransportEvent` to receive typed events for HTTP exchanges, the per‑chat -stream's WebSocket lifecycle, and turn‑segment boundaries. You get timing and -tracing without wrapping `fetch`/`webSocketFactory` or re‑parsing stream frames. +When the agent is bound to a workspace (the `workspaceId` setting), resolve — and +share — the browser URL where a workspace port is served, e.g. the dev server the +agent just started: -Handler exceptions are swallowed, there is zero overhead without a handler, and -events carry no headers or tokens. Example, event reference, and semantics: -[docs/observability.md](./docs/observability.md). +```ts +const { url } = await agent.getPreview({ port: 3000 }); +// → https://3000--main--dev--alice.apps.example.com (private to the workspace owner) -## Timeouts & cancellation +const shared = await agent.sharePreview({ port: 3000, shareLevel: "authenticated" }); +// shared.url is now reachable by any logged-in user; shared.shareLevel is the level in effect +``` -**Cancel a turn** by passing an `abortSignal` to `generate()`/`stream()`. -Aborting **interrupts the server‑side run**, not just the local socket, so the -chat stops generating and releases its resources instead of running on, -orphaned. Tearing down a `stream()` early (cancelling the stream) interrupts the -run too. +| Method | What it does | +| ------------------------------------------------------ | ------------------------------------------------------------------------------------------------------------------------ | +| `getPreview({ port, agentName?, protocol?, signal? })` | Composes the subdomain URL. The URL honors the port's current share level: private to the workspace owner unless shared. | +| `sharePreview({ port, shareLevel?, … })` | Additionally upserts the port's share level (re‑invoking updates it in place) and returns the level in effect. | -**Bound each segment** with `requestTimeoutMs`. If a segment runs longer (e.g. -the server is wedged, or a workspace can't be scheduled), the run is interrupted -and the call rejects with a retryable `CoderChatError` (`kind: "timeout"`) -instead of hanging: +- Ports below 1000 are rejected up front. Coder subdomain URLs only encode 4–5 + digit ports, so `80--agent--…` would be parsed as an app named "80" and never + resolve; serve the preview on a higher port. +- `agentName` is optional when the workspace has exactly one agent. With + several, the error lists the candidates. +- `protocol: "https"` means the app speaks TLS _inside the workspace_. It adds + the `s` label suffix (`3000s--…`) and does not affect the browser scheme. +- `shareLevel` values: -```ts -const agent = new CoderAgent({ /* … */ requestTimeoutMs: 120_000 }); -``` + | `shareLevel` | Reachable by | + | --------------------------- | ---------------------------------------------------------------------- | + | `"authenticated"` (default) | Any logged‑in user | + | `"organization"` | Members of the workspace's organization; requires a newer Coder server | + | `"public"` | Anyone — no auth at all; mind what the port serves | -A segment is one model round‑trip, until it settles or pauses for a client tool. -A multi‑step `generate()` that drives client tools runs several segments, so -`requestTimeoutMs` bounds each one, not the whole call. + Reverting to owner‑only means deleting the share; `"owner"` is not accepted on + upsert. -**Cap total wall‑clock** of a multi‑step call with a deadline signal instead: +**Credentials.** The preview helpers call non‑chat endpoints, so they need +`baseUrl` + `token`. Pass them alongside `client` if you construct one yourself +(or let them default from `CODER_URL`/`CODER_SESSION_TOKEN`). -```ts -await agent.generate({ prompt: "…", abortSignal: AbortSignal.timeout(120_000) }); -``` +
+Compatibility and failure cases (wildcard access URL, Coder < v2.9) -**Stream drops heal themselves.** If the event stream drops mid‑turn, the agent -redials it automatically with exponential backoff, replays the turn's events from -its starting cursor, and deduplicates them on receipt. The server keeps -generating during the gap, so a transient drop costs nothing and the run is -**not** interrupted. +Both helpers are built on the stable v2 workspace APIs (workspace lookup + the +wildcard apps host; `sharePreview` adds a port‑share upsert), so they work +against old Coder servers — no experimental endpoints. -Only when the stream cannot be re‑established (several consecutive failed -attempts, ~15s) is the server run interrupted and the call rejected with a -`CoderStreamError`, an AI SDK `APICallError`: +They fail clearly instead of returning broken URLs: -| `isRetryable` | When | Then | -| ------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | --------------------------------------------------------------------------------------------------------------------------- | -| `true` | The failed turn had just created its chat **and** had no external effects a replay would repeat: no `workspaceId`, no `mcpServerIds`, and no freshly uploaded inline attachments (pre‑uploaded `fileId` references are fine) | The dead session is discarded. `generate()` calls with `maxRetries` set retry the whole turn on a fresh chat automatically. | -| `false` | Otherwise | Retrying is the caller's deliberate decision. | +- A deployment without a wildcard access URL (`--wildcard-access-url`) yields an + explanatory error. +- A server that predates port sharing (< Coder v2.9) yields a 404 + `CoderApiError` saying so. -
-Why the other cases aren't retryable +
-On a chat with prior state (resumed sessions, later turns, tool‑result -segments), a re‑invocation would resubmit the same prompt as a new user turn. -Workspace/MCP tools may already have executed side effects, and inline -attachments would upload again. +## Sources -
+Model configs with web search enabled emit `source` parts. They flow through to +`result.sources` and, in UI message streams, to `source-url` parts. Pass +`sendSources: true` to `toUIMessageStream`; the AI SDK omits them by default. +Earlier releases dropped them. -- For `stream()`, a mid‑stream failure surfaces on the stream itself, outside - the SDK's retry wrapper. Handle it in your consumption loop. -- A non‑transient 4xx upgrade rejection (bad/expired token, deleted chat) fails - fast with a `CoderApiError` instead of retrying. 408/425/429 consume the redial - budget like any other transient failure. +## Auth -## Cleanup +Pass a Coder **API token** or **session token** as `token`. It is sent as the +`Coder-Session-Token` header (REST) and authenticates the streaming WebSocket. +Create one with `coder tokens create`, or reuse your CLI session. -The agent is an **async disposable**, so cleanup can ride scope exit instead of a -`finally` you have to remember: +`baseUrl`/`token` default from the `CODER_URL` and `CODER_SESSION_TOKEN` +environment variables, the same convention as `@coder/ai-sdk-sandbox`'s +transports. Explicit settings win over the environment: ```ts -await using agent = new CoderAgent({/* … */}); -const { text } = await agent.generate({ prompt: "…" }); -// agent.interrupt() + agent.archive() run automatically when the scope exits. +const agent = new CoderAgent({ organizationId }); // uses CODER_URL + CODER_SESSION_TOKEN ``` -- Disposal interrupts any in‑flight run, then archives. -- Disposal is **bounded and never throws** (~15s overall, best‑effort). Its - errors are swallowed so they can't mask the scope's own error. Call - `archive()` directly when you need guaranteed cleanup. -- In a request handler that returns before a fire‑and‑forget `archive()` - settles, the archive can be abandoned. `await using` (or an awaited - `archive()` in `finally`) avoids accumulating live chats. +You can also pass a pre‑built client: -**What `archive()` does:** +```ts +import { CoderAgent, CoderChatClient } from "@coder/ai-sdk-agent"; +const client = new CoderChatClient({ baseUrl, token }); +const agent = new CoderAgent({ client, organizationId }); +``` -- It soft‑hides the chat: the chat stays in listings as `archived: true`. There - is no hard delete yet. -- A freshly interrupted chat keeps winding down server‑side for a few seconds, - and archiving 409s meanwhile. `archive()` retries those 409s (~1s apart, up to - ~15s overall; tune with `settleDeadlineMs` / `settleRetryDelayMs`) and - rethrows the last one if the chat never settles. -- Any other failure, including your own abort, rethrows immediately. +## Configuration -**Return values.** Both methods report what they acted on instead of silently -no‑oping: +`CoderAgentSettings`: -| Method | Acted on a chat | No chat exists at all | -| ------------- | ----------------------------------------------------------------------------------------------- | ------------------------ | -| `archive()` | `{ archived: true, chatId, archivedChatIds }` (each archived id is cleared as a cleanup target) | `{ archived: false }` | -| `interrupt()` | `{ interrupted: true, chatId }` | `{ interrupted: false }` | +| field | description | +| --------------------------------- | ------------------------------------------------------------------------------------------------------------------ | +| `client` \| (`baseUrl` + `token`) | connection (one or the other; `baseUrl`/`token` default from `CODER_URL`/`CODER_SESSION_TOKEN`) | +| `organizationId` | org UUID that owns the chat (required) | +| `model` | model hint: UUID, `provider:model`, model id, or display‑name substring | +| `reasoningEffort` | reasoning effort for chat creation and every user-message submission (see below) | +| `instructions` | system prompt | +| `tools` | AI SDK `ToolSet` (client‑executed) | +| `workspaceId` | bind the chat to a Coder workspace (enables workspace‑scoped tools) | +| `workspaceFiles` | adapter enabling `uploadToWorkspace()` (write files to the workspace FS) | +| `mcpServerIds` | server‑side MCP servers to enable | +| `planMode` | enable plan mode (`"plan"`) | +| `stopWhen` | AI SDK stop condition(s); default `stepCountIs(64)` | +| `maxRetries` | default `0` — SDK retries can duplicate server‑side turns; override with care | +| `requestTimeoutMs` | per‑turn time budget (ms); interrupts the run and rejects (`kind: "timeout"`) instead of hanging | +| `onTransportEvent` | observability hook for typed transport events (see [Observability](#observability)) | +| `settleDeadlineMs` | overall deadline for bounded cleanup (`archive()` 409 retries, disposal); default 15 000 | +| `settleRetryDelayMs` | pause between `archive()` retries while the chat settles; default 1000 | +| `chatId` | resume an existing chat | +| `lastSeenMessageId` | resume cursor for `chatId` (persist `agent.lastSeenMessageId`) — skips the resumed turn's pre‑prompt history probe | -
-Which chat archive() and interrupt() target after a session is dropped +**Model hint resolution.** The `model` hint resolves against the organization's +model configs, first match wins: -- They target the session's chat. After the session was dropped — - `resetSession()`, or the automatic discard after a fresh‑chat stream failure - (see [Handling errors](#handling-errors)) — they target the **last‑known chat - id** (`agent.lastKnownChatId`), so a stranded chat is still cleaned up instead - of leaking. -- Generation never uses the last‑known id: a turn after a drop creates a fresh - chat as always. -- Every chat stranded by an _automatic_ discard is also recorded on a ledger, - `agent.strandedChatIds` (oldest first). With `maxRetries`, several failed - attempts can strand one chat each while only the final attempt's error - surfaces. One `archive()` retires them all, oldest first, alongside its - primary target. -- Deliberate abandonment is different: `resetSession()` does **not** add to the - ledger (you may want that chat kept). After a manual reset, the old chat is - targetable only until a new chat supersedes `lastKnownChatId`. +1. a config UUID, used as‑is +2. an exact `provider:model` match +3. an exact model id +4. a display‑name substring (case‑insensitive) +5. a model‑id substring -
+An unresolvable hint falls back to the server's default model instead of +failing. Use `agent.listModels()` to see what's available. + +
+Where model configs come from, and older deployments + +- Configs come from + `GET /api/v2/organizations/{organizationId}/chats/models`, with the provider + type joined from the response's provider descriptors. +- On older deployments where the organization‑scoped route does not exist yet + (404), resolution falls back once to the legacy deployment‑wide + `/api/experimental/chats/model-configs` listing, independently of chat‑prefix + selection. +- Partial payloads from older/newer servers are tolerated: entries match on the + fields they carry. + +
+ +### Reasoning effort + +Set `reasoningEffort` on `CoderAgentSettings` (or `CoderLanguageModelConfig` +when using the model directly): + +```ts +import { CoderAgent } from "@coder/ai-sdk-agent"; + +const agent = new CoderAgent({ + organizationId: "your-org-uuid", + reasoningEffort: "low", +}); +await agent.generate({ prompt: "Reply with exactly: pong" }); +const chat = await agent.client.getChat(agent.chatId!); +console.log(chat.last_reasoning_effort); // string, null, or absent on the wire +await agent.archive(); +``` + +- **Values.** The global scale is `none`, `minimal`, `low`, `medium`, `high`, + `xhigh`, `max`. The exported `ReasoningEffort` type also accepts other strings + for forward compatibility. +- **Per‑model options.** Model configs returned by `agent.listModels()` / + `client.listModelConfigs(organizationId)` expose the optional + `reasoning_efforts` list of selectable values for each model. +- **Independent of the `model` hint.** +- **No per‑call override.** Configure it on the agent/model instance. + +
+When reasoning_effort is sent, and older servers + +- It is sent as `reasoning_effort` on chat creation and every + user‑message submission, including submissions to resumed or busy chats that + queue the message. When unset, the request key is omitted entirely. + Tool‑result submissions and WebSocket reconnects do not post user messages and + do not send this field. +- Older Coder servers that do not recognize `reasoning_effort` ignore the + unknown JSON field, so setting it is safe but has no effect there. + +
+ +## API compatibility + +Coder 2.37.0 promoted chat routes to `/api/v2/chats` +([server routes](https://github.com/coder/coder/pull/28496), +[Go SDK](https://github.com/coder/coder/pull/28497)). Coder 2.36.4 has no stable +chat mount. During the compatibility window (CODAGT-921) the promoted routes also +remain under `/api/experimental/chats`; legacy removal is tracked in CODAGT-922. + +Each `CoderChatClient` selects its chat prefix from HTTP responses, not version +strings: v2 first, then the experimental prefix on a 404. The first non‑404 HTTP +response locks the prefix for the client's lifetime. + +
+Prefix selection rules + +- Until a prefix is selected, an actual REST request (including JSON requests + and `Blob` uploads) tries v2 first and retries once under the experimental + prefix only on 404. +- The first non‑404 HTTP response (including an error) locks that prefix for the + client's lifetime. Concurrent unresolved calls may probe independently; the + first conclusive response wins. +- Two 404s preserve the original v2 error without locking. Non‑HTTP failures do + not lock either. +- A locked prefix is never renegotiated, even on 404. +- Before opening a per‑chat `/stream` or global `/watch` WebSocket, an + unresolved client preflights with `GET /api/v2/chats`, using the same + selection rules and cancellation signal. +- `ReadableStream` uploads also preflight, so their consumed bodies are never + replayed. +- Legacy model‑config lookup remains independent of chat‑prefix selection (see + [Configuration](#configuration)). + +
+ +## Timeouts & cancellation + +**Cancel a turn** by passing an `abortSignal` to `generate()`/`stream()`. +Aborting **interrupts the server‑side run**, not just the local socket, so the +chat stops generating and releases its resources instead of running on, +orphaned. Tearing down a `stream()` early (cancelling the stream) interrupts the +run too. + +**Bound each segment** with `requestTimeoutMs`. If a segment runs longer (e.g. +the server is wedged, or a workspace can't be scheduled), the run is interrupted +and the call rejects with a retryable `CoderChatError` (`kind: "timeout"`) +instead of hanging: + +```ts +const agent = new CoderAgent({ /* … */ requestTimeoutMs: 120_000 }); +``` + +A segment is one model round‑trip, until it settles or pauses for a client tool. +A multi‑step `generate()` that drives client tools runs several segments, so +`requestTimeoutMs` bounds each one, not the whole call. + +**Cap total wall‑clock** of a multi‑step call with a deadline signal instead: + +```ts +await agent.generate({ prompt: "…", abortSignal: AbortSignal.timeout(120_000) }); +``` + +**Stream drops heal themselves.** If the event stream drops mid‑turn, the agent +redials it automatically with exponential backoff, replays the turn's events from +its starting cursor, and deduplicates them on receipt. The server keeps +generating during the gap, so a transient drop costs nothing and the run is +**not** interrupted. + +Only when the stream cannot be re‑established (several consecutive failed +attempts, ~15s) is the server run interrupted and the call rejected with a +`CoderStreamError`, an AI SDK `APICallError`: + +| `isRetryable` | When | Then | +| ------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | --------------------------------------------------------------------------------------------------------------------------- | +| `true` | The failed turn had just created its chat **and** had no external effects a replay would repeat: no `workspaceId`, no `mcpServerIds`, and no freshly uploaded inline attachments (pre‑uploaded `fileId` references are fine) | The dead session is discarded. `generate()` calls with `maxRetries` set retry the whole turn on a fresh chat automatically. | +| `false` | Otherwise | Retrying is the caller's deliberate decision. | + +
+Why the other cases aren't retryable + +On a chat with prior state (resumed sessions, later turns, tool‑result +segments), a re‑invocation would resubmit the same prompt as a new user turn. +Workspace/MCP tools may already have executed side effects, and inline +attachments would upload again. + +
+ +- For `stream()`, a mid‑stream failure surfaces on the stream itself, outside + the SDK's retry wrapper. Handle it in your consumption loop. +- A non‑transient 4xx upgrade rejection (bad/expired token, deleted chat) fails + fast with a `CoderApiError` instead of retrying. 408/425/429 consume the redial + budget like any other transient failure. ## Handling errors @@ -512,6 +631,64 @@ try { SDK‑level retry could duplicate a turn. Prefer catching `retryable` errors and retrying the whole step deliberately. +## Cleanup + +The agent is an **async disposable**, so cleanup can ride scope exit instead of a +`finally` you have to remember: + +```ts +await using agent = new CoderAgent({/* … */}); +const { text } = await agent.generate({ prompt: "…" }); +// agent.interrupt() + agent.archive() run automatically when the scope exits. +``` + +- Disposal interrupts any in‑flight run, then archives. +- Disposal is **bounded and never throws** (~15s overall, best‑effort). Its + errors are swallowed so they can't mask the scope's own error. Call + `archive()` directly when you need guaranteed cleanup. +- In a request handler that returns before a fire‑and‑forget `archive()` + settles, the archive can be abandoned. `await using` (or an awaited + `archive()` in `finally`) avoids accumulating live chats. + +**What `archive()` does:** + +- It soft‑hides the chat: the chat stays in listings as `archived: true`. There + is no hard delete yet. +- A freshly interrupted chat keeps winding down server‑side for a few seconds, + and archiving 409s meanwhile. `archive()` retries those 409s (~1s apart, up to + ~15s overall; tune with `settleDeadlineMs` / `settleRetryDelayMs`) and + rethrows the last one if the chat never settles. +- Any other failure, including your own abort, rethrows immediately. + +**Return values.** Both methods report what they acted on instead of silently +no‑oping: + +| Method | Acted on a chat | No chat exists at all | +| ------------- | ----------------------------------------------------------------------------------------------- | ------------------------ | +| `archive()` | `{ archived: true, chatId, archivedChatIds }` (each archived id is cleared as a cleanup target) | `{ archived: false }` | +| `interrupt()` | `{ interrupted: true, chatId }` | `{ interrupted: false }` | + +
+Which chat archive() and interrupt() target after a session is dropped + +- They target the session's chat. After the session was dropped — + `resetSession()`, or the automatic discard after a fresh‑chat stream failure + (see [Handling errors](#handling-errors)) — they target the **last‑known chat + id** (`agent.lastKnownChatId`), so a stranded chat is still cleaned up instead + of leaking. +- Generation never uses the last‑known id: a turn after a drop creates a fresh + chat as always. +- Every chat stranded by an _automatic_ discard is also recorded on a ledger, + `agent.strandedChatIds` (oldest first). With `maxRetries`, several failed + attempts can strand one chat each while only the final attempt's error + surfaces. One `archive()` retires them all, oldest first, alongside its + primary target. +- Deliberate abandonment is different: `resetSession()` does **not** add to the + ledger (you may want that chat kept). After a manual reset, the old chat is + targetable only until a new chat supersedes `lastKnownChatId`. + +
+ ## Usage & cost `result.usage` reflects what the whole turn actually consumed: @@ -562,87 +739,15 @@ return result.toUIMessageStream({ }); ``` -## Sources - -Model configs with web search enabled emit `source` parts. They flow through to -`result.sources` and, in UI message streams, to `source-url` parts. Pass -`sendSources: true` to `toUIMessageStream`; the AI SDK omits them by default. -Earlier releases dropped them. - -## Structured output - -Coder Agents has no server‑side `response_format`, so `CoderAgent` cannot -constrain what the model **says** to a JSON schema. A `responseFormat` / -`experimental_output` request emits a warning and is best‑effort at most. - -- **Pure text‑in / JSON‑out, no server‑side tools** → use - [`@coder/ai-sdk-provider`](../provider) with `generateObject` / - `Output.object` (schema‑constrained; requires AI Gateway on the deployment). -- **The answer must come out of an agent run** (server‑side tools, MCP, a - workspace) → have the model submit its answer by _calling a tool_ whose - `inputSchema` is your Zod schema. The answer arrives as the tool call's typed - `input`. - -The tool pattern needs four rules to stay robust (validate client‑side, never -stop on the call, and more). Guide: [docs/structured-output.md](./docs/structured-output.md). -Copyable helper: [`examples/06-structured-output.ts`](./examples/06-structured-output.ts). - -## Workspace previews - -When the agent is bound to a workspace (the `workspaceId` setting), resolve — and -share — the browser URL where a workspace port is served, e.g. the dev server the -agent just started: - -```ts -const { url } = await agent.getPreview({ port: 3000 }); -// → https://3000--main--dev--alice.apps.example.com (private to the workspace owner) - -const shared = await agent.sharePreview({ port: 3000, shareLevel: "authenticated" }); -// shared.url is now reachable by any logged-in user; shared.shareLevel is the level in effect -``` - -| Method | What it does | -| ------------------------------------------------------ | ------------------------------------------------------------------------------------------------------------------------ | -| `getPreview({ port, agentName?, protocol?, signal? })` | Composes the subdomain URL. The URL honors the port's current share level: private to the workspace owner unless shared. | -| `sharePreview({ port, shareLevel?, … })` | Additionally upserts the port's share level (re‑invoking updates it in place) and returns the level in effect. | - -- Ports below 1000 are rejected up front. Coder subdomain URLs only encode 4–5 - digit ports, so `80--agent--…` would be parsed as an app named "80" and never - resolve; serve the preview on a higher port. -- `agentName` is optional when the workspace has exactly one agent. With - several, the error lists the candidates. -- `protocol: "https"` means the app speaks TLS _inside the workspace_. It adds - the `s` label suffix (`3000s--…`) and does not affect the browser scheme. -- `shareLevel` values: - - | `shareLevel` | Reachable by | - | --------------------------- | ---------------------------------------------------------------------- | - | `"authenticated"` (default) | Any logged‑in user | - | `"organization"` | Members of the workspace's organization; requires a newer Coder server | - | `"public"` | Anyone — no auth at all; mind what the port serves | - - Reverting to owner‑only means deleting the share; `"owner"` is not accepted on - upsert. - -**Credentials.** The preview helpers call non‑chat endpoints, so they need -`baseUrl` + `token`. Pass them alongside `client` if you construct one yourself -(or let them default from `CODER_URL`/`CODER_SESSION_TOKEN`). - -
-Compatibility and failure cases (wildcard access URL, Coder < v2.9) - -Both helpers are built on the stable v2 workspace APIs (workspace lookup + the -wildcard apps host; `sharePreview` adds a port‑share upsert), so they work -against old Coder servers — no experimental endpoints. - -They fail clearly instead of returning broken URLs: +## Observability -- A deployment without a wildcard access URL (`--wildcard-access-url`) yields an - explanatory error. -- A server that predates port sharing (< Coder v2.9) yields a 404 - `CoderApiError` saying so. +Pass `onTransportEvent` to receive typed events for HTTP exchanges, the per‑chat +stream's WebSocket lifecycle, and turn‑segment boundaries. You get timing and +tracing without wrapping `fetch`/`webSocketFactory` or re‑parsing stream frames. -
+Handler exceptions are swallowed, there is zero overhead without a handler, and +events carry no headers or tokens. Example, event reference, and semantics: +[docs/observability.md](./docs/observability.md). ## Workspaces & quota @@ -653,97 +758,14 @@ settle. Read [docs/workspaces-and-quota.md](./docs/workspaces-and-quota.md) before running fleets: how chats bind to workspaces, fleet sizing, autostop and cleanup, preventing stuck turns, and troubleshooting. -## Configuration - -`CoderAgentSettings`: - -| field | description | -| --------------------------------- | ------------------------------------------------------------------------------------------------------------------ | -| `client` \| (`baseUrl` + `token`) | connection (one or the other; `baseUrl`/`token` default from `CODER_URL`/`CODER_SESSION_TOKEN`) | -| `organizationId` | org UUID that owns the chat (required) | -| `model` | model hint: UUID, `provider:model`, model id, or display‑name substring | -| `reasoningEffort` | reasoning effort for chat creation and every user-message submission (see below) | -| `instructions` | system prompt | -| `tools` | AI SDK `ToolSet` (client‑executed) | -| `workspaceId` | bind the chat to a Coder workspace (enables workspace‑scoped tools) | -| `workspaceFiles` | adapter enabling `uploadToWorkspace()` (write files to the workspace FS) | -| `mcpServerIds` | server‑side MCP servers to enable | -| `planMode` | enable plan mode (`"plan"`) | -| `stopWhen` | AI SDK stop condition(s); default `stepCountIs(64)` | -| `maxRetries` | default `0` — SDK retries can duplicate server‑side turns; override with care | -| `requestTimeoutMs` | per‑turn time budget (ms); interrupts the run and rejects (`kind: "timeout"`) instead of hanging | -| `onTransportEvent` | observability hook for typed transport events (see [Observability](#observability)) | -| `settleDeadlineMs` | overall deadline for bounded cleanup (`archive()` 409 retries, disposal); default 15 000 | -| `settleRetryDelayMs` | pause between `archive()` retries while the chat settles; default 1000 | -| `chatId` | resume an existing chat | -| `lastSeenMessageId` | resume cursor for `chatId` (persist `agent.lastSeenMessageId`) — skips the resumed turn's pre‑prompt history probe | - -**Model hint resolution.** The `model` hint resolves against the organization's -model configs, first match wins: - -1. a config UUID, used as‑is -2. an exact `provider:model` match -3. an exact model id -4. a display‑name substring (case‑insensitive) -5. a model‑id substring - -An unresolvable hint falls back to the server's default model instead of -failing. Use `agent.listModels()` to see what's available. - -
-Where model configs come from, and older deployments - -- Configs come from - `GET /api/v2/organizations/{organizationId}/chats/models`, with the provider - type joined from the response's provider descriptors. -- On older deployments where the organization‑scoped route does not exist yet - (404), resolution falls back once to the legacy deployment‑wide - `/api/experimental/chats/model-configs` listing, independently of chat‑prefix - selection. -- Partial payloads from older/newer servers are tolerated: entries match on the - fields they carry. - -
- -### Reasoning effort - -Set `reasoningEffort` on `CoderAgentSettings` (or `CoderLanguageModelConfig` -when using the model directly): - -```ts -import { CoderAgent } from "@coder/ai-sdk-agent"; - -const agent = new CoderAgent({ - organizationId: "your-org-uuid", - reasoningEffort: "low", -}); -await agent.generate({ prompt: "Reply with exactly: pong" }); -const chat = await agent.client.getChat(agent.chatId!); -console.log(chat.last_reasoning_effort); // string, null, or absent on the wire -await agent.archive(); -``` - -- **Values.** The global scale is `none`, `minimal`, `low`, `medium`, `high`, - `xhigh`, `max`. The exported `ReasoningEffort` type also accepts other strings - for forward compatibility. -- **Per‑model options.** Model configs returned by `agent.listModels()` / - `client.listModelConfigs(organizationId)` expose the optional - `reasoning_efforts` list of selectable values for each model. -- **Independent of the `model` hint.** -- **No per‑call override.** Configure it on the agent/model instance. - -
-When reasoning_effort is sent, and older servers - -- It is sent as `reasoning_effort` on chat creation and every - user‑message submission, including submissions to resumed or busy chats that - queue the message. When unset, the request key is omitted entirely. - Tool‑result submissions and WebSocket reconnects do not post user messages and - do not send this field. -- Older Coder servers that do not recognize `reasoning_effort` ignore the - unknown JSON field, so setting it is safe but has no effect there. +## Durable workflows: persist, resume, recover -
+Run **one agent session across process boundaries** — queue jobs, +durable‑workflow steps (Vercel Workflow, step functions, Temporal, …), cron +ticks — and survive crashes, stream drops, and timeouts in between. All chat +state lives on the Coder server, so a workflow carries only `agent.chatId` (plus, +optionally, the `agent.lastSeenMessageId` resume cursor) between steps, and +each turn runs inside a durable step. Guide: [docs/durable-workflows.md](./docs/durable-workflows.md). ## How it works @@ -775,15 +797,6 @@ double‑counts:
-## Durable workflows: persist, resume, recover - -Run **one agent session across process boundaries** — queue jobs, -durable‑workflow steps (Vercel Workflow, step functions, Temporal, …), cron -ticks — and survive crashes, stream drops, and timeouts in between. All chat -state lives on the Coder server, so a workflow carries only `agent.chatId` (plus, -optionally, the `agent.lastSeenMessageId` resume cursor) between steps, and -each turn runs inside a durable step. Guide: [docs/durable-workflows.md](./docs/durable-workflows.md). - ## Testing ```bash From 08611ee70b5ff71fe97379421ec7dc81cb24a60c Mon Sep 17 00:00:00 2001 From: Thomas Kosiewski Date: Thu, 24 Sep 2026 10:15:47 +0000 Subject: [PATCH 09/10] docs(agent): scope cutShort pin to timeouts and point rule links at structured-output guide Change-Id: I9af034917593a3063392a959edb0ab2efa0523f1 Signed-off-by: Thomas Kosiewski --- packages/agent/docs/durable-workflows.md | 14 +++++++------- packages/agent/docs/workspaces-and-quota.md | 3 +-- 2 files changed, 8 insertions(+), 9 deletions(-) diff --git a/packages/agent/docs/durable-workflows.md b/packages/agent/docs/durable-workflows.md index e0f45822..7422b27e 100644 --- a/packages/agent/docs/durable-workflows.md +++ b/packages/agent/docs/durable-workflows.md @@ -274,11 +274,11 @@ history; tool effects that already ran are not undone. `retryable` means the _failure_ is transient, not that a re‑run is free: retry only steps that tolerate re‑submission, and reconcile first otherwise. -| Error | Cause | Before retrying | -| --------------------------------------------- | ------------------------------------------------------------------------------------------------------------------ | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | -| `CoderChatError`, `retryable: true` | `requestTimeoutMs` expired (`kind: "timeout"`) or a transient turn failure (`kind: "stream_closed"`, upstream 5xx) | Chat survives. Run the [settle‑wait](#recover-after-a-crash); pin `cutShort = true` in the [result check](#recover-the-result-before-resubmitting). | -| `CoderStreamError` (an AI SDK `APICallError`) | The stream could not be re‑established | `isRetryable: true` only for a turn that just created its chat with no workspace, no MCP servers, and no fresh inline uploads (the SDK then discards the session). Always `false` on a resumed chat. | -| `CoderApiError` (every non‑2xx) | An HTTP request failed | Back off and retry 408, 425, 429, 5xx. Fail the workflow on 401/403/404 and the archived‑chat 400 — every attempt hits the same wall. | +| Error | Cause | Before retrying | +| --------------------------------------------- | ------------------------------------------------------------------------------------------------------------------ | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | +| `CoderChatError`, `retryable: true` | `requestTimeoutMs` expired (`kind: "timeout"`) or a transient turn failure (`kind: "stream_closed"`, upstream 5xx) | Chat survives. Run the [settle‑wait](#recover-after-a-crash) before resubmitting. After a `timeout`, also pin `cutShort = true` in the [result check](#recover-the-result-before-resubmitting): the expiry already fired an interrupt. | +| `CoderStreamError` (an AI SDK `APICallError`) | The stream could not be re‑established | `isRetryable: true` only for a turn that just created its chat with no workspace, no MCP servers, and no fresh inline uploads (the SDK then discards the session). Always `false` on a resumed chat. | +| `CoderApiError` (every non‑2xx) | An HTTP request failed | Back off and retry 408, 425, 429, 5xx. Fail the workflow on 401/403/404 and the archived‑chat 400 — every attempt hits the same wall. |
CoderChatError: why wait, and why pin cutShort @@ -838,7 +838,7 @@ see:
Structured‑output steps: recover the call's input, not the text -A [structured output](../README.md#structured-output) step's answer is the +A [structured output](./structured-output.md) step's answer is the `structured_output` call's typed input, which rehydrates as a `dynamic-tool` part on the recovered turn ([Rehydrating chat history](../README.md#rehydrating-chat-history)). @@ -846,7 +846,7 @@ part on the recovered turn join above cuts at it and returns only the ack prose that follows. Recover the filed call instead: this scan replaces the text join _inside_ the recovery branch above, reusing its `parts`, and validates client‑side exactly like the -live path (rule 2 there — the schema is the real gate): +live path ([rule 2](./structured-output.md#2-validate-clientside) — the schema is the real gate): ```ts // Scan backward to the last call that VALIDATES, as in the live path — a diff --git a/packages/agent/docs/workspaces-and-quota.md b/packages/agent/docs/workspaces-and-quota.md index ed1086c3..81133221 100644 --- a/packages/agent/docs/workspaces-and-quota.md +++ b/packages/agent/docs/workspaces-and-quota.md @@ -198,8 +198,7 @@ The event looks like
Turn settled status: "requires_action"; follow‑up messages queue forever - **Likely cause:** the loop ended on an unanswered client tool call. -- **Fix:** submit the stranded results or interrupt — see rule 4 under - [Structured output](../README.md#structured-output). If a crash left the +- **Fix:** submit the stranded results or interrupt — see [rule 4 of the structured-output guide](./structured-output.md#4-settle-a-turn-that-stopped-on-a-tool-call). If a crash left the pause behind, reconcile effects first ([Make client tools crash-safe](./durable-workflows.md#make-client-tools-crash-safe)). From c23533c2e022aa9fcfd723f75111a7290113a3e5 Mon Sep 17 00:00:00 2001 From: Thomas Kosiewski Date: Thu, 24 Sep 2026 10:25:50 +0000 Subject: [PATCH 10/10] build: ship docs/ in the agent, sandbox, and provider npm packages Change-Id: I4e94061f65d31cc017ec19e2aabf2d9af49a542e Signed-off-by: Thomas Kosiewski --- packages/agent/package.json | 1 + packages/provider/package.json | 1 + packages/sandbox/package.json | 1 + 3 files changed, 3 insertions(+) diff --git a/packages/agent/package.json b/packages/agent/package.json index d2375a47..544bf950 100644 --- a/packages/agent/package.json +++ b/packages/agent/package.json @@ -25,6 +25,7 @@ }, "files": [ "dist", + "docs", "README.md" ], "type": "module", diff --git a/packages/provider/package.json b/packages/provider/package.json index 4a18472a..19f2319e 100644 --- a/packages/provider/package.json +++ b/packages/provider/package.json @@ -27,6 +27,7 @@ }, "files": [ "dist", + "docs", "README.md" ], "type": "module", diff --git a/packages/sandbox/package.json b/packages/sandbox/package.json index 097f1721..52fb6512 100644 --- a/packages/sandbox/package.json +++ b/packages/sandbox/package.json @@ -22,6 +22,7 @@ }, "files": [ "dist", + "docs", "README.md" ], "type": "module",