Skip to content

Commit 37e99e9

Browse files
committed
docs(skills): a hardening change must not turn a failing request into a succeeding one
The most valuable learning of the sweep, and the one every other check missed — sixteen PRs' own suites, both review bots, and the author. Trimming a path identifier looks strictly safer. It is not when the identifier previously went out through a bare encodeURIComponent and reaches a destructive endpoint: box_sign_cancel_request went from a 404 no-op to cancelling a real signature request, and delete_r2_bucket from naming no bucket to destroying prod-data. BigQuery's delete_dataset and delete_table had the same shape on projectId. Records the reasoning error that hid it: "trimming is the helper's contract at all 137 sites" is an average, and the question is an intersection — parameters whose normalisation actually changed, crossed with irreversible operations. On that PR the answer was one of 137. The resolution is strictUrlPathSegment, argued from the values (no legitimate id carries surrounding whitespace, and the previous behaviour was already a clean failure), not from consistency. Two smaller ones folded in: - safeUrlPath rejects only a truly empty path component, never a whitespace-only one. Git tracks a file and a directory named only spaces, and the parser never removes %20%20%20. - A test that calls a function directly can pass while the wrapper does the opposite. executeTool catches a postProcess throw and restores the submit response, so eleven green Enrow failure-path tests sat over a production success: true.
1 parent 4a637e3 commit 37e99e9

2 files changed

Lines changed: 90 additions & 0 deletions

File tree

.agents/skills/add-tools/SKILL.md

Lines changed: 22 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -269,6 +269,26 @@ as slash-bearing — never to make a separator stop erroring on a single-segment
269269
`safeEncodedUrlPathSegment` arrive with the path-safety sweep; if your checkout only exports
270270
`safeUrlPathSegment`, add them there rather than hand-rolling a local encoder.)
271271

272+
### Never let a new guard rescue a request that used to fail
273+
274+
If a parameter you are now routing through a helper previously went out raw or through a bare
275+
`encodeURIComponent`, the helper's trimming is a **behaviour change**, not a tightening: a padded
276+
value that used to 404 now names a real resource. On a DELETE, a cancel, or a revoke that is a
277+
destructive action the caller never asked for.
278+
279+
For a newly-trimmed identifier on an irreversible request, use `strictUrlPathSegment`
280+
(`apps/sim/tools/strict-url-path.ts:41`) — `safeUrlPathSegment` plus a refusal of surrounding
281+
whitespace — rather than trimming. Identifiers that were already trimmed before your change keep
282+
plain `safeUrlPathSegment`. The full rule, its two confirmed instances, and how to scope the check are
283+
in **A hardening change must not turn a failing request into a succeeding one** in
284+
`.agents/skills/validate-integration/SKILL.md`.
285+
286+
Note the matching asymmetry inside `safeUrlPath`: it rejects only a **truly empty** path component
287+
(`url-path.ts:317`), never a whitespace-only one. Git tracks a file and a directory whose entire name
288+
is spaces, and the URL parser never removes `%20%20%20` the way it removes a dot segment — so
289+
rejecting it has no security value and breaks a legitimate path. `safeUrlPathSegment` still rejects an
290+
all-whitespace value, because it trims opaque ids first.
291+
272292
### `params.x?.trim()` guards `undefined`, not the type
273293

274294
A param's declared `type: 'string'` is enforced nowhere between the LLM tool call — or a
@@ -592,6 +612,8 @@ All tool IDs MUST use `snake_case`: `{service}_{action}` (e.g., `x_create_tweet`
592612
- [ ] All params have appropriate `visibility`
593613
- [ ] No param is named `timeout`, `proxyUrl`, or `method` unless it means what the transport means
594614
- [ ] Every param interpolated into a request path goes through a `tools/url-path.ts` helper
615+
- [ ] No newly-guarded parameter on a destructive request turns a previously failing call into a
616+
succeeding one — those use `strictUrlPathSegment`, not `safeUrlPathSegment`
595617
- [ ] All nullable response fields use `?? null`
596618
- [ ] All optional outputs have `optional: true`
597619
- [ ] No raw JSON dumps in outputs

.agents/skills/validate-integration/SKILL.md

Lines changed: 68 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -381,6 +381,56 @@ A sound suite has five properties:
381381
- [ ] The skipped/unbuildable ledger is asserted empty against a justified allowlist
382382
- [ ] Legitimate dot-bearing values pass through unchanged
383383

384+
### A hardening change must not turn a failing request into a succeeding one
385+
386+
State it in exactly those words, and apply it to every guard you add. It is the check that caught a
387+
class of regression the whole sweep otherwise missed — sixteen PRs' own suites, both review bots on
388+
several passes, and the author.
389+
390+
Adding a guard that **trims** a path identifier looks strictly safer. It is not, when that identifier
391+
reaches a destructive endpoint, because trimming is only neutral if the untrimmed value already
392+
worked. Where the parameter previously went out through a bare `encodeURIComponent`, it did not:
393+
394+
```
395+
box_sign_cancel_request (apps/sim/tools/box_sign/cancel_request.ts:34)
396+
before: /2.0/sign_requests/%20%20<uuid>%20%20/cancel -> 404, no-op
397+
after: /2.0/sign_requests/<uuid>/cancel -> cancels a real signature request
398+
399+
cloudflare_delete_r2_bucket (apps/sim/tools/cloudflare/delete_r2_bucket.ts:30)
400+
before: " prod-data " names no bucket that can exist -> request FAILS
401+
after: trimmed to "prod-data" -> DESTROYS the real bucket
402+
403+
google_bigquery_delete_dataset / _delete_table (delete_dataset.ts:53) — same shape on projectId
404+
```
405+
406+
**Reason about the intersection, not the guard's contract.** "Trimming is the helper's contract at all
407+
137 call sites" is an *average*, and it excuses the deletion above. The real question is a set
408+
intersection: which parameters does this change *newly* normalise, **and** which of those sit on an
409+
irreversible request (DELETE, cancel, revoke, purge, drop)? On the Cloudflare PR the answer was
410+
exactly **one of 137** — and it took four review passes escalating P2→P2→P1→P1, with two pushbacks,
411+
to establish it.
412+
413+
**The resolution: refuse the padded value on those parameters.** Not on consistency grounds — on two
414+
facts specific to the values themselves:
415+
416+
1. No legitimate identifier for these providers carries surrounding whitespace (a Box Sign id is a
417+
UUID; a GCP project id matches `[a-z][a-z0-9-]{5,29}`; an R2 bucket name is
418+
`^[a-z0-9][a-z0-9-]*[a-z0-9]`), so refusing excludes nothing a caller could really mean.
419+
2. The previous behaviour was already a clean failure, so refusing preserves it — and improves on it
420+
by replacing an opaque provider 404 with an error naming the parameter.
421+
422+
`strictUrlPathSegment` (`apps/sim/tools/strict-url-path.ts:41`) is `safeUrlPathSegment` with that
423+
precondition; `assertNoSurroundingWhitespace` (`:51`) is the shared check for body-value counterparts
424+
of the same identifier.
425+
426+
Parameters that were **already** trimmed before your change keep plain `safeUrlPathSegment` — that is
427+
not a change you are making, and tightening them would break callers whose stored value works today.
428+
429+
- [ ] Enumerated the parameters whose normalisation this change actually alters
430+
- [ ] Intersected that set with destructive operations and checked each member individually
431+
- [ ] Newly-trimmed identifiers on irreversible requests refuse the padded value rather than trimming it
432+
- [ ] Already-trimmed identifiers were left alone
433+
384434
### These tests are type-checked by nothing
385435

386436
`apps/sim/tsconfig.json` excludes `**/*.test.ts` and `**/*.test.tsx` from `include`, and
@@ -392,6 +442,20 @@ To type-check one, write a temporary tsconfig that extends `apps/sim/tsconfig.js
392442
`**/*.test.ts` exclusion, and includes only the harness — then delete it. Do not commit it; the
393443
exclusion exists deliberately.
394444

445+
### A test that calls a function directly can pass while the wrapper does the opposite
446+
447+
`executeTool` wraps every `postProcess` call in a catch that logs and then restores the
448+
pre-`postProcess` result (`apps/sim/tools/index.ts:1977` and `:2062`). For a submit-then-poll tool
449+
that pre-`postProcess` result is the **submit** response — `success: true` with every result field
450+
null. So a `postProcess` that throws on a timed-out or exhausted poll is reported to the user as a
451+
successful lookup that simply found nothing, and the hosted-key cost hook, gated on
452+
`finalResult.success` (`:1987`), bills it.
453+
454+
Eleven Enrow failure-path tests asserted that throwing contract by calling `postProcess` directly.
455+
Every one passed. Production reported `success: true`. Assert through the real call path — or, where
456+
a test must call the function directly, assert the shape the *executor* will hand back, not the one
457+
the function raises.
458+
395459
## Step 10: Validate Error Handling
396460

397461
- [ ] `transformResponse` checks for error conditions before accessing data
@@ -513,6 +577,10 @@ After fixing, confirm:
513577
`path_safety.test.ts` that enumerates (tool, param) pairs, asserts named rejection, probes
514578
conditional and presence branches, and asserts an empty skip ledger
515579
- [ ] Validated no param collides with a transport-reserved name (`timeout`, `proxyUrl`, `method`)
580+
- [ ] Confirmed no hardening change turns a failing request into a succeeding one — newly-normalised
581+
parameters intersected with destructive operations, each checked individually
582+
- [ ] Confirmed failure-path tests assert through the real call path, not a direct call the executor
583+
wraps differently
516584
- [ ] Validated error handling (error checks, meaningful messages)
517585
- [ ] Validated registry entries (tools and block, alphabetical, correct imports)
518586
- [ ] Validated model-visible/opaque inputs and Sim-durable/internal-execution provenance at their

0 commit comments

Comments
 (0)