feat(cache): add tag-based caching and revalidation helpers - #1964
feat(cache): add tag-based caching and revalidation helpers#1964dinwwwh wants to merge 31 commits into
Conversation
…implementation-09e313 # Conflicts: # README.md # apps/content/docs/procedure.mdx # packages/ai-sdk/README.md # packages/arktype/README.md # packages/bun/README.md # packages/client/README.md # packages/cloudflare/README.md # packages/contract/README.md # packages/effect/README.md # packages/evlog/README.md # packages/hibernation/README.md # packages/json-schema/README.md # packages/nest/README.md # packages/next/README.md # packages/node/README.md # packages/openapi/README.md # packages/opentelemetry/README.md # packages/pinia-colada/README.md # packages/pino/README.md # packages/publisher/README.md # packages/ratelimit/README.md # packages/server/README.md # packages/server/src/procedure-client.test.ts # packages/shared/README.md # packages/swr/README.md # packages/tanstack-query/README.md # packages/trpc/README.md # packages/valibot/README.md # packages/zod/README.md # pnpm-lock.yaml
More templates
@orpc/ai-sdk
@orpc/arktype
@orpc/bun
@orpc/experimental-cache
@orpc/client
@orpc/cloudflare
@orpc/contract
@orpc/experimental-effect
@orpc/evlog
@orpc/hibernation
@orpc/json-schema
@orpc/experimental-msw
@orpc/nest
@orpc/next
@orpc/node
@orpc/openapi
@orpc/opentelemetry
@orpc/pinia-colada
@orpc/pino
@orpc/publisher
@orpc/ratelimit
@orpc/server
@orpc/shared
@orpc/swr
@orpc/tanstack-query
@orpc/trpc
@orpc/valibot
@orpc/zod
commit: |
Deploying with
|
| Status | Name | Latest Commit | Preview URL | Updated (UTC) |
|---|---|---|---|---|
| ✅ Deployment successful! View logs |
orpc | 1c71159 | Commit Preview URL Branch Preview URL |
Sep 04 2026, 09:00 AM |
Codecov Report✅ All modified and coverable lines are covered by tests. 📢 Thoughts on this report? Let us know! |
Merging this PR will improve performance by 10.45%
|
| Benchmark | BASE |
HEAD |
Efficiency | |
|---|---|---|---|---|
| ⚡ | flat object from query params |
116.8 µs | 105.8 µs | +10.45% |
Tip
Curious why performance improved? Comment @codspeedbot explain why performance improved on this PR, or directly use the CodSpeed MCP with your agent.
Comparing claude/orpc-cache-implementation-09e313 (12694be) with main (88ee055)
![pullfrog[bot]](https://avatars.githubusercontent.com/in/1768019?s=60&v=4){kind=small}
There was a problem hiding this comment.
Important
One behavioral issue to resolve: a revalidation failure after a committed mutation surfaces as an error on a request whose write already succeeded. See the inline comment on revalidate.
Reviewed changes
@orpc/cache(new package) —cache()/revalidate()middlewares,CacheStorecontract, tag-version invalidation, stale-while-revalidate,CacheHandlerPluginheader reflection, andMemoryCacheStore/RedisCacheStore/VercelCacheStoreadapters.@orpc/cloudflare—KVCacheStore(real KV bindings) and purge-onlyWorkersCacheStore, plus workerd coverage.@orpc/shared— newdeepSortKeysutil and tests.- Docs/config — new
docs/helpers/cachepage, README/package-list updates, api-reference row, new packagepackage.jsonwith subpath exports, workspace wiring.
Overall this is a careful, well-tested addition. I verified the highest-risk semantics rather than taking them on faith: the tag-version technique errs on the safe side (a lost concurrency race produces a spurious miss and recompute, never a stale hit), the tag header encoding round-trips correctly under case-folding and stays consistent between the reflected cache-tag and WorkersCacheStore purge, blob/streaming outputs are guarded where they cannot be stored, and the docs call out the CDN/purge-store and per-request-shared-key caveats. Two non-blocking nits are inline.
DeepSeek Flash (free via Pullfrog for OSS) ｜ 𝕏
| const resolvedTags = toArray(await value(tags, middlewareOptions, input)) | ||
|
|
||
| if (resolvedTags.length) { | ||
| await (middlewareOptions.context as CacheContext).cache.revalidateTag(resolvedTags as [string, ...string[]]) |
There was a problem hiding this comment.
revalidateTag is awaited with no guard, so when the store is unreachable (e.g. a transient Redis outage) a mutation whose handler already succeeded is reported to the client as a failed request. Clients that retry on error will re-run the mutation, risking a double write/commit. This contrasts with the stale-refresh path just above, which deliberately swallows its background failures (.catch(() => {})).
Consider treating revalidation as best-effort after a successful procedure — catch/log and still return result — so a cache outage can never turn a committed mutation into an error response. If a loud failure is deliberately wanted for observability, that's defensible too, but it should be a documented, conscious choice given the retry implications.
Technical details
# Revalidate errors flip committed mutations into failures
## Affected sites
- packages/cache/src/middleware.ts:167 — `await (…context as CacheContext).cache.revalidateTag(…)` has no try/catch; the error propagates to the caller after `next()` already committed the mutation.
## Required outcome
- A successful procedure must not surface a client-facing error when cache revalidation fails afterward.
## Suggested approach
- Wrap `revalidateTag` (and the tag/value resolution) so revalidation failures are logged/silently dropped and the committed `result` is still returned — mirroring the SWR refresh path's `.catch(() => {})`.
## Open questions
- Is fail-loud is the intended contract here? If so, document it, since the SWR refresh path deliberately does the opposite.| } | ||
| } | ||
|
|
||
| function isUncacheableOutput(output: unknown): boolean { |
There was a problem hiding this comment.
Nit: isUncacheableOutput only catches top-level async iterators and ReadableStream, so a top-level Blob output is still passed to store.set. MemoryCacheStore.set stores it (in-memory it works), but RedisCacheStore/KVCacheStore/VercelCacheStore drop it, so caching semantics silently differ per adapter. Consider also gating Blob (and FormData, which RPCSerializer emits for nested blobs) here so the behavior is uniform regardless of store.
There was a problem hiding this comment.
Important
One behavior to confirm from the delta's cleanup: dropping isUncacheableOutput from the middleware also dropped the "don't record a cache check for streaming outputs" guarantee, so streaming responses are now advertised as cache activity even though every adapter ignores them. Full context inline.
Reviewed changes
This run reviews the single-commit delta e93c93e5 since the last-reviewed head 17c31c21:
CacheStore.revalidateTags— replacedrevalidateTag(string | list)with a non-empty tag list across the interface, all four adapters, and their mocked/integration tests; stores dropped their empty-list branches and the middleware now guards withtags?.length.revalidate({ tags })API — options-object form whosetagsis aValueof a non-empty tuple orundefined; resolving toundefined(or[]) skips both the store call and the plugin-context recording.- Context keys —
CacheContextmoved to namespaced'cache/store'/'cache/waitUntil'; tests, type tests, docs, and e2e all updated, with nocontext.cache/context.waitUntilstragglers. cache()key semantics — any providedkey(string or object) is now used as given; only the omitted-key default derives[path, input]; documented as "procedures sharing a key share an entry".- Streaming-output drops moved into stores —
isUncacheableOutputdeleted;MemoryCacheStore.setnow ignoresReadableStream/async-iterator outputs, matching Redis/KV/Vercel.
DeepSeek Flash (free via Pullfrog for OSS) ｜ 𝕏
`CacheStore.revalidateTag` becomes `revalidate({ tags })`, taking a non-empty
tag list, and every duration is now in seconds rather than milliseconds,
matching what Redis, Workers KV, the Vercel Runtime Cache, and `Cache-Control`
all accept. Entries without tags carry `undefined` instead of an empty array,
and stores no longer inspect output, passing it straight to their serializer.
The store and its background-work hook move to the namespaced `cache/store`
and `cache/waitUntil` context keys. A background refresh is handed over
uncaught so `cache/waitUntil` can report its failures. `revalidate` takes an
options object with a required `tags`, and a provided `key` is used as given.
The handler plugin takes header names as plain literals rather than exported
constants, sets them whatever the request method, and emits `max-age` instead
of `s-maxage`, which carries the `proxy-revalidate` semantics that would
forbid the stale reuse `stale-while-revalidate` grants.
Tag header encoding and `nowInSeconds` move to `@orpc/shared`, and each store
builds its key serializer once instead of per call.
…mplementation-09e313 # Conflicts: # README.md # packages/ai-sdk/README.md # packages/arktype/README.md # packages/bun/README.md # packages/client/README.md # packages/cloudflare/README.md # packages/cloudflare/package.json # packages/contract/README.md # packages/effect/README.md # packages/evlog/README.md # packages/hibernation/README.md # packages/json-schema/README.md # packages/nest/README.md # packages/next/README.md # packages/node/README.md # packages/openapi/README.md # packages/opentelemetry/README.md # packages/pinia-colada/README.md # packages/pino/README.md # packages/publisher/README.md # packages/ratelimit/README.md # packages/server/README.md # packages/shared/README.md # packages/swr/README.md # packages/tanstack-query/README.md # packages/trpc/README.md # packages/valibot/README.md # packages/zod/README.md # pnpm-lock.yaml
- RedisCacheStore and experimental_KVCacheStore take their client as the first argument - experimental_WorkersCacheStore defaults to the cache exported by cloudflare:workers - Redis, Upstash, and Bun stores share the shorter e:/t: key families and one envelope format - race-condition tests for every store and the cache middleware
e93c93e to
8a12845
Compare
- Redis and Upstash stores share one Upstash database over rediss:// and REST - Bun and Redis stores share REDIS_URL - race tests wait for the held read to complete before the racing revalidation
There was a problem hiding this comment.
Important
Removing the output guards from every store turned "un-storable output → cache miss" into "un-storable output → stored and served as {}", so a procedure returning a Blob/File/FormData/ReadableStream/async iterator now serves a corrupted empty value on every within-ttl hit instead of recomputing. The tag-invalidation semantics themselves check out — a revalidation race can only produce a spurious miss, never a stale serve. Two nits inline (a runtime-only Upstash edge and a stale JSDoc).
Reviewed changes
This run reviews the PR-owned delta since the last-reviewed head e93c93e5. The branch was force-pushed — e93c93e5 was replaced by a rework commit, main was re-merged, and new work landed — so the substantive delta is commits c2356fbc, e9c80635, and 8a12845b.
- Store contract rework —
revalidateTag(list)becamerevalidate({ tags })with a non-empty tag list; every duration switched from milliseconds to seconds across adapters, envelopes, middleware, and docs; untagged entries now carryundefinedinstead of[]; key serializers are built once per store andencodeCacheKey/nowInSecondsmoved to@orpc/shared. - Output guards removed — every store now passes output straight to its serializer; the blob/stream/iterator drop logic and its tests were deleted, and the docs replaced the "every adapter ignores them" guarantee with a do-not-cache warning.
- Handler plugin rework —
headersis now a required list of plain literals (exported constant helpers removed);cache-tag/cache-controlare set on any request method and override existing headers;cache-controlswitched froms-maxagetomax-age(proxy-revalidate semantics), with stale hits reflectingmax-age=0. - Background refresh semantics — a stale-hit refresh is handed to
cache/waitUntiluncaught so the runtime can report failures, and only.catch(() => {})d when nothing owns it. - New adapters —
UpstashCacheStore(@orpc/experimental-cache/upstash) andBunRedisCacheStore(@orpc/bun), sharing the Redis key/envelope format, plus a cross-adapter compatibility suite; all store constructors now take their client positionally. @orpc/shared—nowInSecondsand case-safeencodeCacheTag/encodeCacheTagHeader/decodeCacheTagHeader, each with tests.- Testing — a shared
describeCacheStoreContractsuite now runs against Memory/Redis/Upstash/Vercel, withholdResultrace tests, concurrency suites for the middleware, and a merge of upstream main (all CI green on the head).
Inline threads
packages/cache/src/middleware.ts:120— un-storable outputs are persisted and served as corrupted{}(IMPORTANT).packages/cache/src/adapters/upstash.ts:136—revalidate({ tags: [] })throws on Upstash, no-ops elsewhere (nit).packages/cache/src/handler-plugin.ts:47—headersJSDoc still claims GET/HEAD-only and never-override (nit).
DeepSeek Flash (free via Pullfrog for OSS) ｜ 𝕏
| for (const tag of tags) { | ||
| multi.incr(this.tagKey(tag)) | ||
| } | ||
| await multi.exec() |
There was a problem hiding this comment.
At runtime, revalidate({ tags: [] }) throws here — Upstash's Pipeline.exec() rejects with Error: Pipeline is empty — while Redis, Memory, Bun, and KV treat an empty list as a no-op. The middleware only guards if (tags), so a resolver producing [] at runtime reaches the store; the type-level non-empty tuple cannot stop a dynamic resolver. A one-line if (!tags.length) return keeps the adapters consistent.
…/waitUntil Without a runtime to hand the refresh to, the request now completes it before returning the stale output, so it cannot be dropped when the process or invocation ends. Failed refreshes still serve the stale entry.
…l only The refresh is started regardless and handed to cache/waitUntil when one is provided; without it nothing owns the refresh, so a failed one surfaces as an unhandled rejection rather than being awaited or swallowed.
There was a problem hiding this comment.
Important
This delta removes the fallback .catch from the stale-while-revalidate refresh path, so on the common Node setup (no cache/waitUntil wired) a failed background refresh now becomes an unhandled rejection — which terminates the process by default. Prior reviews' IMPORTANT threads on revalidate fail-loud and un-storable outputs stored as {} also remain open.
Reviewed changes
This run reviews the PR-owned delta since the prior pullfrog review at 8a12845b:
- Removed
experimental_KVCacheStorefrom@orpc/cloudflare— breaking; only the purge-onlyWorkersCacheStoreremains. Clean: no dangling references in code, exports,tsconfig.json, README, wrangler KV binding, or the docs page. - Reworked SWR refresh ownership — the middleware now hands the refresh to
cache/waitUntilunconditionally with no fallback catch, so without a waiter a refresh failure surfaces as an unhandled rejection. - Updated the handler-plugin
headersJSDoc to match current behavior (any method, overriding), and dropped the now-orphaned "overrides existing headers" test, leaving thepresetHeadersPlugin/presettest scaffolding unused. - Hardened the shared
holdResultrace-test helper with areadpromise so the racing-revalidation tests sequence deterministically across Memory/Redis/Upstash/Bun. - Expanded cross-adapter compatibility suites (tag counters, retention staging) and gated the Upstash integration suite off Node 26 with a TODO.
- Synced the packages table across every package README (markdown-table restyle).
Inline feedback anchors on the SWR refresh change. The two prior IMPORTANT threads (revalidate fail-loud; un-storable outputs persisted and served as {}) are unchanged by this delta and remain to address. Two nits are inline.
DeepSeek Flash (free via Pullfrog for OSS) ｜ 𝕏
| const refresh = Promise.resolve(middlewareOptions.next()) | ||
| .then(result => store.set(key, result.output, { tags, ttl, swr })) | ||
|
|
||
| middlewareOptions.context['cache/waitUntil']?.(refresh) |
There was a problem hiding this comment.
Without cache/waitUntil in the context — the default on runtimes other than Workers, since nothing auto-wires it — a failed stale-while-revalidate refresh is now an unhandled rejection, and Node terminates the process on an unhandled rejection by default. The previous delta deliberately .catch(() => {})d the refresh when nothing owned it; this delta drops that fallback. The new 'leaves refresh failures unhandled without waitUntil' test asserts the new contract, but outside the test runner a transient origin error during a background refresh would take the whole server down.
| } | ||
| } | ||
|
|
||
| function createTestingHandler(headers?: readonly CacheHandlerPluginHeader[], preset?: StandardHeaders) { |
There was a problem hiding this comment.
Removing the "sets its headers over existing ones" test left presetHeadersPlugin and the preset parameter of createTestingHandler with no callers, yet the docs/headers JSDoc still claim the plugin sets headers over anything already on the response — a documented behavior now without test coverage. Either restore the override test or drop the dead preset scaffolding.
1 participant
Adds
@orpc/experimental-cache, a new package for tag-based caching and revalidation of procedure outputs, with stale-while-revalidate, five store adapters, and a handler plugin that reflects cache activity into response headers for client-side revalidation (e.g. TanStack Query auto-invalidation on mutation) or HTTP response caches.Features
cache()middleware caches procedure output in the context'scache/store(one store per router). Keys default to the procedure path and full input, canonically encoded so structurally equal keys always hit the same entry; a providedkeyis used as given.key,tags,ttl,swr, andenabledare all dynamic on middleware options and input.ttlbut withinswrare served immediately while the procedure re-executes in the background; acache/waitUntilcontext hook keeps refreshes alive on Workers-like runtimes.revalidate({ tags })middleware invalidates tags after successful mutations, with compile-time non-empty tags; resolvingtagstoundefinedskips it.CacheHandlerPluginis inert by default; aheadersallowlist enablesorpc-cache-tag/orpc-cache-tag-invalidation(client-facing, never consumed by CDNs) andcache-control/cache-tag(for response caches in front, GET/HEAD only, never overriding). Only the root procedure's checks are reflected, never nested calls, and only on successful responses. Tag encoding survives Cloudflare Workers Caching's strict rules: printable ASCII only, and uppercase percent-encoded so case-insensitive matching cannot collide distinct tags.MemoryCacheStore,RedisCacheStore,VercelCacheStore(@orpc/experimental-cache), plusexperimental_KVCacheStoreand the purge-onlyexperimental_WorkersCacheStorein@orpc/cloudflare, following theexperimental_prefix convention for experimental APIs inside stable packages (with anew-caplint exception to support it). All share theCacheStorecontract and a uniform options-object constructor;revalidateTagstakes a non-empty tag list so no store handles an empty or single-string case. Outputs serialize viaRPCSerializer(blob and streaming outputs ignored), keys via the sharedencodeCacheKey.Server
deepSortKeysutil in@orpc/shared.Testing
@orpc/experimental-cacheand the new@orpc/cloudflarestores: unit, type-level, handler, and e2e tests, mocked-client Redis suites plus env-gated Redis integration tests, and workerd tests against real KV bindings.Docs
docs/helpers/cachepage (usage, adapters, SWR, handler plugin, cross-origin notes) with JSDoc backlinks, api-reference row, and package lists updated.