protocol: silkd wire v2 — multiplexed streams + binary bulk frames over one long-lived connection

[CMGS]

Summary

Design for silkd wire protocol v2: multiplexed streams and binary bulk frames over one long-lived connection per sandbox, replacing one-connection-per-RPC + base64-in-JSON for payloads. This is the one remaining order-of-magnitude lever in this repo (per the #30 ledger) and a cross-cutting change to silkd (Rust), the Go SDK, the Python SDK, and the fixture corpus — worth a full design before any code. The relay is byte-blind and stays untouched.

Current state (surveyed)

• One connection per RPC. Every verb pays a triple setup: SDK TCP dial + HTTP upgrade (GET /v1/sandboxes/{id}/agent) → relay hijack (server/relay.go:39) → fresh vsock dial to silkd (relay.go:62, DialSilkd). At exec-RTT p50 0.27ms (n=1000, bare metal, performance governor) the fixed setup cost is the bulk of the round trip.
• A pre-dialed spare connection was measured and rejected (perf: remaining performance ledger after the 2026-07 hotspot round #30, H-4): p50 across 5 rounds 0.20/0.48/0.36/0.35/0.39ms vs a stable 0.27–0.31ms baseline, p90/p99 consistently worse — the background dial contends with the foreground RPC. Do not re-litigate; multiplexing is the fix.
• Bulk data rides base64 in JSON data fields (sdk/go/silkd/frame.go:3, conn.go:70; mirrored in silkd/src/proto.rs and the Python SDK). Cost: +33% on the wire plus codec CPU. The perf: sandbox-side hotspot round — fork snapshot clones, codec fast path, async journal #28 fast path (fastBulk, frame.go:100-108) already bypasses full JSON decode for data frames, and fs_pull measures ~602 MiB/s / fs_push ~343 MiB/s steady — the remaining ceiling is the encoding itself, plus a per-chunk decode buffer that generates ~600 MB/s of GC garbage at max pull (perf: remaining performance ledger after the 2026-07 hotspot round #30 deferred item; needs a payload-ownership contract on Recv, which v2 framing provides naturally).
• Server-side state is already connection-independent. Sessions and processes are addressed by id; a dropped connection loses nothing (attach/logs resume). The connection-bound verbs are the stream-shaped ones: fs_watch, pty, port_forward, lsp_request — each burns a full connection today (silkd/src/server.rs spawns a feeder per connection).
• 8 MiB frame cap; requests carry "v": 1; unknown fields ignored. The corpus in protocol/fixtures/v1 is round-tripped by Rust, Go, and Python in CI.

Goals / non-goals

Goals: eliminate per-RPC connection setup; raw binary payloads (target fs_pull ≥ 800 MiB/s); N concurrent RPCs and streams per sandbox over one connection with per-stream flow control; strict backward compatibility (v1 stays; old SDK ↔ new silkd and new SDK ↔ old silkd both work); explicit reconnect semantics.

Non-goals: verb semantic changes (the v1 verb set carries over 1:1); TLS/transport security (the relay path is inside the deployment trust boundary — docs/security.md); compression (orthogonal, measure separately); changing the relay (it stays a byte pipe).

Proposal

1. Framing

Length-prefixed binary frames on the existing relay/vsock byte stream:

[u32 len][u32 stream_id][u8 opcode][payload]

Opcodes: OPEN (payload = the v1 JSON request object, unchanged), CTRL (payload = a v1 JSON response frame minus its data field), DATA (raw bytes — replaces every base64 data field), DATA_END (half-close, replaces stdin_close/data_end), RESET (abort stream), CREDIT (flow control), PING/PONG. Control payloads stay the v1 JSON objects, so the existing fixture corpus keeps pinning the semantic layer; a new protocol/fixtures/v2 corpus pins the binary framing (golden hex frames + a scripted mux exchange).

2. Multiplexing

Client-initiated streams only (silkd never initiates), monotonically increasing u32 ids. One OPEN per RPC; responses and data arrive as CTRL/DATA frames tagged with the stream id; terminal CTRL (exit/done/error) or RESET ends the stream. The stream-shaped verbs (fs_watch, pty, port_forward, lsp_request) map to long-lived streams — port_forward alone stops costing one TCP+vsock connection per forwarded socket. Stream cap (e.g. 256 per connection) bounds a misbehaving peer, alongside the existing 8 MiB frame cap.

3. Flow control (the hard part)

v1 gets per-RPC backpressure for free: each RPC owns a TCP connection, so a slow client paces the guest child via TCP. On a mux, one slow consumer must not head-of-line-block the connection: per-stream byte credit windows (initial e.g. 1 MiB, replenished by CREDIT), sender never exceeds granted credit. This preserves v1's documented exec-output pacing semantics per stream while keeping a stalled pty from freezing a concurrent fs_pull. This piece is non-negotiable — a mux without flow control is a regression dressed as a feature.

4. Negotiation and compatibility

First byte disambiguates: v1 requests start with {; a v2 client opens with magic SLK2 before the first frame. Old silkd sees the magic, fails the JSON parse, answers a v1 bad_request and closes — the SDK falls back to v1 and remembers per handle (one wasted round trip per sandbox, first contact only). New silkd serves both indefinitely; v1 costs nothing to keep. Old SDKs keep speaking v1 forever. The info verb's proto field reports the ceiling for debugging.

5. Reconnect semantics

A dropped connection now fails every in-flight stream at once. Rules, explicit and tested: all streams fail crisply (no partial replay); server-side state is untouched (sessions, procs — same as v1); the SDK reconnects lazily on next use; automatic retry only for idempotent verbs (fs_read/stat/list/find/ps/logs/info; fs_write is temp+rename atomic, so retry-safe); exec is never auto-retried — the error surfaces and the caller decides. No server-side request dedup in v2 (rejected below).

6. Rollout

silkd lands first (v1 default untouched, v2 behind the sniff), images rebake; Go SDK next (attempt-v2-with-fallback), Python after; benches and e2e on the testbed gate each step; flip nothing — v2 engages wherever both ends speak it. Multiple PRs, one design; mux and binary frames ship together because the negotiation, reconnect, and fixture work is shared (#30: "do both together or not at all").

Explicitly rejected

• gRPC / HTTP/2 (h2c). The Python SDK is stdlib-only (no http2 in stdlib) and dependency-free SDKs are a product property; the relay is byte-blind so h2c through it means hand-implementing h2 framing in silkd anyway — all of the complexity, none of the ecosystem.
• WebSocket framing. Provides message boundaries but neither multiplexing nor flow control; we'd still build both on top.
• QUIC. Dependencies everywhere, and vsock has no datagram path here.
• Pre-dialed connection pool. Rejected with data (perf: remaining performance ledger after the 2026-07 hotspot round #30 H-4).
• Server-side idempotency keys / exactly-once retry. Session/proc state already survives reconnects; exec-retry policy belongs to callers; dedup tables in the guest daemon are complexity without a driving failure case.
• Compression. Bulk payloads are often incompressible; CPU sits on the data path. Revisit with measurements if WAN relays appear.

Hot-path cost

Negative, by design: removes TCP dial + HTTP upgrade + vsock dial from every RPC after the first. Targets on the bench node (.79-class, bare metal): exec RTT p50 ≤ 0.15ms (from 0.27), fs_pull ≥ 800 MiB/s (from ~602), fs_push not regressed (its ceiling may be guest tar extract — measure before promising). Memory bounded by stream cap × credit window per connection. Claim path untouched.

Open questions

• Connection pool >1 per sandbox for bulk parallelism (one TCP conn ≈ one relay goroutine pair ≈ one core of memcpy)? Measure single-conn ceiling first; the design leaves room (per-handle conn set) without committing.
• Initial credit window and stream cap values — pick from bench data, not taste.
• Keep the 8 MiB frame cap for DATA frames or raise it? Larger frames = fewer syscalls, coarser fairness granularity.
• Python sync SDK: mux needs a demux reader — background thread guarded by a lock (proxy_port already spawns threads), or a v2-lite mode (binary frames, one stream per connection)? Leaning full mux for parity; the corpus pins both ends regardless.

Acceptance evidence

• rpcbench n=1000, performance governor, bare metal: v2 p50 ≤ 0.15ms and p99 ≤ v1 p99; A/B against v1 in the same run.
• pullbench 128 MiB ≥ 800 MiB/s; pushbench within noise of v1; GC garbage on sustained pull measurably reduced (the perf: remaining performance ledger after the 2026-07 hotspot round #30 deferred buffer item closes).
• Full e2e verb smoke on both lanes over v2; fallback exercised against a v1-only silkd image (old image A/B) — both directions of the compat matrix.
• Fault injection: kill the relay connection with ≥10 mixed in-flight streams (exec + pull + pty + watch) — every stream fails crisply, sessions/procs intact, watch re-arms, idempotent verbs auto-retry, exec surfaces the error.
• Fairness soak: concurrent slow-consumer pty + full-rate fs_pull on one connection — pull throughput unaffected, memory bounded.
• Fixture corpora (v1 JSON + v2 framing) round-tripped by Rust, Go, and Python in CI.

Estimated size: silkd ~600–900 lines (framing, mux, credit, sniff), Go SDK ~500–700, Python SDK ~400–600, fixtures/benches/e2e ~300. Tracks item 1 of the #30 ledger.