Skip to content

feat(kernel): construct and reconstruct logical model requests - #2793

Merged
jbeda merged 5 commits into
mainfrom
impl/2663-modelrequest-construction
Nov 17, 2026
Merged

jbeda merged 5 commits into
mainfrom
impl/2663-modelrequest-construction

Conversation

@jbeda

@jbeda jbeda commented Nov 15, 2026 •

Copy link
Copy Markdown
Contributor

Summary

Implements the isolated-kernel logical model-request construction and reconstruction slice of #2663. Merged on 2026-11-17 as 91e3155236feb5dd684f0f6bf51e805cad1d30a4.

Production execution entry points are unchanged. This PR does not execute models/tools, integrate provider SDKs, or close #2663 or its parent #2659. Finalized accounting and capture-parity reassessment remain under #2663; executable integration/recovery and production provider fidelity remain #2664/#2665/#2667.

What changed

Pinned sources and request commitment

  • session.CommitView captures immutable accepted plus proposed records in O(1), alongside existing accepted Snapshot inspection.
  • Capture-bound AddModelRequest checks the originating batch and unchanged proposed prefix. Same-batch creation, run start, preparation and request commitment are supported, including a pre-genesis source anchor.
  • An accepted model request is a commitment, not evidence that it was sent or authority to invoke. Existing accepted-request/tool prerequisite barriers remain.

Logical request construction

  • New engine/kernel/modelrequest and modelrequest/format packages own provider-neutral logical requests, generated codecs/borrowed views, construction recipes and canonical content CIDs.
  • Built-in chronological-v1 projects retained instruction layers, ordered tool definitions, typed model settings and conversation across runs. It preserves original provider call IDs and requested argument bytes, assistant-item ordering, requested-call result order, explicit outcomes and truthful tool/resource renderings. It performs no network/resource retrieval.
  • RootInstructions is the harness root contract; UserInstructions holds prepared persona/project guidance. Dynamically triggered guidance remains chronological EffectiveInput. Required InstructionBlock.Key values are unique per complete collection and provide producer bookkeeping only: slice order wins, and keys are absent from logical content/CID.
  • Private continuation is included only by explicit exact API/model/producer/format/version selectors. This is provisional replay policy, not a compatibility guarantee.
  • Consecutive text in a content sequence coalesces byte-for-byte with no implicit separators. Empty text is omitted; media, messages, instruction blocks and assistant items retain their boundaries.

Reconstruction and ownership

  • Reconstruct uses the request's original record prefix and run, including preparation earlier in its commit, rather than the current active run, context or configuration. It does not rebuild a historical lifecycle Snapshot or perform backend I/O.
  • The recipe check is the canonical logical-request content CID, not a source-record CID manifest. Unsupported recipes, malformed parameters/checks and CID mismatches fail explicitly without preventing ordinary session inspection.
  • Requests own projected DTO backing. Private finalization consumes that exclusively owned backing without an extra clone; public MutableClone and recipe export detach mutable data.
  • Existing codec limits bound accepted parameters/output, not peak construction memory: projection precedes final output validation/encoding. The earlier manual sizing/bookkeeping pass was removed during review.

Reading guide

  • engine/kernel/session/commit_view.go and request admission in session_commit.go / lifecycle_state.go: source capture, freshness and same-batch legality.
  • engine/kernel/modelrequest/construct.go, projection.go, tool_result_projection.go: selection and model-facing rendering.
  • engine/kernel/modelrequest/request.go, construction_result.go, reconstruct.go: owned output, recipe/CID and historical reconstruction.
  • engine/kernel/modelrequest/example_test.go: executable public flow; construction/reconstruction tests cover real Submit and storage-backed Load, later appends/runs, exact-byte rendering, failures and ownership. Session tests also demonstrate selective persona replacement by contribution key while retaining project guidance and old snapshots.

Verification and review

The initial delivery passed the complete offline suite and race suite, all-module lint, standalone engine tests, generation/API checks, docs and the offline demo (tool call/result plus permission ask/approval).

Subsequent review-driven changes—standing instruction naming/keys, simpler projection, adjacent-text coalescing and finalizer backing transfer—were checked with relevant kernel/id/architecture/history-adapter race tests, standalone engine tests (GOWORK=off), all-module lint, API checks and docs. The full root test/race suites and demo were not rerun at final HEAD; initial full-suite evidence is historical, not final-HEAD certification. No live-provider or paid calls were used.

Spec, Standards, Test adequacy and relevant specialist reviews found no remaining findings at delivered HEAD 6547d67910b50c1a564a451f6fc9e1b218f7f430.

PANEL: ship_blockers=0 important=0 advisory=0 reviewer_failures=0

API compatibility and limits

  • Wire API: no production remote-observable changes, protobuf/feature/major changes or client upgrades.
  • Experimental Go API/stored payloads: capture-bound AddModelRequest, optional pre-genesis source, typed settings, original provider call IDs and keyed root/user instruction collections change experimental contracts in place, as approved. Chronological-v1 is not yet locked. Earlier experimental payloads may fail to load/reconstruct; there are no migrations, dual readers, old-algorithm implementations or waived CID checks. Histories are not rewritten or deleted.
  • No production/provider integration, execution/recovery, compaction, public request-from-DTO factory, external algorithm registry, auxiliary-call lifecycle or full request persistence is added. A logical CID promises neither raw provider-wire equality nor identical provider handling of arbitrary block segmentation.

Design provenance

The archived reviewed plan preserves the approved design and earlier naming choices. Its verification steps are intentions, not executed evidence; the merged behavior and qualification above include later review changes. Current code/Godoc and tests are authoritative.

@jbeda

jbeda commented Nov 15, 2026 •

Copy link
Copy Markdown
Contributor Author

Archived implementation plan — merged in #2793

Reader's note (2026-11-17): this comment preserves the reviewed plan, not the final implementation report. #2793 merged as 91e3155236feb5dd684f0f6bf51e805cad1d30a4, from delivered head 6547d67910b50c1a564a451f6fc9e1b218f7f430. The PR description summarizes merged behavior, code entry points, qualification and remaining work. #2663 remains open for finalized accounting and capture-parity reassessment; production adoption is separate.

Later human review simplified projection by removing the manual sizing pass (codec limits bound accepted output, not peak projection memory), coalesced adjacent text byte-for-byte without separators while preserving semantic boundaries, and transferred exclusively owned DTO backing at finalization instead of cloning it again. These changes supersede conflicting implementation details in the archived plan. RootInstructions / UserInstructions, required contribution keys and detached MutableClone exports are the final names/contracts; original names farther below are planning history. The reconstruction check is the logical request CID, not a source-record manifest; experimental v1 remains changeable in place, with no historical-algorithm or migration promise.

The plan body below is unchanged. Verification sections describe intended checks, not a claim that all full-repository gates ran at final HEAD; see the PR description for that distinction. Local scratch paths and branch names below are historical provenance, not required reading or persistent resources.

  • Slug: 2663-modelrequest-construction
  • Source provenance: .scratch/2663-modelrequest-construction-plan.md (local working source, not a remote link)
  • Implementation branch: impl/2663-modelrequest-construction
  • Original base/start: refs/heads/main, c4ff786a7467eb2315de679ae78711d118775383
  • Explicit rebase base containing feat(dtogen): generate MutableClone for DTO-backed views #2795: 89a9f42d79071e4f83eb6220e5e92536716968fa
  • Follow-up fixed point: 154cebba6ccb31259f969ac94e54152401dfba5e
  • Historical publication head: e58d683bb9eb701f1df384c9edd6aa496c002771 (before the later projection, coalescing and ownership review changes).
  • Original archive update: 2026-11-16T16:44:43Z; merged-reader context added 2026-11-17 without changing the archived plan body.
  • Plan portion SHA-256: 9ac10528d478e7234bdb37ff387455c03acdabbea08ad0f3efaaf19823c4b886
  • Publication: complete plan; no redactions. The approved follow-up amendment supersedes historical naming/export choices below, records the required contribution-key contract and selective resume flow, and preserves initial planning provenance.
  • Verification below describes intentions, not claims of executed gates. The initial implementation's qualification does not certify subsequent local follow-ups; current execution evidence remains separate.

Kernel model-request construction

Slug: 2663-modelrequest-construction
Implementation branch: impl/2663-modelrequest-construction
Base branch: main
Date: 2026-11-11
Status: implementation authorized on 2026-11-14; implemented in the isolated kernel. Actual qualification and delivery results are recorded in the implementation PR, separately from the verification intentions below.

Authorization update (2026-11-14): Joe explicitly requested implementation of this reviewed plan and opening a PR afterward. The planning-only authority statements in the historical review provenance below describe the earlier planning session, not the current authorization. No design scope or deferrals changed.

Approved follow-up amendment (2026-11-16)

This amendment supersedes conflicting original names and DTO-export wording below. The initial implementation is complete; this is a review follow-up, not a restart. #2795 supplied generated MutableClone and was explicitly rebased into this checkout. Follow-up fixed point: 154cebb; post-rebase main base: 89a9f42. No production or wire API behavior changes: this remains isolated kernel code. Experimental Go/stored-data changes below are explicitly approved; no real retained instruction data exists, so no migrations, old aliases, or dual readers are required.

  1. Rename standing session and logical-request collections coherently to RootInstructions / UserInstructions, including views, replacements, record kinds, generated artifacts and tests. Root means the harness root operating contract; user means prepared persona/preferences (SOUL.md) and applicable project guidance (AGENTS.md/CLAUDE.md). The distinction is intended placement/authority, not producer, filename, or provider message role. Dynamically triggered guidance enters attributed chronological EffectiveInput rather than silently rewriting the standing baseline. Upstream owns admission/trust/framing. Load never rereads a ContentSource file; source descriptions are not identities and file labels are optional workspace-relative provenance.
  2. Move shared InstructionBlock declaration/validation to session/format/instruction_block.go and alias/view to session/instruction_block.go. Keep each container's code in its own file. Use generated MutableClone for detached DTO export, including logical Request, rather than duplicate Format export paths.
  3. Add InstructionBlock.Key string, required even for empty/non-file-backed contributions: 1..256 UTF-8 bytes, valid UTF-8, no Unicode whitespace or control runes. Exact byte equality, with no normalization, trimming, or case folding. Open vocabulary, illustrative labels only (harness/core, operator/persona, workspace/root-guidance); no exported constants, reserved prefixes or special semantics. Keys are producer-owned bookkeeping, stable by convention across source/content changes. Each complete collection rejects duplicate keys, including identical/empty blocks. Identical contents under different keys and the same key in root and user are valid. Slice order remains authoritative; no slot-level patch API, sorting, merging, lifetime registry, cross-layer identity continuity, or inferred refresh behavior. Producers choose granularity and may edit part of a contribution themselves. Keys confer no trust/authority/precedence and are omitted from logical content and CID; key-only changes still change retained session facts.
  4. Document these contracts beside owning declarations and in session/doc.go; no new architecture chapter. Add an executable storage-backed Load -> latest UserInstructions -> MutableClone -> select persona by Key -> retain project guidance/order -> complete replacement -> Submit flow. Prove detached ownership, unchanged old snapshots and historical request reconstruction, independent root/user scope and key-only CID invariance, with content-change positive controls. Cover direct validation, bounded codecs, malformed decoding and real aggregate/storage rejection. Preserve canonical bytes for the newly approved experimental schemas; do not silently waive CID verification.
  5. Dependency order: coherent rename/organization -> keys/fixtures/admission -> resume example and projection invariance -> regeneration, focused/race/standalone/API/docs/lint gates and four-axis review of final-tree delta. Gate intentions are not results. Record actual evidence in ignored run notes; preserve prior review evidence as historical. Joe subsequently authorized committing/pushing the completed follow-up, PR review prep, and a clear-ready HANDOFF update. Merge remains human-only. Update the existing archived plan comment in place to reflect this approved amendment without claiming unpushed code is remote.

Goal, authority, and scope

Implement the next isolated-kernel slice of #2663: construct an owned provider-neutral logical model request from pinned session records; retain its reconstruction recipe in a ModelRequest commitment; reconstruct that request from later accepted history and verify its content CID. Session remains independent of the constructor. This is NOT production adoption and does not close #2663; finalized accounting and capture-parity reassessment remain separate.

This plan consolidates Joe's completed grill through Q106 and supersedes conflicting earlier scratch notes and the current #2663 issue body. In particular: no source-record CID manifest; same-batch preparation is required; preserve original provider call IDs, not generated call_N references; typed minimal settings replace opaque settings JSON; no schema-version field/envelope in the logical request; no new traversal-limit system. The issue was read, not edited. Do not publish its old manifest/accepted-only/mandatory historical-version-support wording as this design.

Explicit non-goals: provider/SDK or production-engine integration, execution/recovery, hooks/guardrail execution, network/resource retrieval, compaction/generalized edits, external algorithm registration, auxiliary-call lifecycle/persistence, request persistence, public FromFormat factory, old-schema migrations or old-algorithm implementations. Auxiliary calls are a recognized future consumer of the source-independent request data model, not an extra framework in this slice.

Experimental policy: Joe explicitly permits revising chronological-v1 and session payload schemas in place until he declares them locked. Do not add compatibility/changelog bookkeeping or dual readers. Earlier experimental sessions may no longer load; generic History can still expose raw records. Never fabricate missing provider IDs, discard old settings to make them load, or rewrite/delete stored histories. Keep verification honest: changed rendering may make an old experimental Check mismatch; never silently waive it.

Grounded current seams

  • engine/kernel/session/session_commit.go: original-pointer-only, append-only SessionCommit owns immutable materialized facts. Successful additions append; failures preserve prior state. Submit freezes exact proposals for uncertain retry. check, add, Discard, and SubmitCommit already define lifetime/authority.
  • session/snapshot.go: O(1) pinned accepted Snapshot; Records/RecordAt and delivered RecordsBetween(after, through) provide immutable scoped views. Preserve that range API; do not add sentinels or historical lifecycle Snapshots.
  • session/record.go: materializedRecord owns canonical bytes and decoded DTOs; immutable backing is usable for both proposed and accepted inspection. Current RecordView/child Godoc sometimes incorrectly assumes every inspected fact must be accepted once proposals are exposed.
  • session/lifecycle_state.go: current request admission consults accepted run/binding/frontier and captures accepted coverage. Authoring, preflight, accepted application, and Load share admission. Same-batch request preparation needs an intentional request-specific change in all of those paths; never globally substitute candidate for accepted prerequisites.
  • session/format/model_request.go: SourceHead currently rejects zero; ConstructionRecipe contains Algorithm/Version/Parameters/Check; session treats recipe meaning as opaque.
  • session/format/model_response.go: RequestedCall currently retains Name/Arguments but omits provider call ID. Kernel RequestedCallID already correlates invocation/settlement independently of provider IDs.
  • session/format/model_binding.go, instructions.go, tool_disposition.go: existing replacement state, attributed prepared blocks, explicit outcome/capture distinctions, and private diagnostics are reusable foundations.
  • codec/{limits,budget,marshaler}.go: existing explicit codec.Limits and allocation-before-check protections. codec.Check checks without output allocation; generated Encode checks before allocating. Content identity is contentid.Sum, with exact fixed-profile 36-byte Binary/ParseBinary helpers.
  • Read internal/dtogen/README.md before declarations. Use generated codecs/clone/equal/views; no new serializer or generator framework.

Material contracts

1. General pinned commit inspection and request authoring

Add session.CommitView captured by (*SessionCommit).View() (CommitView, error). It pins the complete accepted prefix plus all currently proposed records, in order, without copying payloads or reconstructing lifecycle aggregates. Capture is O(1); RecordAt is O(1) over two bounded segments; iteration is repeatable and early-stoppable. Capture requires a valid writable original batch, but no request eligibility or active run. Existing captured views remain readable after later additions, submit, discard, and uncertain/final submission outcomes; capture grants no acceptance/execution authority. Zero views are safe and empty.

Shared inspection starts with HistoryID, ActiveRun, RecordCount, RecordAt, and Records. Snapshot keeps accepted semantics; CommitView.ActiveRun is the captured candidate scalar. Do not add speculative common methods or a session-owned polymorphic interface. modelrequest.Source is the small consumer-owned interface matching those methods; add lookups only when actual algorithm code demonstrates need. CommitView privately retains originating batch identity and proposed-prefix length. Correct scoped view docs to distinguish immutability from acceptance.

Replace, do not overload/retain a bypass for:

func (b *SessionCommit) AddModelRequest(run RunID, source CommitView, v ModelRequest) (ModelRequestID, error)

Validate live/writable destination, active run/scope, exact originating batch and unchanged prefix, then zero synthesized fields and payload. Incoming RunID/ID/Source must be zero. Supply Source from the capture's accepted anchor and destination identity from the batch. Original batch identity plus prefix length proves freshness in O(1): no record comparison, record CID, public generation token, interface provenance plumbing, or capture registry. Any successful intervening addition invalidates attachment; failed additions do not. Failure consumes no identity and preserves the batch. Submit/freeze/discard prevents later additions; an uncertain append must use existing exact-proposal retry.

Caller supplies session.ModelRequest{Recipe: built.Recipe()} and optional RetryOf. Session validates recipe structure, not whether it came from this source or hashes the request the caller intends to invoke. The caller pairs source/result/recipe correctly. Do not create a new constructor dependency or pretend the freshness check proves recipe correctness. session.ModelRequest is an invocation commitment, NOT 'sent' and NOT the constructed logical request.

Source semantics: accepted anchor A plus every record preceding the request in its own commit; exclude request/later records. Represent pre-genesis by an absent anchor: change ModelRequest.Source to *SourceHead with optional-field encoding, where nil means pre-genesis only in the initial commit. Keep a present SourceHead strictly positive with a nonzero CID; partial-zero pairs and present-zero anchors remain invalid. The logical anchor is zero before genesis, but never encode a zero contentid.CID link: its independently generated child validation correctly rejects unset CIDs. Do not weaken shared CID validation. AddModelRequest requires incoming Source nil and supplies either nil or the captured positive anchor; adapt Source inspection to preserve that presence distinction. Aggregate admission verifies exact anchor and initial-commit restriction. Evaluate next-request run/binding/input eligibility and captured contribution frontier against the valid preceding candidate: first-commit Created/StartRun/preparation/request is legal; preceding outcomes for accepted model requests (without outstanding calls) or valid settlements of accepted tools can enable the next request. Preserve accepted-request barriers for outcomes and existing accepted tool-invocation/result/prerequisite barriers. RetryOf remains earlier-accepted/same-run. Update admission, replay, reserve simulation, and tests together.

Run start is explicit in the batch before capture, alongside EffectiveInput; construction never starts a run. A source without an active run, or with unresolved model/tool work, is not constructible. Do not infer that mutable Session state is the source of a captured view.

2. Retained session DTO changes

  • sessionformat.UserContextPrefix{Blocks []InstructionBlock}, plus session alias/view, RecordView branch, ReplaceUserContextPrefix and latest snapshot inspection following Instructions conventions. Latest session-scoped replacement wins, explicit empty clears, no implicit run reset; root Instructions remains independently replaceable. Both contain prepared retained content. Reuse InstructionBlock with clarified container-specific semantics/docs, rather than claiming a prefix block is root/system authority. Preserve provenance scope checks and presence/emptiness.
  • sessionformat.ModelSettings{ReasoningEffort *string; MaxOutputTokens *int64}, plus session alias. Use it in ModelBinding and the logical request. Effort is a nonempty exact target-native label when present, not a universal closed enum. MaxOutputTokens is positive when present. Nil means intentional upstream omission, never inheritance/current defaults. Preparation resolves preferences; reconstruction does not clamp/normalize. Defer all additional provider-specific fields and grouping until adapter integration.
  • Add exact UTF-8 ProviderCallID string to retained RequestedCall. Retain missing/duplicate observations in history, as with malformed call names/arguments; construction rejects empty or response-local duplicate IDs. Distinct responses may reuse an ID. Preserve existing kernel occurrence IDs for membership, prerequisites and settlement; no new occurrence identity type.
  • Extend NotInvoked reasons for user denial, tool unavailable, and known infrastructure failure, alongside policy rejection/preparation failure/cancelled/other. Exact labels use existing categorical conventions. Infrastructure belongs here only with known non-invocation; possible execution remains Indeterminate. Diagnostics remain private. Session schema changes are experimental and may be in place; history envelope unchanged.

3. New packages and concrete DTOs

New engine/kernel/modelrequest owns construction, reconstruction, immutable request/child views, CID calculation and errors. New modelrequest/format owns mutable DTOs, payload-local validation and generated codecs/whole-value operations. Dependency points toward session inspection/format, history identity, codec/contentid/views only as needed. Session never imports modelrequest. No SDKs, legacy session aggregate, host packages, model registry, current configuration, filesystem, or network dependencies.

// modelrequest/format
// Tags/field documentation follow the generator and neighboring kernel conventions.
type RequestV1 struct {
    API string
    Model string
    Settings sessionformat.ModelSettings
    RootInstructions []ContentBlock
    UserContextPrefix []ContentBlock
    Tools []sessionformat.ToolDefinition
    Conversation []ConversationItem
}
type ContentBlock struct { Parts []sessionformat.ContentPart }
type ConversationItem struct { User *UserMessage; Assistant *AssistantMessage; ToolResult *ToolResult }
type UserMessage struct { Parts []sessionformat.ContentPart }
type AssistantMessage struct { Items []AssistantItem; Private *sessionformat.OpaqueProviderData }
type AssistantItem struct { Output *ContentBlock; Refusal *ContentBlock; Call *ToolCall }
type ToolCall struct { ProviderCallID string; Name string; Arguments []byte }
type ToolResult struct { ProviderCallID string; Outcome ToolOutcome; Parts []sessionformat.ContentPart }
type ToolOutcome string // returned-success | returned-error | not-invoked | indeterminate

type ParametersV1 struct { PrivateReplay []PrivateReplaySelector }
type PrivateReplaySelector struct { SourceAPI string; SourceModel string; Producer string; Format string; Version int64 }

Exactly one pointer branch in each union. Reuse identity-free ContentPart/Media, ToolDefinition, OpaqueProviderData and ModelSettings DTOs; do not duplicate media schemas, move a shared primitive framework, or embed history-scoped ModelBinding/ModelResponse/ToolDisposition. ContentBlock preserves root/prefix and assistant-item boundaries, not new semantics or provenance. Tool definitions remain ordered with unique names. Calls preserve original name/argument bytes, including empty names or malformed/non-JSON arguments; never substitute effective invocation values. Correlate results through kernel call membership internally, emit original provider IDs, and check correlation scoped to each assistant response rather than session-global ID uniqueness. One logical result per call, requested-call order; no orphan/duplicate result occurrences. Assistant items retain output/refusal/call interleaving; refusal is provider-typed content, not inferred from prose or a terminal status.

RequestV1 has NO version member, family envelope, route, credentials, source/history/run/request IDs, recipe, diagnostics, usage, evidence or embedded CID. Type/algorithm context supplies schema interpretation. Canonical bytes alone determine CID; identical bytes across future Go types share CID. General version-dispatch decoding and persistence envelopes (possibly the history Record envelope) are deferred. Nil/empty collections have the same meaning and canonical bytes; preserve meaningful optional present-empty leaves/envelopes.

modelrequest.Request is an immutable handle over owned RequestV1 DTO backing with stored CID. Use generated borrowed nested accessors, views.Bytes and views.SliceView (no materialized []View). Safe zero views, explicit detached Format/export, no scalar getter cloning siblings, no retained mutable caller slices. ConstructionResult holds Request and a private owned session.ConstructionRecipe; Request() returns the handle, Recipe() returns a detached DTO. It does not construct a session.ModelRequest or choose RetryOf for the caller.

Public operations:

func Construct(source Source, parameters format.ParametersV1, limits codec.Limits) (ConstructionResult, error)
func Reconstruct(snapshot session.Snapshot, requestID session.ModelRequestID, limits codec.Limits) (Request, error)

Construct derives the active run from Source; no separate RunID argument. Source is finite, immutable and internally consistent by contract, not hostile executable code. Zero/unscoped/unsuitable sources fail; errors return zero results, never partial usable requests/recipes. Do not expose public FromFormat yet. Keep internal finalization reusable without adding an auxiliary-call framework.

4. Chronological-v1 semantics and identity

Construct selects the built-in chronological, version 1. No public resolver/registry, configurable callbacks or caller-supplied algorithm/version combination. Reconstruct explicitly dispatches retained (Algorithm, Version), decodes that version's parameters and rejects unsupported pairs. The mechanism exists now; third-party extensibility is deferred. Experimental v1 may change in place; no historical implementation obligation before lock.

Check = contentid.Sum(codec.Encode(RequestV1, limits)).Binary(): exactly the existing 36-byte binary CID, no text, bare digest or wrapper. Parameters encode canonically into ConstructionRecipe.Parameters; ParametersV1 belongs specifically to chronological-v1 and is not in RequestV1. Empty raw parameter bytes are not an ambient-default shorthand; strict supported-shape decoding rejects unknown/missing/malformed data. Nil/empty PrivateReplay means omit all envelopes and has one canonical encoding. Recipe name/version/parameters and Route are not mixed into request CID. Recipes differing in irrelevant parameters can produce the same request CID.

PrivateReplay selects whole envelopes by exact originating request API/model and envelope Producer/Format/Version. Empty list omits all; no wildcard interpretation, current registry lookup, target-equality inference or internal opaque-data edits. Selector order cannot reorder conversation; duplicate/equivalent matches do not duplicate envelopes. Caller-provided allow-list is provisional best-effort authorization, not proof of compatibility. Actual adapter policy discovery and replay constraints need usage experience; do not invent them here.

Reconstruct locates the accepted request and its original boundary in the supplied pinned Snapshot, including same-commit preceding preparation. Use accepted range/record inspection and a small internal prefix reader, NOT a reconstructed historical lifecycle Snapshot or new permanent history index. Derive original active-run context from the request and validated prefix, never today's ActiveRun/latest context. Recompute from retained recipe and fail on malformed Check or mismatch. Unknown algorithm/mismatch must not prevent ordinary session inspection of otherwise supported data.

5. Rendering policy

  • Order: latest root blocks, latest user-context prefix blocks, advertised tools, chronological effective conversation. EffectiveInput is incremental across runs; UserInput alone is not model input. Repeated selections remain repeated. Run endings do not withdraw unconsumed effective contributions. No compaction/reset inference.
  • Each nonempty EffectiveInput item becomes its own user entry. Selected submissions retain prepared parts. Inline PromptInstructions uses opening <supplemental-instructions>\n and closing \n</supplemental-instructions> text parts around retained parts; PromptContext analogously uses supplemental-context. XML-like markers only: no XML parser/escaping framework or authority claim. Actor/ContentSource are not automatically rendered. Remove empty parts before framing so empty items do not become marker-only entries. Root/prefix prepared framing is not wrapped again. Proper upstream treatment of untrusted data remains required; markers are not a security boundary.
  • Drop exact-empty text parts, retain whitespace. Drop otherwise empty user/root/prefix/assistant content occurrences. Keep calls and selected private-only assistant messages; readable reasoning is not assistant prose. Preserve content/refusal/call order. Empty collections are not dummy messages.
  • ToolResult.Outcome is explicit; status notices precede capture notices, then retained output parts. Returned success has no routine status boilerplate. Returned error: Tool returned an error. Non-invoked broad notices: User denied tool execution; Tool execution was denied by policy; Requested tool was unavailable; Tool invocation could not be prepared; Tool was not invoked because of an infrastructure failure; Tool invocation was cancelled before execution; fallback Tool was not invoked. These expose no diagnostic/policy-rule details. Indeterminate: Tool outcome is unknown; side effects may have occurred. No retry advice or assertion of absent effects.
  • Capture notices: complete with content -> none; complete with no rendered content -> Tool returned no output.; omitted -> Tool output was not retained.; truncated (even zero parts/non-prefix subset) -> Tool output is incomplete. NotInvoked has no invented output capture. Explicit dispositions win over derived exceptional closeout. Active unsettled work cannot yield an invocation-ready request; bulk closeout uses existing truthful derived NotInvoked/Indeterminate semantics.
  • Result output is eligible; diagnostics/evidence are not. Ordinary typed text/media remain typed. Tool JSON becomes text containing exact retained UTF-8 JSON bytes, no parse/re-encode. Resource links render fixed metadata labels; embedded text adds metadata then exact body as separate parts; generic binary adds metadata plus Binary body not included in this request. and no binary body. Include exact URI and supplied name/title/MIME with quoted/escaped metadata values; do not infer kind from MIME, promise retrieval, fetch/upload, or suggest re-execution. Quoting metadata is not reencoding retained JSON/body. Exactness is relative to retained effective facts, not original external transport.
  • Current-production parity was traced: generic MCP blobs are not auto-promoted/fetched, including image/png resources; typed media follows a different path. Explicit ReadMcpResource/FetchMcpResource/CallMcpWithQuery and ACP/TUI attachment preparation may fetch/decode/transform before retention. Do not reproduce those in construction. Source seams: engine/session/content.go:352, internal/adapter/mcp/tool.go:302,349, internal/adapter/acp/agent.go:869, provider request builders. Our metadata/omission wording is deliberately clearer, not byte-for-byte legacy rendering.
  • No automatic messages for run start/end, request commitments, infrastructure invocation failures, RetryOf or invocation evidence. Refusal is actual typed model content, not infrastructure failure. Additional explanations require retained EffectiveInput. Tool closeout notices are the explicit exception to lifecycle omission.
  • Logical content is NOT SDK wire bytes. Later driver conversion must preserve correlated tool-result shape and pre-rendered notices, use native error signaling where supported, and reject unsupported representation rather than silently drop/substitute content. This plan implements no drivers and promises no universal deliverability of every media/settings combination.

6. Resource/ownership bounds

Use existing codec.Limits directly for parameter processing and new request output; no new Limits struct, MaxVisitedItems, mutable defaults or traversal framework. Session policy already bounds accepted records/payload/decoded work; provisional builders and custom Source implementations remain trusted finite in-process inputs, not sandboxes. Avoid claiming that every open proposal has passed cumulative submission policy.

Protect amplification: many tiny selections may repeat one large input. Preflight selected output sizes/counts and fixed framing/metadata overhead using overflow-safe arithmetic BEFORE governed slice growth, formatting or payload copies. Reuse codec budgeting/generated checks where applicable; a small sizing pass is preferable to unchecked build-then-Encode. Source-bounded association maps are not output-sized payload clones. Final generated Encode is an additional complete check, not the first bound. Too-small policy returns limit error and zero outputs; increasing policy cannot change successful bytes. Do not apply an output-size cap to the entire unrelated retained history or add semantic truncation. Report errors without dumping source content/private data.

Dependency-ordered implementation tasks

  1. Session DTO foundations. Add typed ModelSettings, ProviderCallID, NotInvoked categories and UserContextPrefix DTO/aliases/views; wire materialization, sizing, support dispatch, latest state/snapshot getters and replacement authoring. Follow owning-type files and Godoc. Regenerate; update affected experimental fixtures, no migrations. Completion: real Submit/Load preserves settings, original IDs, prefix empty replacement and scope; kernel IDs/settlement behavior unchanged.
  2. General CommitView. Add captured two-segment view, ActiveRun/shared record inspection, private origin/prefix tracking; correct acceptance-only view comments. No constructor dependency. Completion: O(1) capture/indexing; complete proposed/accepted scoped traversal, immutable old answers after append/submit/discard, no payload clone on scalar reads; zero/closed/foreign/batch-copy behavior tested. Benchmark capture separately from traversal with nonempty sibling payloads.
  3. Capture-bound authoring and prospective request admission. Change AddModelRequest signature and update all callers/tests. Encode pre-genesis as nil ModelRequest.Source, not a zero CID link; add generated codec round-trips rejecting present-zero/partial-zero anchors and aggregate rejection of nil outside genesis. Use candidate preparation/frontier only for next-request admission; preserve other accepted barriers and retry behavior. Tasks 2/3 must integrate as one coherent admission change, not an optional direct-add bypass. Completion: first commit preparation/request succeeds through codec materialization, Submit and Load; stale/foreign/discarded capture and any intervening successful add fail atomically; failed additions preserve eligibility; preceding legal settlement/outcome cases succeed, forbidden same-batch execution prerequisites still fail.
  4. Logical-request contracts first. Add modelrequest/format DTOs/validation, generated codecs/clone/equal, immutable Request/child views and ConstructionResult; ParametersV1 with provisional notes; internal bounded finalization/CID. Add precise new core/layering/depguard ownership rows and API snapshot coverage before consumers. Completion: canonical bytes, unions, nil/empty/present-empty, original opaque call args, settings, provider-ID scope, and detached/leaf-only copying tests pass. No public factory or registry.
  5. Chronological construction. Implement Source and Construct, bounded selection/projection, prefix replacement, inline framing, assistant ordering/private policy, tool outcomes/capture/resource rendering, recipe encoding/check. Use existing public scoped views; do not decode/replay a second session aggregate. Completion: representative real SessionCommit.View fixture yields exact logical DTO and recipe, unchanged after source closure; output amplification fails before large allocations. Test malformed provider IDs independently of preserved malformed call names/arguments.
  6. Historical reconstruction. Implement retained request lookup, exact original prefix, supported recipe dispatch/parameter decode, Construct-core reuse, CID verification. Completion: reconstruct requests at beginning/middle of commits from later snapshots and storage-backed Load, after new runs/context/binding appends, without current config/hooks/SDK/network calls. Include original-run-not-current-active regression and zero-anchor genesis.
  7. Integrated qualification and docs. Add one complete public flow example with errors: Create/StartRun/preparation -> View -> Construct -> session.ModelRequest{Recipe: built.Recipe()} -> AddModelRequest(run, view, record) -> Submit -> Reconstruct. Explain accepted vs provisional identities and that invocation is out of scope/acceptance is required. Document package overview in doc.go and stable kernel scope in existing README/architecture only where existing claims change. No user-facing production docs claim. Regenerate/API refresh, run focused/standalone gates, review the integrated implementation. Reconstruct accepted session state from committed history #2663 remains open.

Acceptance and verification

Use isolated memory history/reference adapters and controlled fixtures; no operator state or live providers. Test the actual authoring/admission/Submit/Load path, not only fake sources. Observable checks include:

  • Source capture navigation and scalar getters borrow; selected leaf copies detach only that leaf. Old views and completed requests remain unchanged after caller mutation, append, submit, discard; construction outputs do not retain mutable source DTOs. Capture does not replay full history or rebuild indexes; measure captured-adapter setup separately from full traversal.
  • Stale capture after even a context-only/empty successful addition rejects without consuming request identity; discarded/replacement batches cannot reuse capture even when provisional IDs/record counts match. Failed additions do not invalidate it. Frozen uncertain retry remains exact and never authorizes a new request/invocation.
  • Same-batch initial creation/start/context/UserInput/EffectiveInput/request and preceding valid tool settlement or accepted model outcome; precise negative controls retain model/tool/prerequisite acceptance barriers. Later records in same commit cannot affect earlier request; later run/binding/prefix appends cannot affect reconstruction.
  • Full conversation across runs, repeated selections/unconsumed contributions, independent root/prefix replacement/clear, inline endpoints, no automatic actor/lifecycle messages, interleaved output/refusal/calls, requested rather than settlement result order, whitespace and entirely empty/private-only cases.
  • Every tool outcome/reason and complete-empty/omitted/truncated capture combination, including bulk-derived settlement with explicit dispositions winning; no diagnostic/evidence disclosure or unsafe retry advice. Typed media versus generic binary, exact JSON/body, supplied metadata, no URI fetch or MIME promotion.
  • Original provider IDs survive Submit/Load and result correlation; same-response duplicates/missing reject construction while distinct responses may reuse IDs. Non-JSON args/empty names remain exact. Private exact match/empty list/mismatch/whole-byte preservation; source binding at originating request, not latest binding.
  • CID independent of route/history identities/unused parameters; sensitive to changed rendered bytes/settings/order/private selection. Different recipes producing identical output share CID. Strict malformed/unknown parameters, unsupported algorithm/version, malformed binary Check and changed-v1 wording mismatch return errors with no partial output. Session remains inspectable for supported payloads regardless of recipe support.
  • Low limits/overflow/counted amplification reject before governed output allocation; high versus adequate limits yield identical successful requests. Trusted Source contract is not a hostile-code sandbox.

During iteration, run focused Go tests for changed kernel packages. After integration run cd engine && go test -race ./id ./kernel/... ./arch ./adapter/history/..., task generate:data:check, task api:check, and task test:engine-standalone (give standalone full suite an outer 1800-second timeout). Use root task generate:data and task api:update for generated artifacts; never edit them by hand. Register new packages in engine/arch/surface.go, coreImportRules and precise .golangci.yml allowlists; no unrelated exceptions.

This is isolated engine work: engine/AGENTS.md permits focused race/architecture/API/standalone checks rather than a full repository suite when no host behavior is affected. If implementation touches root generator logic, host wiring or another module, broaden checks per root guidance: task test once per integrated set (1800s); before PR-ready handoff task lint (600s), task test:race (1800s) and offline go run ./cmd/mecademo. Markdown documentation edits require task docs (600s). Record the actual scope/gates used; do not claim tests ran during planning.

Initial implementation panel-review baseline is the implementation branch's recorded start commit, captured when plan-implement creates it—not moving main and not an invented planning OID. Address findings with focused incremental review; no implicit rebase, commit, push or PR creation. When a PR is later authorized, retain a complete publication-safe copy of the reviewed plan in one identifiable PR conversation comment; record URL/ID in ignored run metadata and verify readback. Do not post/update issues or comments as part of this planning request.

Remaining risk and review checkpoint

No material behavioral decisions are intentionally open. Ordinary implementation choices (CBOR tags, internal helper names, concrete error identifiers, efficient projection-specific lookup) follow existing conventions; do not turn them into frameworks. Private-replay policy and future adapter/settings integration are explicitly provisional/deferred, not claims of verified compatibility. Same-version experimental changes can break prior CID reconstruction by design. If implementing or reviewing reveals a new material contradiction, stop and ask rather than weaken source/ownership/acceptance guarantees or expand scope.

Independent review outcome

2026-11-11: fresh-context read-only plan critique completed against the full inline plan and current source. One important correction: a zero CID cannot be serialized as a SourceHead child. Incorporated an optional ModelRequest.Source (nil only for pre-genesis), preserving strict present-CID validation and adding codec/materialization/Submit/Load regression coverage. The same reviewer rechecked the corrected contract against dtogen and confirmed the finding resolved; no remaining blocker or important gaps identified. Review agent: subagent-1550ab1a64c8647853c6b52488f4eb3d-call_21jQWj3yNvj2jAPSmmA8y2h5.

Limitations: static design/source review only; no implementation, generated artifacts, or runtime qualification yet. No provider compatibility claim was independently verified; adapter integration remains deferred. No implementation tests were run during planning. Planning changed only this ignored scratch plan; no branch, tracked source, HANDOFF, issue or PR was changed. Implementation requires separate approval.

jbeda and others added 2 commits November 15, 2026 16:53
Add pinned commit inspection and capture-bound request commitments with
prospective same-batch preparation while retaining accepted execution barriers.
Introduce typed settings, user-context prefixes and retained provider call IDs.

Build owned chronological-v1 requests with bounded projection, explicit tool
outcomes and exact private replay selection. Reconstruct accepted requests from
their original prefixes and verify canonical request CIDs. Keep production
entry points unchanged and add complete persistence/ownership examples.

Refs #2663

Co-authored-by: Mecatl <noreply@mecatl.dev>
Align root and user instruction contracts, validate producer-owned keys, and demonstrate selective resume refresh with detached DTO clones. Preserve historical requests and keep contribution metadata out of logical request identity.

Co-authored-by: Mecatl <noreply@mecatl.dev>
@jbeda
jbeda force-pushed the impl/2663-modelrequest-construction branch from db4e5d1 to e58d683 Compare November 16, 2026 16:39
jbeda and others added 3 commits November 16, 2026 17:56
Build the request DTO directly and let final codec encoding enforce output limits. Remove the sizing pass and manual schema accounting; retain semantic checks and document the construction-memory tradeoff.

Co-authored-by: Mecatl <noreply@mecatl.dev>
Join adjacent text byte-for-byte within each content sequence while preserving media and enclosing content boundaries. Cover segmentation-independent CIDs and storage-backed reconstruction; retain final codec validation.

Co-authored-by: Mecatl <noreply@mecatl.dev>
Remove the redundant final request clone. Document success-only ownership transfer and retain public MutableClone and recipe isolation tests alongside canonical identity and limit checks.

Co-authored-by: Mecatl <noreply@mecatl.dev>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Reconstruct accepted session state from committed history

1 participant