Repository navigation
feat(kernel): construct and reconstruct logical model requests - #2793
Conversation
Archived implementation plan — merged in #2793Reader's note (2026-11-17): this comment preserves the reviewed plan, not the final implementation report. #2793 merged as Later human review simplified projection by removing the manual sizing pass (codec limits bound accepted output, not peak projection memory), coalesced adjacent text byte-for-byte without separators while preserving semantic boundaries, and transferred exclusively owned DTO backing at finalization instead of cloning it again. These changes supersede conflicting implementation details in the archived plan. The plan body below is unchanged. Verification sections describe intended checks, not a claim that all full-repository gates ran at final HEAD; see the PR description for that distinction. Local scratch paths and branch names below are historical provenance, not required reading or persistent resources.
Kernel model-request constructionSlug: 2663-modelrequest-construction Authorization update (2026-11-14): Joe explicitly requested implementation of this reviewed plan and opening a PR afterward. The planning-only authority statements in the historical review provenance below describe the earlier planning session, not the current authorization. No design scope or deferrals changed. Approved follow-up amendment (2026-11-16)This amendment supersedes conflicting original names and DTO-export wording below. The initial implementation is complete; this is a review follow-up, not a restart. #2795 supplied generated MutableClone and was explicitly rebased into this checkout. Follow-up fixed point: 154cebb; post-rebase main base: 89a9f42. No production or wire API behavior changes: this remains isolated kernel code. Experimental Go/stored-data changes below are explicitly approved; no real retained instruction data exists, so no migrations, old aliases, or dual readers are required.
Goal, authority, and scopeImplement the next isolated-kernel slice of #2663: construct an owned provider-neutral logical model request from pinned session records; retain its reconstruction recipe in a ModelRequest commitment; reconstruct that request from later accepted history and verify its content CID. Session remains independent of the constructor. This is NOT production adoption and does not close #2663; finalized accounting and capture-parity reassessment remain separate. This plan consolidates Joe's completed grill through Q106 and supersedes conflicting earlier scratch notes and the current #2663 issue body. In particular: no source-record CID manifest; same-batch preparation is required; preserve original provider call IDs, not generated call_N references; typed minimal settings replace opaque settings JSON; no schema-version field/envelope in the logical request; no new traversal-limit system. The issue was read, not edited. Do not publish its old manifest/accepted-only/mandatory historical-version-support wording as this design. Explicit non-goals: provider/SDK or production-engine integration, execution/recovery, hooks/guardrail execution, network/resource retrieval, compaction/generalized edits, external algorithm registration, auxiliary-call lifecycle/persistence, request persistence, public FromFormat factory, old-schema migrations or old-algorithm implementations. Auxiliary calls are a recognized future consumer of the source-independent request data model, not an extra framework in this slice. Experimental policy: Joe explicitly permits revising chronological-v1 and session payload schemas in place until he declares them locked. Do not add compatibility/changelog bookkeeping or dual readers. Earlier experimental sessions may no longer load; generic History can still expose raw records. Never fabricate missing provider IDs, discard old settings to make them load, or rewrite/delete stored histories. Keep verification honest: changed rendering may make an old experimental Check mismatch; never silently waive it. Grounded current seams
Material contracts1. General pinned commit inspection and request authoringAdd Shared inspection starts with HistoryID, ActiveRun, RecordCount, RecordAt, and Records. Snapshot keeps accepted semantics; CommitView.ActiveRun is the captured candidate scalar. Do not add speculative common methods or a session-owned polymorphic interface. Replace, do not overload/retain a bypass for: func (b *SessionCommit) AddModelRequest(run RunID, source CommitView, v ModelRequest) (ModelRequestID, error)Validate live/writable destination, active run/scope, exact originating batch and unchanged prefix, then zero synthesized fields and payload. Incoming RunID/ID/Source must be zero. Supply Source from the capture's accepted anchor and destination identity from the batch. Original batch identity plus prefix length proves freshness in O(1): no record comparison, record CID, public generation token, interface provenance plumbing, or capture registry. Any successful intervening addition invalidates attachment; failed additions do not. Failure consumes no identity and preserves the batch. Submit/freeze/discard prevents later additions; an uncertain append must use existing exact-proposal retry. Caller supplies Source semantics: accepted anchor A plus every record preceding the request in its own commit; exclude request/later records. Represent pre-genesis by an absent anchor: change Run start is explicit in the batch before capture, alongside EffectiveInput; construction never starts a run. A source without an active run, or with unresolved model/tool work, is not constructible. Do not infer that mutable Session state is the source of a captured view. 2. Retained session DTO changes
3. New packages and concrete DTOsNew // modelrequest/format
// Tags/field documentation follow the generator and neighboring kernel conventions.
type RequestV1 struct {
API string
Model string
Settings sessionformat.ModelSettings
RootInstructions []ContentBlock
UserContextPrefix []ContentBlock
Tools []sessionformat.ToolDefinition
Conversation []ConversationItem
}
type ContentBlock struct { Parts []sessionformat.ContentPart }
type ConversationItem struct { User *UserMessage; Assistant *AssistantMessage; ToolResult *ToolResult }
type UserMessage struct { Parts []sessionformat.ContentPart }
type AssistantMessage struct { Items []AssistantItem; Private *sessionformat.OpaqueProviderData }
type AssistantItem struct { Output *ContentBlock; Refusal *ContentBlock; Call *ToolCall }
type ToolCall struct { ProviderCallID string; Name string; Arguments []byte }
type ToolResult struct { ProviderCallID string; Outcome ToolOutcome; Parts []sessionformat.ContentPart }
type ToolOutcome string // returned-success | returned-error | not-invoked | indeterminate
type ParametersV1 struct { PrivateReplay []PrivateReplaySelector }
type PrivateReplaySelector struct { SourceAPI string; SourceModel string; Producer string; Format string; Version int64 }Exactly one pointer branch in each union. Reuse identity-free ContentPart/Media, ToolDefinition, OpaqueProviderData and ModelSettings DTOs; do not duplicate media schemas, move a shared primitive framework, or embed history-scoped ModelBinding/ModelResponse/ToolDisposition. ContentBlock preserves root/prefix and assistant-item boundaries, not new semantics or provenance. Tool definitions remain ordered with unique names. Calls preserve original name/argument bytes, including empty names or malformed/non-JSON arguments; never substitute effective invocation values. Correlate results through kernel call membership internally, emit original provider IDs, and check correlation scoped to each assistant response rather than session-global ID uniqueness. One logical result per call, requested-call order; no orphan/duplicate result occurrences. Assistant items retain output/refusal/call interleaving; refusal is provider-typed content, not inferred from prose or a terminal status. RequestV1 has NO version member, family envelope, route, credentials, source/history/run/request IDs, recipe, diagnostics, usage, evidence or embedded CID. Type/algorithm context supplies schema interpretation. Canonical bytes alone determine CID; identical bytes across future Go types share CID. General version-dispatch decoding and persistence envelopes (possibly the history Record envelope) are deferred. Nil/empty collections have the same meaning and canonical bytes; preserve meaningful optional present-empty leaves/envelopes.
Public operations: func Construct(source Source, parameters format.ParametersV1, limits codec.Limits) (ConstructionResult, error)
func Reconstruct(snapshot session.Snapshot, requestID session.ModelRequestID, limits codec.Limits) (Request, error)Construct derives the active run from Source; no separate RunID argument. Source is finite, immutable and internally consistent by contract, not hostile executable code. Zero/unscoped/unsuitable sources fail; errors return zero results, never partial usable requests/recipes. Do not expose public FromFormat yet. Keep internal finalization reusable without adding an auxiliary-call framework. 4. Chronological-v1 semantics and identityConstruct selects the built-in Check = contentid.Sum(codec.Encode(RequestV1, limits)).Binary(): exactly the existing 36-byte binary CID, no text, bare digest or wrapper. Parameters encode canonically into ConstructionRecipe.Parameters; ParametersV1 belongs specifically to chronological-v1 and is not in RequestV1. Empty raw parameter bytes are not an ambient-default shorthand; strict supported-shape decoding rejects unknown/missing/malformed data. Nil/empty PrivateReplay means omit all envelopes and has one canonical encoding. Recipe name/version/parameters and Route are not mixed into request CID. Recipes differing in irrelevant parameters can produce the same request CID. PrivateReplay selects whole envelopes by exact originating request API/model and envelope Producer/Format/Version. Empty list omits all; no wildcard interpretation, current registry lookup, target-equality inference or internal opaque-data edits. Selector order cannot reorder conversation; duplicate/equivalent matches do not duplicate envelopes. Caller-provided allow-list is provisional best-effort authorization, not proof of compatibility. Actual adapter policy discovery and replay constraints need usage experience; do not invent them here. Reconstruct locates the accepted request and its original boundary in the supplied pinned Snapshot, including same-commit preceding preparation. Use accepted range/record inspection and a small internal prefix reader, NOT a reconstructed historical lifecycle Snapshot or new permanent history index. Derive original active-run context from the request and validated prefix, never today's ActiveRun/latest context. Recompute from retained recipe and fail on malformed Check or mismatch. Unknown algorithm/mismatch must not prevent ordinary session inspection of otherwise supported data. 5. Rendering policy
6. Resource/ownership boundsUse existing codec.Limits directly for parameter processing and new request output; no new Limits struct, MaxVisitedItems, mutable defaults or traversal framework. Session policy already bounds accepted records/payload/decoded work; provisional builders and custom Source implementations remain trusted finite in-process inputs, not sandboxes. Avoid claiming that every open proposal has passed cumulative submission policy. Protect amplification: many tiny selections may repeat one large input. Preflight selected output sizes/counts and fixed framing/metadata overhead using overflow-safe arithmetic BEFORE governed slice growth, formatting or payload copies. Reuse codec budgeting/generated checks where applicable; a small sizing pass is preferable to unchecked build-then-Encode. Source-bounded association maps are not output-sized payload clones. Final generated Encode is an additional complete check, not the first bound. Too-small policy returns limit error and zero outputs; increasing policy cannot change successful bytes. Do not apply an output-size cap to the entire unrelated retained history or add semantic truncation. Report errors without dumping source content/private data. Dependency-ordered implementation tasks
Acceptance and verificationUse isolated memory history/reference adapters and controlled fixtures; no operator state or live providers. Test the actual authoring/admission/Submit/Load path, not only fake sources. Observable checks include:
During iteration, run focused Go tests for changed kernel packages. After integration run This is isolated engine work: engine/AGENTS.md permits focused race/architecture/API/standalone checks rather than a full repository suite when no host behavior is affected. If implementation touches root generator logic, host wiring or another module, broaden checks per root guidance: task test once per integrated set (1800s); before PR-ready handoff task lint (600s), task test:race (1800s) and offline Initial implementation panel-review baseline is the implementation branch's recorded start commit, captured when plan-implement creates it—not moving main and not an invented planning OID. Address findings with focused incremental review; no implicit rebase, commit, push or PR creation. When a PR is later authorized, retain a complete publication-safe copy of the reviewed plan in one identifiable PR conversation comment; record URL/ID in ignored run metadata and verify readback. Do not post/update issues or comments as part of this planning request. Remaining risk and review checkpointNo material behavioral decisions are intentionally open. Ordinary implementation choices (CBOR tags, internal helper names, concrete error identifiers, efficient projection-specific lookup) follow existing conventions; do not turn them into frameworks. Private-replay policy and future adapter/settings integration are explicitly provisional/deferred, not claims of verified compatibility. Same-version experimental changes can break prior CID reconstruction by design. If implementing or reviewing reveals a new material contradiction, stop and ask rather than weaken source/ownership/acceptance guarantees or expand scope. Independent review outcome2026-11-11: fresh-context read-only plan critique completed against the full inline plan and current source. One important correction: a zero CID cannot be serialized as a SourceHead child. Incorporated an optional ModelRequest.Source (nil only for pre-genesis), preserving strict present-CID validation and adding codec/materialization/Submit/Load regression coverage. The same reviewer rechecked the corrected contract against dtogen and confirmed the finding resolved; no remaining blocker or important gaps identified. Review agent: subagent-1550ab1a64c8647853c6b52488f4eb3d-call_21jQWj3yNvj2jAPSmmA8y2h5. Limitations: static design/source review only; no implementation, generated artifacts, or runtime qualification yet. No provider compatibility claim was independently verified; adapter integration remains deferred. No implementation tests were run during planning. Planning changed only this ignored scratch plan; no branch, tracked source, HANDOFF, issue or PR was changed. Implementation requires separate approval. |
Add pinned commit inspection and capture-bound request commitments with prospective same-batch preparation while retaining accepted execution barriers. Introduce typed settings, user-context prefixes and retained provider call IDs. Build owned chronological-v1 requests with bounded projection, explicit tool outcomes and exact private replay selection. Reconstruct accepted requests from their original prefixes and verify canonical request CIDs. Keep production entry points unchanged and add complete persistence/ownership examples. Refs #2663 Co-authored-by: Mecatl <noreply@mecatl.dev>
Align root and user instruction contracts, validate producer-owned keys, and demonstrate selective resume refresh with detached DTO clones. Preserve historical requests and keep contribution metadata out of logical request identity. Co-authored-by: Mecatl <noreply@mecatl.dev>
db4e5d1 to
e58d683
Compare
Build the request DTO directly and let final codec encoding enforce output limits. Remove the sizing pass and manual schema accounting; retain semantic checks and document the construction-memory tradeoff. Co-authored-by: Mecatl <noreply@mecatl.dev>
Join adjacent text byte-for-byte within each content sequence while preserving media and enclosing content boundaries. Cover segmentation-independent CIDs and storage-backed reconstruction; retain final codec validation. Co-authored-by: Mecatl <noreply@mecatl.dev>
Remove the redundant final request clone. Document success-only ownership transfer and retain public MutableClone and recipe isolation tests alongside canonical identity and limit checks. Co-authored-by: Mecatl <noreply@mecatl.dev>
Summary
Implements the isolated-kernel logical model-request construction and reconstruction slice of #2663. Merged on 2026-11-17 as
91e3155236feb5dd684f0f6bf51e805cad1d30a4.Production execution entry points are unchanged. This PR does not execute models/tools, integrate provider SDKs, or close #2663 or its parent #2659. Finalized accounting and capture-parity reassessment remain under #2663; executable integration/recovery and production provider fidelity remain #2664/#2665/#2667.
What changed
Pinned sources and request commitment
session.CommitViewcaptures immutable accepted plus proposed records in O(1), alongside existing acceptedSnapshotinspection.AddModelRequestchecks the originating batch and unchanged proposed prefix. Same-batch creation, run start, preparation and request commitment are supported, including a pre-genesis source anchor.Logical request construction
engine/kernel/modelrequestandmodelrequest/formatpackages own provider-neutral logical requests, generated codecs/borrowed views, construction recipes and canonical content CIDs.RootInstructionsis the harness root contract;UserInstructionsholds prepared persona/project guidance. Dynamically triggered guidance remains chronologicalEffectiveInput. RequiredInstructionBlock.Keyvalues are unique per complete collection and provide producer bookkeeping only: slice order wins, and keys are absent from logical content/CID.Reconstruction and ownership
Reconstructuses the request's original record prefix and run, including preparation earlier in its commit, rather than the current active run, context or configuration. It does not rebuild a historical lifecycleSnapshotor perform backend I/O.MutableCloneand recipe export detach mutable data.Reading guide
engine/kernel/session/commit_view.goand request admission insession_commit.go/lifecycle_state.go: source capture, freshness and same-batch legality.engine/kernel/modelrequest/construct.go,projection.go,tool_result_projection.go: selection and model-facing rendering.engine/kernel/modelrequest/request.go,construction_result.go,reconstruct.go: owned output, recipe/CID and historical reconstruction.engine/kernel/modelrequest/example_test.go: executable public flow; construction/reconstruction tests cover real Submit and storage-backed Load, later appends/runs, exact-byte rendering, failures and ownership. Session tests also demonstrate selective persona replacement by contribution key while retaining project guidance and old snapshots.Verification and review
The initial delivery passed the complete offline suite and race suite, all-module lint, standalone engine tests, generation/API checks, docs and the offline demo (tool call/result plus permission ask/approval).
Subsequent review-driven changes—standing instruction naming/keys, simpler projection, adjacent-text coalescing and finalizer backing transfer—were checked with relevant kernel/id/architecture/history-adapter race tests, standalone engine tests (
GOWORK=off), all-module lint, API checks and docs. The full root test/race suites and demo were not rerun at final HEAD; initial full-suite evidence is historical, not final-HEAD certification. No live-provider or paid calls were used.Spec, Standards, Test adequacy and relevant specialist reviews found no remaining findings at delivered HEAD
6547d67910b50c1a564a451f6fc9e1b218f7f430.PANEL: ship_blockers=0 important=0 advisory=0 reviewer_failures=0API compatibility and limits
AddModelRequest, optional pre-genesis source, typed settings, original provider call IDs and keyed root/user instruction collections change experimental contracts in place, as approved. Chronological-v1 is not yet locked. Earlier experimental payloads may fail to load/reconstruct; there are no migrations, dual readers, old-algorithm implementations or waived CID checks. Histories are not rewritten or deleted.Design provenance
The archived reviewed plan preserves the approved design and earlier naming choices. Its verification steps are intentions, not executed evidence; the merged behavior and qualification above include later review changes. Current code/Godoc and tests are authoritative.