Skip to content

Spec errata: repository-artifact-layer names three vocabularies codeanalyzer-python v1.3.0 never implemented #48

Open

Description

@rahlk

Problem

docs/design/specs/repository-artifact-layer.md, tracked by #45, mandates three shared vocabularies that do not exist in the shipped reference implementation. codeanalyzer-python's child (codellm-devkit/codeanalyzer-python#152) is closed and v1.3.0 is released, so this is drift between a committed contract and a completed train — not work-in-progress.

Verified directly against the v1.3.0 tag (git rev-parse HEAD == git rev-parse v1.3.0 == 2520bcb, working tree clean):

  1. artifact_kind — does not exist. The spec defines it as a typed closed enum (build_manifest | dependency_lockfile | configuration | deployment_manifest | container | infrastructure | ci | script | documentation | data | other). What ships is two bare str fields on PyArtifact: format: str (one value: toml|yaml|json|ini|properties|requirements|dockerfile|text|binary) and roles: List[str] (many values: dependency-manifest|tool-config|container-image|service-topology|ci|env|packaging|legal|docs|iac|unknown|script), both minted by a module-level RULES table in codeanalyzer/artifacts/discovery.py with first-match-wins glob semantics.
  2. scope — does not exist. The spec defines runtime | development | test | build | optional | unknown. What ships is PyDependency.kind: str = "runtime" with the vocabulary runtime|dev|optional|build — four values, not six, and dev rather than development.
  3. The ecosystem mapping table — does not exist. The spec says it is "coined once in the schema, not per-analyzer". What ships is ecosystem: str = "pypi" on the model plus two hardcoded ecosystem="pypi" literals (artifacts/dependencies.py:111,184) and one in the Neo4j projection (neo4j/project.py:363). The field's own comment reads "the only ecosystem this analyzer emits".

For completeness: the entire codeanalyzer-python package contains exactly one Enum — EmitTarget in options/options.py:7.

Why this needs resolving now, not later

codellm-devkit/codeanalyzer-java#197 says the shared vocabulary is "coined verbatim per the spec's parity clause, identical to the python/typescript trains". Those two clauses now contradict each other: Java cannot be simultaneously faithful to the spec and to the train it is supposed to match. Whichever it picks, one of them silently becomes wrong, and the parity clause exists precisely to stop a term being coined twice.

Two further spec claims are also outdated by what shipped, and belong in the same reconciliation:

  • The spec scopes transitive/lockfile-only packages out. Python emits them — PyDependency(direct=False, prov=["lockfile"]) for every lock pin with no manifest declaration (artifacts/dependencies.py:181-186). The direct: bool field exists to carry exactly that distinction.
  • The spec treats artifacts as first-class nodes in the containment tree. Python hangs artifacts/dependencies/config_uses as sibling maps and lists on PyApplication, keeping symbol_table strictly code-keyed; only config_keys nest, inside their owning artifact.

Options

  1. Amend the spec to what shipped — format + roles[], kind with four values, ecosystem as a per-analyzer literal, sibling maps, transitive-in. Cheapest, and makes the reference implementation the contract.
  2. Amend python to the spec — a breaking change to a released analyzer with no consumers yet (its graph SCHEMA_VERSION deliberately stayed 2.0.0 through the whole layer, so no consumer can even detect its presence).
  3. Split: keep the spec's richer vocabulary as the target, mark python v1.3.0 as a documented partial, and have Java implement the spec — accepting that the two analyzers disagree until python catches up.

My recommendation is 1, on the grounds that the spec's enums were never validated against a real implementation and the shipped vocabulary is strictly more expressive in one place that matters (roles is many-valued, so a pyproject.toml is both dependency-manifest and tool-config, which a single closed artifact_kind cannot represent).

Caveats and known risks

  • Option 1 loses the closed-enum guarantee, so a typo in a role string becomes silent data rather than a validation error. Python has no conformance-schema coverage for the artifact layer at all — test_v2_conformance.py, test_v2_superset.py and test_v2_two_projection_agreement.py contain zero references to artifacts or config — so nothing currently catches that class of drift in any case.
  • Option 2 is the only one that preserves the spec as written, and it is the most expensive.
  • Whichever is chosen, codeanalyzer-typescript's child (Repository-artifact layer: artifacts, dependencies, config keys (python parity) codeanalyzer-typescript#101) is still open and should be told before it starts, not after.

Definition of done

Activity

  1. rahlk commented on Aug 31, 2026

    @rahlk
    ContributorAuthor

    Errata addendum from implementing the Java train (codellm-devkit/codeanalyzer-java#206): the config-key id grammar has now diverged, deliberately, and the reference's form is the one that is wrong.

    codeanalyzer-python v1.3.0 mints the compose-style env dual-mint id as <artifact-id>@key/env.<NAME> (config_keys.py:576-578 into ids.py:39). That self-collides: a plain YAML dotted path can itself begin with the segment env., so a document carrying both a top-level env: block and a compose services.<name>.environment block produces two records — different namespace, different value — sharing one id. The reference's own docstring shows it guarded the bare-top-level-key-versus-bare-var-name case and missed the dotted-path one.

    Java now mints <artifact-id>@key:env/<NAME> instead. The two forms share the <artifact-id>@key prefix and diverge at a fixed character position before any variable content, so collision is impossible by construction rather than by luck. It is pinned by a schema-level test asserting the two shapes can never collide for the same key name, and the divergence is recorded in CanId.configKeyEnvDualMintId's javadoc.

    Why this belongs on this errata rather than in one repo's tracker: ConfigKey is an un-prefixed cross-language merge target — the whole point of the un-prefixed label is that a sibling-language analyzer over the same repository lands on the same node. Until python adopts the fixed grammar, the two analyzers mint different nodes for the same env var in the same docker-compose.yml, splitting exactly the nodes the shared label exists to unify. That is a worse failure than the collision it fixes, and it is not resolvable from the Java side.

    Recommend python converge on the @key:env/ form, and that whichever grammar this errata settles on be stated in the spec so the TypeScript train (codellm-devkit/codeanalyzer-typescript#101) does not have to rediscover it.

    One further item for this errata while it is open, raised by the same work: the graph wipe. Java's projector deliberately does not traverse into Artifact/Package/ConfigKey during a scoped wipe, because doing so would DETACH DELETE nodes a sibling analyzer shares and destroy its edges. Python's wipe does not reach them either, so both analyzers currently accept the same staleness: remove a config file, re-push, and its nodes linger orphaned. The better fix is an orphan sweep run after the re-push's merges, but that is a cross-analyzer contract — both need it, and both must agree on ordering, or one tool's sweep deletes what the other's next push expects to re-attach. Worth a decision here rather than in either repo.

  2. rahlk commented on Sep 11, 2026

    @rahlk
    ContributorAuthor

    The vocabulary item (format + open roles[], dependency kind, per-analyzer ecosystem) is closed by spec in #80, which states the shipped vocabulary as the contract on main and adds the first role under it (view-template). The @key:env/ id grammar and the orphan sweep on re-push are not addressed there and keep this issue open.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions