Skip to content

feat(search)!: engine- and domain-agnostic query model, Typesense compiler, and GraphQL surface#529

Open
ddeboer wants to merge 10 commits into
mainfrom
feat/search-core
Open

feat(search)!: engine- and domain-agnostic query model, Typesense compiler, and GraphQL surface#529
ddeboer wants to merge 10 commits into
mainfrom
feat/search-core

Conversation

@ddeboer

@ddeboer ddeboer commented Jun 28, 2026

Copy link
Copy Markdown
Member

What

Reworks @lde/search and @lde/search-typesense into a unified, engine- and
domain-agnostic
search API, and adds @lde/search-api-graphql (the new GraphQL surface).
@lde/search and @lde/search-typesense already existed in the repo; this reworks them
(breaking), it does not introduce them. One declarative SearchSchema drives projection, the
engine collection schema, the query semantics, and the GraphQL surface – so they cannot drift. The domain type (Dataset,
Person, …) and the engine choice (Typesense, …) are the consumer’s, configured at the
seams; the libraries never name a domain.

Packages

@lde/search (core) — breaking

  • Unifies the field model: one SearchField / SearchSchema replaces the projection
    FieldSpec/Projection and the discriminated FieldKind.
  • Adds the neutral query IR (SearchQuery / Filter / Sort) and filter-operator semantics.
  • Adds the SearchEngine port and logical result types (SearchResult / SearchHit /
    ResultDocument / Reference / LocalizedValue / FacetBucket).
  • Adds physicalFields (the shared physical-fanout convention) and schema selectors
    (searchableFields, facetableFields, filterableFields, sortableFields, outputFields).
  • Rewrites projectDocument/projectGraph onto the unified model; projection output is
    unchanged — the guardrail test was ported field-for-field.
  • BREAKING: FieldSpec, Projection, and the discriminated FieldKind are removed.

@lde/search-typesense (engine adapter) — additive

  • buildCollectionSchema derives a Typesense collection from the field model (kind→type, the
    physical fanout via physicalFields, per-locale stemming, required / default-sorting-field).
  • buildSearchParams compiles SearchQuery into Typesense params — filter_by / sort_by /
    facet_by / query_by with active-locale weighting, grouped-facet OR, and exact membership
    for non-facet fields.
  • createTypesenseSearchEngine implements the SearchEngine port end to end: it reconstructs
    logical documents and resolves reference (and reference-facet) labels from the sidecar
    labels collection in a single lookup.
  • Covered by unit tests plus a Typesense testcontainer integration test.

@lde/search-api-graphql (GraphQL surface) — new

  • buildSearchSchema(schema, { typeName }) builds an executable GraphQLSchema at runtime
    from any SearchSchema (no codegen, no SDL artifact), served by one generic resolver over
    any SearchEngine.
  • Derives output types, where/orderBy/facet inputs, named reference types, and nullability
    (from required / array / kind) from the field model; best-first Accept-Language
    output ordering; a nullable facet label resolved for reference facets only.
  • Exports printSearchSchema for a consumer-side SDL snapshot guard.

Notes

  • ADRs 0003/0004 updated: the unified model, the SearchEngine rename (the interface is the
    port; the Typesense class is the adapter), sizeFloat (int64 overflow), the
    typed-surface design, and facet labels.
  • Each generator ships a neutral-fixture snapshot to pin its output across versions.
  • Deferred: the idOnly/inline reference strategies, the OutputOf<S> typed-surface
    overlay, and a REST surface.

ddeboer added 3 commits June 28, 2026 20:26
…d result types

- replace FieldSpec and Projection with one SearchField/SearchSchema model

- add SearchQuery, Filter, Sort and the filter-operator semantics

- add the SearchEngine port and result types (SearchResult/SearchHit/ResultDocument/Reference)

- add physicalFields (the shared fanout convention) and schema selectors

- rewrite projectDocument and projectGraph onto the unified model; projection output unchanged

- remove FieldSpec, Projection and the discriminated FieldKind (breaking)
… and SearchEngine

- buildCollectionSchema derives a Typesense collection from the unified SearchField model

- buildSearchParams compiles SearchQuery into Typesense params (filter_by/sort_by/facet_by/query_by)

- createTypesenseSearchEngine implements the SearchEngine port: compile, search, reconstruct

- resolve reference and reference-facet labels from the sidecar labels collection in one lookup

- add a testcontainer integration test and a generator-stability snapshot
- buildSearchSchema builds an executable GraphQLSchema from any SearchSchema at runtime (no codegen)

- one generic resolver maps args to SearchQuery, calls the engine, and maps the result back

- derive output, where, orderBy and facet types plus nullability from the field model

- best-first Accept-Language output ordering; nullable facet label for reference facets

- add printSearchSchema for a consumer SDL snapshot, plus a generator-stability snapshot
@ddeboer ddeboer force-pushed the feat/search-core branch from ae639ea to 66969a2 Compare June 28, 2026 18:51
ddeboer added 2 commits June 29, 2026 09:38
- state the decisions directly as the reconciled architecture, not deviations from a draft

- remove the deviation/reconcile framing and the deviations-to-reconcile lists

- align wording with the stack platform layer
- number fields now project as floats (not truncated like integer)

- closes the step-1 gap so an int64-magnitude field mapped to number (Float) indexes
Replace the repo-path breadcrumb with a direct link to the docs site, so the
status note points readers at the rendered page rather than a source file path.
@ddeboer ddeboer changed the title feat: unified engine- and domain-agnostic search API (@lde/search* family) feat(search)!: engine- and domain-agnostic query model, Typesense compiler, and GraphQL surface Jul 1, 2026
ddeboer added 4 commits July 1, 2026 09:47
… the group companion

- Keyed per-type facets object on the GraphQL surface (ValueBucket / RangeBucket),
  selection-is-the-request with skip-own-filter.
- Numeric range facets and an opt-in label cache in the Typesense adapter.
- Reconcile ADRs 0003 and 0004 with the implementation.

BREAKING CHANGE: remove SearchField.group and its *_group companion field, collection
column and query split. Deployments denormalize group tokens into the field values
instead, so a group is an ordinary facet value with no engine mechanism.
…@lde/* pins

npm ci failed because the lockfile lacked the new @lde/search-api-graphql workspace.
Regenerating against npmjs adds it and brings ~24 @lde/* internal deps up to their latest
in-range patches; no third-party or duplicate-version changes.
… search-engine test

`result.facets` is a `Partial` record, so a facet is `FacetBucket[] | undefined`; guard the two
spreads with `?? []` so the `typecheck` target passes (it never ran in CI before the lockfile fix).
…ations

Fold the unified-field-model blockquote and the dated Consequences bullet into running
text, so the ADR reads as the current design rather than a change log.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant