Skip to content

feat: add dedicated gathering invoice read scaffolding - #4740

Draft
turip wants to merge 5 commits into
mainfrom
feat/gathering-invoice-table-search-view
Draft

turip wants to merge 5 commits into
mainfrom
feat/gathering-invoice-table-search-view

Conversation

@turip

@turip turip commented Jul 17, 2026 •

Copy link
Copy Markdown
Member

Summary

  • add the dedicated gathering invoice table and invoice search view scaffolding for legacy and dedicated storage
  • read gathering invoices and gathering lines from both the legacy billing tables and the dedicated gathering tables
  • route generic invoice listing through the search view and hydrate results from their actual storage table
  • extend subscription and split-line-group reads to include dedicated gathering lines

Migration behavior

This is transitional scaffolding for both storage sources. Reads support legacy and dedicated gathering invoice data while migration is in progress.

Conflicting rows are surfaced as errors instead of selecting one implicitly. We expect migrations between the legacy and dedicated tables to be atomic, so the same namespaced invoice or line existing in both sources indicates an invalid migration state.

Validation

  • POSTGRES_HOST=127.0.0.1 go test -tags=dynamic ./test/billing -run '^TestBillingAdapter$' -count=1
  • go vet -tags=dynamic ./openmeter/billing/adapter ./openmeter/billing/service ./test/billing
  • make migrate-check-validate

Greptile Summary

This PR adds dual-storage reads for gathering invoices during migration. The main changes are:

  • A dedicated gathering-invoice table and unified invoice search view.
  • Invoice hydration from legacy or dedicated storage.
  • Dual-source subscription-line and split-line-group reads.
  • New Ent schemas, generated clients, migrations, and adapter tests.

Confidence Score: 4/5

The dual-storage migration path can hide conflicting invoices, reject existing line data, and return split children twice.

  • The search view prevents read-side conflict checks from seeing both invoice rows.
  • Bulk deletion cannot remove dedicated gathering invoices.
  • Dedicated split-group lines can appear both inside a hierarchy and as top-level lines.
  • Existing gathering-line rows can prevent the foreign-key migration from applying.

openmeter/billing/adapter/gatheringinvoice.go, openmeter/billing/adapter/gatheringlines.go, openmeter/ent/schema/billing_invoice_search.go, and the gathering-invoice migration

Important Files Changed

Filename Overview
openmeter/billing/adapter/gatheringinvoice.go Adds search-based dual-storage hydration, but bulk deletion remains limited to legacy storage.
openmeter/billing/adapter/gatheringlines.go Adds dedicated subscription-line reads that can expose split-group children as independent lines.
openmeter/billing/adapter/invoice.go Routes generic invoice listing and type lookup through the unified search view.
openmeter/billing/adapter/invoicelinesplitgroup.go Extends split-line hierarchies to include both legacy and dedicated gathering lines.
tools/migrate/migrations/20260717060930_create_billing_gathering_invoices.up.sql Creates dedicated storage and the search view, but hides conflicting rows and may reject existing line data when repointing the foreign key.
openmeter/ent/schema/billing_invoice_search.go Defines the unified invoice read model used for storage routing and pagination.

Flowchart

%%{init: {'theme': 'neutral'}}%%
flowchart TD
    A[Invoice read] --> B[billing_invoice_search_v1s]
    B --> C[Legacy billing_invoices]
    B --> D[Dedicated billing_gathering_invoices]
    C --> E[Hydrate by namespace and ID]
    D --> E
    E --> F[Invoice response]
    G[Subscription line read] --> H[Legacy gathering lines]
    G --> I[Dedicated gathering lines]
    H --> J[Combined persisted state]
    I --> J
Loading
%%{init: {'theme': 'base', 'themeVariables': {"darkMode": true, "background": "#0d1117", "primaryColor": "#21262d", "primaryTextColor": "#e6edf3", "primaryBorderColor": "#8b949e", "lineColor": "#8b949e", "textColor": "#e6edf3", "edgeLabelBackground": "#161b22", "actorBkg": "#21262d", "actorBorder": "#8b949e", "actorTextColor": "#e6edf3", "actorLineColor": "#8b949e", "signalColor": "#8b949e", "signalTextColor": "#e6edf3", "noteBkgColor": "#373320", "noteBorderColor": "#d4a72c", "noteTextColor": "#f0e6c0", "labelBoxBkgColor": "#21262d", "labelBoxBorderColor": "#8b949e", "labelTextColor": "#e6edf3", "loopTextColor": "#e6edf3", "activationBkgColor": "#30363d", "activationBorderColor": "#8b949e"}}}%%
flowchart TD
    A[Invoice read] --> B[billing_invoice_search_v1s]
    B --> C[Legacy billing_invoices]
    B --> D[Dedicated billing_gathering_invoices]
    C --> E[Hydrate by namespace and ID]
    D --> E
    E --> F[Invoice response]
    G[Subscription line read] --> H[Legacy gathering lines]
    G --> I[Dedicated gathering lines]
    H --> J[Combined persisted state]
    I --> J
Loading

Fix All in Claude Code Fix All in Codex

Prompt To Fix All With AI
Fix the following 4 code review issues. Work through them one at a time, proposing concise fixes.

---

### Issue 1 of 4
tools/migrate/migrations/20260717060930_create_billing_gathering_invoices.up.sql:47
**View Hides Conflicting Invoices**

The legacy gathering branch excludes a row whenever the dedicated table contains the same invoice ID. During a partial or invalid migration, the search view therefore exposes only the dedicated row, so `GetGatheringInvoiceById`, `GetInvoiceType`, and list hydration cannot report the conflicting state and instead select one source implicitly.

### Issue 2 of 4
openmeter/billing/adapter/gatheringinvoice.go:30-50
**Dedicated Invoices Cannot Be Deleted**

This adapter now serves gathering invoices from both storage tables, but bulk deletion still updates only `BillingInvoice`. Passing an invoice stored in `billing_gathering_invoices` updates zero rows, returns `invoices failed to delete`, and leaves the dedicated invoice active.

### Issue 3 of 4
openmeter/billing/adapter/gatheringlines.go:67-88
**Split Children Become Top-Level Lines**

The dedicated query returns lines with a `SplitLineGroupID`, while `GetStandardLinesForSubscription` also loads those rows inside their split-line hierarchy. `Service.GetLinesForSubscription` appends both results, so subscription synchronization can receive the same dedicated line once as a hierarchy child and again as an independent gathering line, leading to duplicate or conflicting reconciliation.

### Issue 4 of 4
tools/migrate/migrations/20260717060930_create_billing_gathering_invoices.up.sql:42-44
**Foreign Key Repoint Can Block Migration**

The migration repoints every existing `billing_gathering_invoice_lines.invoice_id` from `billing_invoices` to the newly created `billing_gathering_invoices` table without first copying its parent invoice. If this line table already contains legacy rows, PostgreSQL rejects the new constraint because their parent IDs are absent from the new table, preventing the migration from applying.

Reviews (1): Last reviewed commit: "feat: support dedicated gathering invoic..." | Re-trigger Greptile

Greptile also left 4 inline comments on this PR.

Context used (3)

  • Context used - CLAUDE.md (source)
  • Context used - AGENTS.md (source)
  • Context used - api/spec/AGENTS.md (source)

@coderabbitai

coderabbitai Bot commented Jul 17, 2026

Copy link
Copy Markdown
Contributor

Important

Review skipped

Draft detected.

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro

Run ID: 97c79a07-2db3-4129-bc47-52a25ef74a07

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

  • 🔍 Trigger review
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch feat/gathering-invoice-table-search-view

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

CREATE INDEX "billinggatheringinvoiceline_invoice_id" ON "billing_gathering_invoice_lines" ("invoice_id");
-- create "billing_invoice_search_v1s" view
CREATE VIEW "billing_invoice_search_v1s" AS
SELECT "billing_invoices"."id", "billing_invoices"."namespace", "billing_invoices"."customer_id", "billing_invoices"."customer_name", "billing_invoices"."currency", 'billing_invoice' AS "storage_table", "billing_invoices"."type" AS "invoice_type", "billing_invoices"."status", "billing_invoices"."issued_at", "billing_invoices"."period_start" AS "service_period_start", "billing_invoices"."period_end" AS "service_period_end", "billing_invoices"."created_at", "billing_invoices"."updated_at", "billing_invoices"."deleted_at", "billing_invoices"."draft_until", "billing_invoices"."collection_at", "billing_invoices"."status_details_cache", "billing_invoices"."invoicing_app_external_id", "billing_invoices"."payment_app_external_id", "billing_invoices"."tax_app_external_id", "billing_invoices"."schema_level" FROM "billing_invoices" WHERE "billing_invoices"."status" <> 'gathering' UNION ALL SELECT "billing_invoices"."id", "billing_invoices"."namespace", "billing_invoices"."customer_id", "t1"."name" AS "customer_name", "billing_invoices"."currency", 'billing_invoice' AS "storage_table", 'gathering' AS "invoice_type", 'gathering' AS "status", NULL::timestamptz AS "issued_at", "billing_invoices"."period_start" AS "service_period_start", "billing_invoices"."period_end" AS "service_period_end", "billing_invoices"."created_at", "billing_invoices"."updated_at", "billing_invoices"."deleted_at", NULL::timestamptz AS "draft_until", "billing_invoices"."collection_at", NULL::jsonb AS "status_details_cache", NULL::text AS "invoicing_app_external_id", NULL::text AS "payment_app_external_id", NULL::text AS "tax_app_external_id", "billing_invoices"."schema_level" FROM "billing_invoices" JOIN "customers" AS "t1" ON "t1"."namespace" = "billing_invoices"."namespace" AND "t1"."id" = "billing_invoices"."customer_id" LEFT JOIN "billing_gathering_invoices" AS "t2" ON "t2"."namespace" = "billing_invoices"."namespace" AND "t2"."id" = "billing_invoices"."id" WHERE "billing_invoices"."status" = 'gathering' AND "t2"."id" IS NULL UNION ALL SELECT "billing_gathering_invoices"."id", "billing_gathering_invoices"."namespace", "billing_gathering_invoices"."customer_id", "t1"."name" AS "customer_name", "billing_gathering_invoices"."currency", 'billing_gathering_invoice' AS "storage_table", 'gathering' AS "invoice_type", 'gathering' AS "status", NULL::timestamptz AS "issued_at", "billing_gathering_invoices"."service_period_start", "billing_gathering_invoices"."service_period_end", "billing_gathering_invoices"."created_at", "billing_gathering_invoices"."updated_at", "billing_gathering_invoices"."deleted_at", NULL::timestamptz AS "draft_until", "billing_gathering_invoices"."next_collection_at" AS "collection_at", NULL::jsonb AS "status_details_cache", NULL::text AS "invoicing_app_external_id", NULL::text AS "payment_app_external_id", NULL::text AS "tax_app_external_id", "billing_gathering_invoices"."schema_level" FROM "billing_gathering_invoices" JOIN "customers" AS "t1" ON "t1"."namespace" = "billing_gathering_invoices"."namespace" AND "t1"."id" = "billing_gathering_invoices"."customer_id";

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 View Hides Conflicting Invoices

The legacy gathering branch excludes a row whenever the dedicated table contains the same invoice ID. During a partial or invalid migration, the search view therefore exposes only the dedicated row, so GetGatheringInvoiceById, GetInvoiceType, and list hydration cannot report the conflicting state and instead select one source implicitly.

Context Used: AGENTS.md (source)

Prompt To Fix With AI
This is a comment left during a code review.
Path: tools/migrate/migrations/20260717060930_create_billing_gathering_invoices.up.sql
Line: 47

Comment:
**View Hides Conflicting Invoices**

The legacy gathering branch excludes a row whenever the dedicated table contains the same invoice ID. During a partial or invalid migration, the search view therefore exposes only the dedicated row, so `GetGatheringInvoiceById`, `GetInvoiceType`, and list hydration cannot report the conflicting state and instead select one source implicitly.

**Context Used:** AGENTS.md ([source](https://app.greptile.com/openmeter/github/openmeterio/openmeter/-/custom-context?memory=eb020e3b-8e3b-45ea-a8b7-4006b2b2065b))

How can I resolve this? If you propose a fix, please make it concise.

Fix in Claude Code Fix in Codex

Comment on lines 30 to +50
var _ billing.GatheringInvoiceAdapter = (*adapter)(nil)

func (a *adapter) DeleteGatheringInvoices(ctx context.Context, input billing.DeleteGatheringInvoicesInput) error {
if err := input.Validate(); err != nil {
return billing.ValidationError{
Err: err,
}
}

return entutils.TransactingRepoWithNoValue(ctx, a, func(ctx context.Context, tx *adapter) error {
nAffected, err := tx.db.BillingInvoice.Update().
Where(billinginvoice.IDIn(input.InvoiceIDs...)).
Where(billinginvoice.Namespace(input.Namespace)).
Where(billinginvoice.StatusEQ(billing.StandardInvoiceStatusGathering)).
ClearPeriodStart().
ClearPeriodEnd().
SetDeletedAt(clock.Now()).
Save(ctx)
if err != nil {
return err
}

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Dedicated Invoices Cannot Be Deleted

This adapter now serves gathering invoices from both storage tables, but bulk deletion still updates only BillingInvoice. Passing an invoice stored in billing_gathering_invoices updates zero rows, returns invoices failed to delete, and leaves the dedicated invoice active.

Context Used: AGENTS.md (source)

Prompt To Fix With AI
This is a comment left during a code review.
Path: openmeter/billing/adapter/gatheringinvoice.go
Line: 30-50

Comment:
**Dedicated Invoices Cannot Be Deleted**

This adapter now serves gathering invoices from both storage tables, but bulk deletion still updates only `BillingInvoice`. Passing an invoice stored in `billing_gathering_invoices` updates zero rows, returns `invoices failed to delete`, and leaves the dedicated invoice active.

**Context Used:** AGENTS.md ([source](https://app.greptile.com/openmeter/github/openmeterio/openmeter/-/custom-context?memory=eb020e3b-8e3b-45ea-a8b7-4006b2b2065b))

How can I resolve this? If you propose a fix, please make it concise.

Fix in Claude Code Fix in Codex

Comment on lines +67 to +88
dedicatedQuery := tx.db.BillingGatheringInvoiceLine.Query().
Where(billinggatheringinvoiceline.Namespace(in.Namespace)).
Where(billinggatheringinvoiceline.SubscriptionID(in.SubscriptionID)).
Where(
billinggatheringinvoiceline.Or(
billinggatheringinvoiceline.DeletedAtIsNil(),
billinggatheringinvoiceline.And(
billinggatheringinvoiceline.DeletedAtNotNil(),
billinggatheringinvoiceline.ManagedByEQ(billing.ManuallyManagedLine),
),
),
).
WithTaxCode()

if !in.IncludeChargeManaged {
dedicatedQuery = dedicatedQuery.Where(billinggatheringinvoiceline.ChargeIDIsNil())
}

dbDedicatedLines, err := dedicatedQuery.All(ctx)
if err != nil {
return nil, fmt.Errorf("fetching dedicated gathering lines: %w", err)
}

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Split Children Become Top-Level Lines

The dedicated query returns lines with a SplitLineGroupID, while GetStandardLinesForSubscription also loads those rows inside their split-line hierarchy. Service.GetLinesForSubscription appends both results, so subscription synchronization can receive the same dedicated line once as a hierarchy child and again as an independent gathering line, leading to duplicate or conflicting reconciliation.

Context Used: AGENTS.md (source)

Prompt To Fix With AI
This is a comment left during a code review.
Path: openmeter/billing/adapter/gatheringlines.go
Line: 67-88

Comment:
**Split Children Become Top-Level Lines**

The dedicated query returns lines with a `SplitLineGroupID`, while `GetStandardLinesForSubscription` also loads those rows inside their split-line hierarchy. `Service.GetLinesForSubscription` appends both results, so subscription synchronization can receive the same dedicated line once as a hierarchy child and again as an independent gathering line, leading to duplicate or conflicting reconciliation.

**Context Used:** AGENTS.md ([source](https://app.greptile.com/openmeter/github/openmeterio/openmeter/-/custom-context?memory=eb020e3b-8e3b-45ea-a8b7-4006b2b2065b))

How can I resolve this? If you propose a fix, please make it concise.

Fix in Claude Code Fix in Codex

Comment on lines +42 to +44
ALTER TABLE "billing_gathering_invoice_lines" DROP CONSTRAINT "billing_gathering_line_invoice_fk", ADD CONSTRAINT "service_period_not_inverted" CHECK (service_period_start <= service_period_end), ADD CONSTRAINT "billing_gathering_line_invoice_fk" FOREIGN KEY ("invoice_id") REFERENCES "billing_gathering_invoices" ("id") ON UPDATE NO ACTION ON DELETE CASCADE;
-- create index "billinggatheringinvoiceline_invoice_id" to table: "billing_gathering_invoice_lines"
CREATE INDEX "billinggatheringinvoiceline_invoice_id" ON "billing_gathering_invoice_lines" ("invoice_id");

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Foreign Key Repoint Can Block Migration

The migration repoints every existing billing_gathering_invoice_lines.invoice_id from billing_invoices to the newly created billing_gathering_invoices table without first copying its parent invoice. If this line table already contains legacy rows, PostgreSQL rejects the new constraint because their parent IDs are absent from the new table, preventing the migration from applying.

Context Used: AGENTS.md (source)

Prompt To Fix With AI
This is a comment left during a code review.
Path: tools/migrate/migrations/20260717060930_create_billing_gathering_invoices.up.sql
Line: 42-44

Comment:
**Foreign Key Repoint Can Block Migration**

The migration repoints every existing `billing_gathering_invoice_lines.invoice_id` from `billing_invoices` to the newly created `billing_gathering_invoices` table without first copying its parent invoice. If this line table already contains legacy rows, PostgreSQL rejects the new constraint because their parent IDs are absent from the new table, preventing the migration from applying.

**Context Used:** AGENTS.md ([source](https://app.greptile.com/openmeter/github/openmeterio/openmeter/-/custom-context?memory=eb020e3b-8e3b-45ea-a8b7-4006b2b2065b))

How can I resolve this? If you propose a fix, please make it concise.

Fix in Claude Code Fix in Codex

@turip turip added the keep-open Exempt from stale processing; requires a concrete maintainer reason label Aug 11, 2026

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

keep-open Exempt from stale processing; requires a concrete maintainer reason

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant