Skip to content

build(deps)!: update Substrait packages to 0.96.0#209

Open
bvolpato wants to merge 1 commit into
substrait-io:mainfrom
bvolpato:bvolpato/update-substrait-0.96
Open

build(deps)!: update Substrait packages to 0.96.0#209
bvolpato wants to merge 1 commit into
substrait-io:mainfrom
bvolpato:bvolpato/update-substrait-0.96

Conversation

@bvolpato

Copy link
Copy Markdown
Member

Summary

Changes

  • Regenerate both uv.lock and pixi.lock.
  • Remove legacy timestamp, time, timestamp_tz, and aggregate grouping fields removed from Substrait.
  • Migrate derivation evaluation to 0.96 ANTLR contexts, including arithmetic, comparison, equality, boolean, ternary, and if expressions.
  • Keep external engine round-trip tests opt-in with SUBSTRAIT_ENGINE_TESTS=1; bundled consumers can crash natively when reading 0.96 plans.

Repro

PR #181 leaves pixi.lock stale and fails against removed protobuf fields and renamed ANTLR contexts. This PR regenerates both locks and adapts affected code paths.

Testing

  • pixi install --locked
  • uv run --python 3.10 --frozen pytest (204 passed, 30 skipped)
  • pixi run lint
  • pixi run format --check
  • ./check_substrait_package_versions.sh
  • All four example scripts

Did this cause problems?

External DuckDB, DataFusion, and PyArrow round-trip tests cannot safely gate this upgrade yet because their bundled Substrait consumers lag 0.96 and can segfault. They remain available through explicit opt-in.

BREAKING CHANGE: removes support for legacy timestamp, time, timestamp_tz, and Grouping.grouping_expressions fields removed from Substrait.

Regenerate uv and Pixi locks, remove fields dropped from the spec, and migrate derivation expressions to the refactored ANTLR contexts.

Keep native engine round trips opt-in while their bundled Substrait consumers lag the pinned spec and can crash the interpreter.

BREAKING CHANGE: removes support for legacy timestamp, time, timestamp_tz, and Grouping.grouping_expressions fields removed from Substrait.

Co-authored-by: Niels Pardon <par@zurich.ibm.com>
@tokoko

tokoko commented Jul 13, 2026

Copy link
Copy Markdown
Contributor

looks good, but I don't want to create additional work for Niels to rebase unless we are in a rush. #204 is probably not too far from mergable state anyway. @nielspardon wdyt?

@nielspardon

Copy link
Copy Markdown
Member

looks good, but I don't want to create additional work for Niels to rebase unless we are in a rush. #204 is probably not too far from mergable state anyway. @nielspardon wdyt?

Yes, that also was my initial reaction. Rebases are less of a hassle for me these days since I can usually also just delegate that to an AI.

I think it is awesome to have you @bvolpato as another committer helping out getting our code base updated to the latest spec though I feel like we should coordinate amongst ourselves. If we manage to coordinate then we could e.g. also tackle the other integrations like Datafusion and DuckDB which then would allow us to enable the integration tests again in substrait-python.

There's also quite a long list of spec upgrade PRs waiting for review in e.g. the substrait-java repo where a committer could help us make progress by reviewing those and then tackling some of the other outstanding upgrade issues.

@nielspardon

Copy link
Copy Markdown
Member

@bvolpato we managed to get #204 merged in the meantime. Is there anything left in this PR we did not cover in #204 that you need?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Update to Substrait v0.96.0

3 participants