[VL] Fix run-scala-test.sh and add skills for rebase#12558
Draft
FelixYBW wants to merge 4 commits into
Draft
Conversation
Upstream Velox's New Commits: 8bece69ac by Linsong Wang, Merge branch 'ci-fix-pr' into staging-rebase-pr 669c9f2e4 by wforget, [OAP] Change SpillPartitionId::kMaxSpillLevel to 7 a6e61e605 by Ke Jia, [OAP]feat: Implement se/dser method for HashTable f0b511b59 by Yuan, [OAP] [11771] Fix smj result mismatch issue in semi, anit and full outer join 7ab4ec425 by Rui Mo, feat: Allow subfield rename and deletion for Parquet format 53a990726 by Zhenyuan Zhao, fix(dwrf): Bound PATCHED_BASE gap chain to avoid OOB read (#18106) e1dd48045 by Hongze Zhang, refactor: Split `dwio/common/Statistics.h` to `.h` and `.cpp` 5219cafbd by oerling, feat(torchwave): Free intermediate tensors at last use (#18119) d8ceba11c by Wechar Yu, fix(spark): Support VARBINARY type in to_json function 1861f170f by Krishna Pai, fix: Revert folly double/float formatting goldens (#18116) ea4ae633b by Apurva Kumar, feat: Replace (union) an existing deletion vector on repeated mutation (#18117) a3c8b8657 by Zhenyuan Zhao, fix: Fix test failure (#18103) 949c7309b by Deepak Majeti, feat(parquet): Add support for Timestamp type rowgroup filtering 0b01ba0ad by Deepak Majeti, fix(build): FBThrift warnings for parquet.thrift 1e7952ade by Apurva Kumar, test: NIMBLE Iceberg field-id end-to-end write+read test (#18056) 332b91f61 by PRASHANT GOLASH, perf: Use DirectBufferedInput for Nimble (enable loadQuantum) (#17948) db15b91d3 by Liangcai Li, feat(cudf): Add Spark substring support fe6aae45d by Reema, fix(cudf): Fix multi column `hash_with_seed` on GPU 049fe62d6 by Philo He, fix: Enable ANSI-compliant cast from VARCHAR to DECIMAL c59294974 by Amit Dutta, fix(window): Fail gracefully on oversized window partitions (#18102) efb391ea4 by Richard Barnes, Update test goldens for folly double/float string formatting change d12dbd60e by jkhaliqi, feat: Add CompressionKind_LZ4_HADOOP 7f6858340 by Daniel Bauer, fix(cudf): Support ms/s timestamp scalars on GPU 4991078e0 by Ping Liu, docs: Scope BMI/BMI2/F16C requirements to x86 4920e8bf3 by Ismaël Mejía, perf(dwio): Cross-platform bit-unpacking and level conversion 4cf8f85db by Ismaël Mejía, fix(hive): Wire file-handle-expiration-duration-ms to SimpleLRUCache (#18072) c1287a247 by meta-codesync[bot], misc: Add Thrift AllowLegacyMissingUris annotation to Velox files (#18012) c759d48ad by Raymond Lin, fix(dwio): Correct flat-map prefetch eligibility (#18038) 039614eda by Ismaël Mejía, fix(dwio): Advance pointers in unpackNaive after decoding loop a8c470526 by Durvesh Pilankar, docs: Fix duplicated words and typos affccb3dc by Kyle Hubert, fix(ucx): Preserve shared exchange client on operator close (#17752) d1f09a430 by binwei yang, upgrade to gcc-133 + spark-3.5 build 6841cd103 by binwei yang, remove spark3.4 7531da59e by binwei yang, add cache key e962632aa by Yuan, fix: Enable CI on IBM repo Signed-off-by: glutenperfbot <glutenperfbot@glutenproject-internal.com>
Contributor
There was a problem hiding this comment.
Pull request overview
This PR updates Velox integration and developer tooling to better support the Velox daily rebase workflow, including fixing local Scala test execution when suites live in dependency modules.
Changes:
- Bump the default Velox branches used by the build scripts to the latest daily rebase snapshot.
- Improve
dev/run-scala-test.shclasspath resolution by locating suite.classfiles across moduletarget/*/test-classesdirectories when needed. - Update Iceberg writer construction to match an updated Velox
IcebergInsertTableHandleconstructor signature (adds an empty deletion-vector map argument).
Reviewed changes
Copilot reviewed 3 out of 3 changed files in this pull request and generated 1 comment.
| File | Description |
|---|---|
| ep/build-velox/src/get-velox.sh | Updates the pinned Velox branch names to the newer daily snapshot. |
| dev/run-scala-test.sh | Enhances classpath assembly to include dependency modules’ test-classes / classes when suites aren’t found in the driver module. |
| cpp/velox/compute/iceberg/IcebergWriter.cc | Adjusts Iceberg insert table handle creation to pass a new (empty) deletion-vector map parameter. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
We are handling velox daily rebase. We need to use AI to accelerate the process