Repository navigation
F16 coverage: pruning, chunked, Pco, CSV, CLI, Calcite, Parquet (follow-up to #515) - #521
Merged
Merged
Conversation
ScanIterator#boxedScalar had no Float16Array case, so an F16 zone-map table decoded to no min/max at all: a filter on an F16 column read every chunk and VortexReader#columnStats reported nothing. It now boxes the widened half as a Float, like F32. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Sko6B2519bE1TkpPx7WKxf
ChunkedArrayCombiner had no F16 case, so a nested chunked layout or a vortex.chunked array of half-precision values threw 'unsupported ptype for chunked layout'. New ChunkedFloat16Array concatenates the chunks as raw 16-bit halves, like the other chunked views; the default branch of the ptype switch is gone, so a new PType fails to compile there. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Sko6B2519bE1TkpPx7WKxf
CsvExporter#cellValue had no Float16Array case and threw 'unsupported array type for CSV export'. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Sko6B2519bE1TkpPx7WKxf
FilterCommand#compareValue threw 'filter not supported for column type' on F16, and the grid and inspector renderers printed the '<MaterializedFloat16Array ...>' placeholder instead of the value. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Sko6B2519bE1TkpPx7WKxf
) VortexEnumerator returned a Double for an F32 column that VortexTable declares REAL, which Calcite's generated code casts to Float: every SQL query that read such a column failed with a ClassCastException. The adapter's own test pinned the Double (it read rows without going through SQL); Float32ColumnSqlTest runs select and where over JDBC. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Sko6B2519bE1TkpPx7WKxf
VortexTable#toSqlType threw 'unsupported ptype: F16', so a table with a half-precision column could not be created at all; its rows, SUM and filter literals had no F16 case either. SQL has no half type, so F16 maps to REAL like F32 and reads as the Float it widens to exactly. The default branches of those ptype switches are gone, so a new PType fails to compile there. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Sko6B2519bE1TkpPx7WKxf
Rust's compact preset runs Pco on half-precision floats (pco has f16: 16-bit latents ordered like f32's), but PcoEncodingEncoder rejected F16 and PcoEncodingDecoder threw 'unsupported ptype'. The latents machinery was already width-generic (I16/U16), so F16 adds the ordered mapping, the half array and the FloatMult/FloatQuant joins (explicit F16/F32/F64 switches, no default). PcoF16RustParityTest checks both directions against pco 1.0.1's own output (fixtures dumped by a scratch crate): Java encodes the Classic shapes byte for byte, and decodes all five shapes, among them FloatQuant. FloatMult is untested: pco never chose it for a half shape. Float16 round-trip and vortex-jni interop tests now include forced Pco and the compact cascade. Java's 16-bit delta binning is far worse than Rust's (2474 vs 192 bytes on a ramp, u16 too); not touched here. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Sko6B2519bE1TkpPx7WKxf
… ALP-RD (#515) Pins the F16 encoder set to Rust's: btrblocks declines half floats for ALP and ALP-RD only. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Sko6B2519bE1TkpPx7WKxf
…, both directions (#515) Float16InteropIntegrationTest adds a nullable column written in three chunks (Java writes, JNI reads) and in three batches (JNI writes, Java reads), and a filtered scan over a sorted column written by either side: no matching row may be lost, and Java's zone map must prune. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Sko6B2519bE1TkpPx7WKxf
…mitive (#515) ChunkImpl#adaptPrimitive took only short[] for F16, so the null-marking boxed form that I8..F64 all accept was rejected as a type mismatch. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Sko6B2519bE1TkpPx7WKxf
ParquetExporter wrote nothing for an F16 column (UnsupportedOperationException) and ParquetImporter rejected every FIXED_LEN_BYTE_ARRAY. F16 now maps to FIXED_LEN_BYTE_ARRAY(2) with the Float16Type annotation, the little-endian half; the importer reads only that annotated form (any other fixed-length byte array stays unsupported), keeps the 16 raw bits so a NaN payload or -0.0 survives, and handles nullable columns. Nested F16 in a list or struct is still unsupported. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Sko6B2519bE1TkpPx7WKxf
dfa1
added a commit
that referenced
this pull request
Oct 10, 2026
Eleven one-per-fix lines (#515, #517, #521) become a single F16 entry that lists the commits; the Calcite F32 fix stays separate, being a different bug. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Sko6B2519bE1TkpPx7WKxf
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Follow-up to #515 (merged in #517). After the first fix, I audited F16 end to end: every encoding x data shape x plain/nullable/multi-chunk, read back by the Java reader and by vortex-jni, then every
switchthat handles F32 but not F16. These are the remaining gaps. Eleven commits, one per fix; please rebase-merge, not squash.5dfb50bdcolumnStats()for F16 (boxedScalarhad noFloat16Arraycase, so nothing was ever pruned)58c7db6dChunkedFloat16Array)65db1cc7d9435345filter, grid and inspector handle F163bbcc6b0Float, so SQL on them runs (existing bug:ClassCastException)0be3dc55REALb2af32ebvortex.pcoencodes and decodes F16 (Rust's compact preset)3ec8914c9c20c2b40573f067Short[]like every other primitive058a0816FLOAT16Verification
Float16InteropIntegrationTest: 58 cases against vortex-jni 0.86.1 (Java writes / JNI reads, JNI writes / Java reads, nullable, multi-batch, filtered)../mvnw testand./mvnw verify -pl integration -am(427 tests, 2 skipped) pass locally.Not fixed here (filed separately)
-0.0vs0.0)vortex.sparseon all-equal data writes zero patches, which Rust rejectsNested F16 inside a Parquet list or struct is still unsupported.
🤖 Generated with Claude Code