[HynekBlaha]

DuckDB-WASM has no Azure filesystem: the official azure extension has no wasm build, so LoQE could not read abfss:// tables and ADLS warehouses were excluded from the storage whitelist.

azure_wasm (https://community-extensions.duckdb.org, source: HynekBlaha/duckdb-azure-wasm) is a loadable wasm-only extension that claims the abfss/abfs/az/wasbs schemes, registers the azure secret type and rewrites requests to https://<host>/<container>/<key>?<sas>. That is exactly the shape DuckDB's iceberg extension produces from Lakekeeper's vended adls.sas-token.<account>.<suffix> credentials (account_name + AccountName=...;SharedAccessSignature=...), so no fork of duckdb-wasm is needed.

It is only published for DuckDB core v1.5.5, hence the duckdb-wasm bump:

• pin @duckdb/duckdb-wasm to 1.33.1-dev64.0 (core v1.5.5; the latest dist-tag still predates the v1.5.5 bump, and a caret range would not match 1.33.1-dev*)
• vendor extensions for v1.5.5 and pull azure_wasm from the community repository, which uses the same /v<ver>/<platform>/ layout, so the air-gapped mirror keeps working unchanged
• LOAD azure_wasm before the first catalog scan, otherwise DuckDB tries to autoload the official azure extension and 404s against the mirror. Best-effort: s3/gcs warehouses do not need it
• ATTACH option is STAGE_CREATE_TABLES since iceberg 1.5.x; the old SUPPORT_STAGE_CREATE is silently ignored, which disabled staged creates
• allow adls in the DuckDB-WASM storage whitelist

The lockfile also gains duckdb-wasm's new qs dependency tree.

Verified against the published binaries: core reports v1.5.5 / wasm_eh, INSTALL + LOAD azure_wasm succeed, azure shows up in duckdb_secret_types() and abfss:// paths reach the extension.

Tested complete integration with local lakekeeper fork build:

Summary by CodeRabbit

• New Features

  • Added support for connecting to Azure Data Lake Storage (ADLS) using SAS-token authentication.
  • Added compatibility for Azure-specific storage extensions while preserving support for other catalog types.

• Bug Fixes

  • Updated Iceberg catalog attachment behavior for compatibility with the latest DuckDB release.
  • Improved catalog connections so unavailable optional Azure functionality does not prevent non-Azure catalogs from working.

[coderabbitai]

Important

• 🔍 Trigger review

This repository does not receive automatic reviews because it has fewer than 10 stars.

⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Advanced

Run ID: e50a1957-b517-4cdd-9ce6-3c737d52153f

Walkthrough

This change updates DuckDB WASM and extension versions, adds community extension downloads for azure_wasm, enables ADLS validation and runtime loading, and replaces the Iceberg catalog attachment option with STAGE_CREATE_TABLES.

Changes

DuckDB and ADLS integration

Layer / File(s)	Summary
DuckDB extension packaging package.json, scripts/download-duckdb-extensions.mjs, src/composables/loqe/LoQEEngine.ts	Pins DuckDB WASM to 1.33.1-dev64.0, updates extensions to v1.5.5, and downloads azure_wasm from the community repository.
ADLS validation and runtime loading src/composables/useStorageValidation.ts, src/composables/loqe/LoQEEngine.ts	Adds adls storage support and loads azure_wasm before Iceberg, while continuing when the community extension is unavailable.
Iceberg attachment option migration src/composables/loqe/CatalogManager.ts	Uses STAGE_CREATE_TABLES true for initial catalog attachment and reattachment flows.

Estimated code review effort: 3 (Moderate) | ~20 minutes

Sequence Diagram(s)

sequenceDiagram
  participant attachCatalog
  participant azure_wasm
  participant Iceberg
  attachCatalog->>azure_wasm: Install and load extension
  azure_wasm-->>attachCatalog: Return loaded or unavailable
  attachCatalog->>Iceberg: Load and attach catalog
  Iceberg-->>attachCatalog: Return attachment result

Loading

Possibly related PRs

• lakekeeper/console-components#89: Updates DuckDB WASM dependencies and related assets.
• lakekeeper/console-components#228: Changes LoQEEngine.ts extension repository handling.
• lakekeeper/console-components#231: Updates pinned DuckDB extension downloads.

Suggested reviewers: v-kessler

Merge Risk: 🟡 Moderate · up to 58d0f

This change adds ADLS warehouse support and updates Iceberg staging behavior, but an azure_wasm loading failure may leave an ADLS catalog appearing available while queries fail later, and the staged-create option still needs confirmation against the pinned runtime. Merge should wait for targeted verification or explicit owner acceptance.

🚥 Pre-merge checks | ✅ 5

✅ Passed checks (5 passed)

Check name	Status	Explanation
Description Check	✅ Passed	Check skipped - CodeRabbit’s high-level summary is enabled.
Title check	✅ Passed	The title clearly and concisely describes the main change: enabling LoQE to read ADLS warehouses through the azure_wasm community extension.
Docstring Coverage	✅ Passed	No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.

✨ Finishing Touches

🧪 Generate unit tests (beta)

• Create PR with unit tests

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{I’m a rabbit with Azure in sight,
DuckDB extensions now load just right.
Catalog stages hop into place,
SAS tokens guide the data chase.
azure_wasm joins the burrow tonight.}

_{Comment @coderabbitai help to get the list of available commands.}

[cla-assistant]

All committers have signed the CLA.

[HynekBlaha]

@coderabbitai review

[coderabbitai]

✅ Action performed

Review finished.

Note: CodeRabbit is an incremental review system and does not re-review already reviewed commits. This command is applicable only when automatic reviews are paused.

[coderabbitai]

🔇 Additional comments (5)

package.json (1)

90-90: LGTM!

scripts/download-duckdb-extensions.mjs (1)

17-20: LGTM!

Also applies to: 39-48, 89-98, 108-108

src/composables/loqe/LoQEEngine.ts (1)

598-603: 🎯 Functional Correctness | 🟠 Major | 🏗️ Heavy lift

⚠️ Unverified finding
Sandbox verification was unavailable.

Do not silently accept an ADLS catalog when azure_wasm fails.

useStorageValidation.ts now marks adls as supported, but this catch only logs the error and lets attachCatalog resolve. An ADLS query can then fail later with no user-facing feedback. Route the error through handleError(e, context, notify?). If the catalog is ADLS, propagate the failure or mark the catalog unavailable. Keep the fallback only for non-ADLS catalogs.

As per coding guidelines, always use handleError(error, context, notify?) for error handling in components and API functions.

src/composables/useStorageValidation.ts (1)

16-19: LGTM!

src/composables/loqe/CatalogManager.ts (1)

52-60: 📐 Maintainability & Code Quality

Verify the option behavior against the exact vendored Iceberg build.

The current DuckDB REST-catalog documentation lists STAGE_CREATE_TABLES. However, a recent DuckDB-WASM report shows support_stage_create producing an “Unhandled options found” error. Run the initial attach and both reattach flows against the pinned 1.33.1-dev64.0 WASM bundle and vendored v1.5.5 extension. Reword the comment if unknown options are rejected rather than silently collected. (duckdb.org)

Also applies to: 159-159, 231-231

---

ℹ️ Review info

⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro Plus

Run ID: 10e5bcda-60f5-4e1d-aad4-299c61a597e9

📥 Commits

Reviewing files that changed from the base of the PR and between ebcd080 and 58d0fd0.

⛔ Files ignored due to path filters (1)

• package-lock.json is excluded by !**/package-lock.json

📒 Files selected for processing (5)

• package.json
• scripts/download-duckdb-extensions.mjs
• src/composables/loqe/CatalogManager.ts
• src/composables/loqe/LoQEEngine.ts
• src/composables/useStorageValidation.ts

Included review availability: Your plan includes up to 1 review per rolling hour; 0 remain after this review.

[v-kessler]

Tested this locally against a Lakekeeper 0.14 build (console dev server, local
lakekeeper-plus), merged onto current main. Summary of what we found:

Works

• ADLS reads via azure_wasm — verified against a real ADLS Gen2 account with SAS
  vended credentials.
• S3 (AWS STS warehouse): full DML on core 1.5.5 — CREATE, INSERT, UPDATE, DELETE,
  MERGE, CTAS, ALTER ADD/RENAME COLUMN, DROP.
• GCS: works on 1.5.5 as well.
• Air-gap holds: the v1.5.5 mirror (incl. azure_wasm, both platforms) is served from
  our own origin and INSTALL azure_wasm resolves there — no runtime CDN traffic.

Important finding: the reason LoQE was pinned to duckdb-wasm 1.4.x was an apparent
1.5.x regression breaking Iceberg writes. That was in fact the silently-renamed ATTACH
option this PR fixes — SUPPORT_STAGE_CREATE was ignored rather than rejected, which
disabled staged creates and broke every write. With STAGE_CREATE_TABLES set, writes
work again on 1.5.5. So this PR unblocks the DuckDB upgrade in general, not just ADLS.

ADLS is read-only, which is worth stating explicitly:
INSERT fails with Not implemented Error: AzureFileSystem: DirectoryExists is not implemented!. Is write support planned for the wasm build, or out of scope?

Requested changes

1. Rebase onto main — the PR is based on 0.21.0 and currently conflicts. Only
   package-lock.json conflicts; no source conflicts and it builds clean.
2. Prune stale extension versions in download-duckdb-extensions.mjs — bumping
   EXTENSION_VERSIONS leaves old version dirs (v1.4.3 etc.) in duckdb-extensions/
   and they get copied into dist/ and into consumer apps' public/. The script
   should delete version dirs not in EXTENSION_VERSIONS.
3. Suppress duckdb-wasm's request-header logging. The 1.33.x line added an
   unconditional console.log("HEAD", <header>, <value>) in the sync-XHR glue
   (absent in 1.32.1-dev1.0), so every S3 read prints a live SigV4 Authorization
   and the full x-amz-security-token to the browser console. Not controllable via
   ConsoleLogger/LogLevel. Since we build the worker from a Blob ourselves, a
   filter before importScripts handles it:
   console.log = (...a) => { if (a[0] === 'HEAD' && a.length === 3) return; __log(...a); }
   Worth an upstream issue on duckdb/duckdb-wasm too — it affects every consumer.
4. Surface the read-only status for ADLS: a friendlyQueryError mapping for the
   AzureFileSystem: … not implemented case (that helper already does this for the
   CORS/404 case) and/or a notice, since useStorageValidation has no read/write
   distinction and ADLS now presents as fully supported.
5. Docs: the storage account needs a Blob service CORS rule (account-wide; the
   dfs endpoint honours it) with the console origin, GET/HEAD/OPTIONS and exposed
   headers *. Without it every ADLS read fails at the preflight — first thing users
   will hit.

Also: the CLA check is still pending.