Skip to content

fix: handle shallow clones in dbt clone, table, and incremental rebuilds - #1592

Merged
sd-db merged 9 commits into
databricks:mainfrom
saishreeeee:PECOBLR-3690
Jul 20, 2026
Merged

fix: handle shallow clones in dbt clone, table, and incremental rebuilds#1592
sd-db merged 9 commits into
databricks:mainfrom
saishreeeee:PECOBLR-3690

Conversation

@saishreeeee

@saishreeeee saishreeeee commented Jul 16, 2026

Copy link
Copy Markdown
Collaborator

Resolves #1165

Description

Shallow clones have a distinct table_type that can't be changed in place, so dbt clone and table/incremental rebuilds over a clone failed (UPDATE_TABLE_TYPE, CLUSTER_BY_AUTO_UNSUPPORTED_TABLE_TYPE_ERROR).
Adds is_shallow_clone, drops the clone before recreating it (clone/table/incremental), and skips CLUSTER BY AUTO when altering a clone in place.

Checklist

  • I have run this code in development and it appears to resolve the stated issue
  • This PR includes tests, or tests are not required/relevant for this PR
  • I have updated the CHANGELOG.md and added information about my change to the "dbt-databricks next" section.
  • [Optional] I have run /dbt-databricks-pr-ready (AI agent skill in .claude/skills/) and addressed its merge-readiness feedback

@saishreeeee saishreeeee changed the title fix: drop shallow clones before rebuild and skip CLUSTER BY AUTO on c… fix: handle shallow clones in dbt clone, table, and incremental rebuilds Jul 16, 2026
@saishreeeee
saishreeeee marked this pull request as ready for review July 17, 2026 04:42
Comment thread CHANGELOG.md Outdated
Comment thread dbt/include/databricks/macros/materializations/clone/strategies.sql Outdated
Comment thread dbt/include/databricks/macros/materializations/incremental/incremental.sql Outdated
Comment thread dbt/include/databricks/macros/materializations/incremental/incremental.sql Outdated
Comment thread tests/unit/macros/materializations/test_clone_macros.py
@saishreeeee
saishreeeee requested a review from sd-db July 20, 2026 06:58

@sd-db sd-db left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Changes look good !!

@sd-db
sd-db merged commit 8c6456a into databricks:main Jul 20, 2026
@lgpreston75

Copy link
Copy Markdown

@sd-db do you know when this fix will be released?

@sd-db

sd-db commented Jul 23, 2026

Copy link
Copy Markdown
Collaborator

@lgpreston75 hopefully this coming monday

sd-db added a commit that referenced this pull request Aug 9, 2026
Resolves a conflict in `table.sql` between this branch's `replaced_in_place`
refactor and #1592, which added `is_shallow_clone` to the two drop conditions
that `replaced_in_place` negates.

Since a shallow clone's table type cannot be changed in place, it must be
dropped and recreated -- so it is not replaced in place and inherits no tags.
Folded the term into the predicate rather than into the drop sites, keeping
both as `not replaced_in_place`:

  replaced_in_place = existing_relation
      and not existing_relation.is_shallow_clone
      and existing_relation.type == 'table'
      and existing_relation.can_be_replaced
      and adapter.resolve_file_format(config) in ('delta', 'iceberg')

`can_be_replaced` alone does not cover this: it tests relation type plus
delta/iceberg provider, so a shallow clone of a delta table passes it.

Add `TestRebuildOverShallowCloneAppliesAllTags`, which covers the interaction
both changes touch: rebuilding a tagged table over a shallow clone must drop
the clone and apply all tags to the fresh table. Without the `is_shallow_clone`
term it fails on `MANAGED_SHALLOW_CLONE != MANAGED` -- i.e. it guards #1592's
fix, not only the tag diff.

Also move this branch's changelog entry to the 1.12.4 (TBD) section; the
automatic merge placed it under the already-released 1.12.2 heading.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

dbt clone doesn't work out of the box

3 participants