Skip to content

[Bug][Fix] Handle concatenation of 1D empty tensors with higher-dimensional tensors - #20762

Open
Netra-Srinivasan-McW wants to merge 2 commits into
apache:mainfrom
Netra-Srinivasan-McW:concat-empty-tensor-fix
Open

Netra-Srinivasan-McW wants to merge 2 commits into
apache:mainfrom
Netra-Srinivasan-McW:concat-empty-tensor-fix

Conversation

@Netra-Srinivasan-McW

@Netra-Srinivasan-McW Netra-Srinivasan-McW commented Oct 16, 2026 •

Copy link
Copy Markdown

Issue

PyTorch supports concatenating a 1D empty tensor (shape (0,)) with a higher-dimensional tensor. On the other hand, TVM Relax raises an error when the input tensors have different numbers of dimensions.
This issue occurs in models such as Perplexity AI's pplx-embed-v2-late-9b, where the KV cache is initially represented as a 1D empty tensor. Concatenating it with higher-dimensional key/value tensors causes the Relax frontend conversion to fail.

Solution

This PR updates the Relax frontend to handle concatenation of a 1D empty tensor with a higher-dimensional tensor, enabling model execution to proceed without this error.

Closes #20761.

@Netra-Srinivasan-McW

Copy link
Copy Markdown
Author

@tqchen @tlopex Could you review this PR when you find time. Thanks.

@Netra-Srinivasan-McW

Copy link
Copy Markdown
Author

Hi @mshr-h, @yzhliu, @tqchen, @tlopex,
Could you please take a look at this PR when you get a chance. Thanks.

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

Filtering empty inputs produces incorrect dtype promotion, while the singleton shortcut bypasses rank and axis validation, allowing invalid concatenations to succeed.

2 open issues
What changed in this PR

Updates Relax’s PyTorch frontend to handle concatenation of empty 1D tensors with higher-rank tensors, addressing initially empty model caches described in #20761.

Changes:

  • Filters known (0,) inputs and returns the remaining tensor directly when only one remains.
  • Preserves the original input list when all tensors are empty.
  • Adds regression tests for both frontend import paths.
File Description
tests/​python/​relax/​test_frontend_from_fx.py Adds an empty-tensor concatenation regression for FX import.
tests/​python/​relax/​test_frontend_from_exported_program.py Adds the corresponding exported-program regression.
python/​tvm/​relax/​frontend/​torch/​base_fx_graph_translator.py Implements empty-input filtering and the single-tensor shortcut.

Review effort: Max

@Netra-Srinivasan-McW

Copy link
Copy Markdown
Author

Hi @mshr-h, @yzhliu, @tqchen, @tlopex,

I have updated the PR to address all the review changes:

  • Mixed-Dtype Rejection Preserved: Added a check that bypasses filtering and delegates directly to relax.op.concat, ensuring TVM's dtype validation catches mixed types.
  • 1D Empty Tensor Filtering: Added additional condition for filtering check to prevent Relax rank-mismatch errors when concatenated with higher-dimensional tensors.
  • Singleton Rank & Axis Validation: Added bound checks before returning a single retained tensor (len(filtered) == 1), ensuring out-of-bounds dimensions and rank-0 scalars raise ValueError matching PyTorch semantics instead of silently returning unchecked.

Could you please take a look into the changes when you get a chance? Thanks!

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Bug] [Relax] Support concatenation of empty 1D tensors with higher-rank tensors

2 participants