Skip to content

[Relax][FRONTEND][Pytorch] Add support for searchsorted - #20760

Open
Netra-Srinivasan-McW wants to merge 3 commits into
apache:mainfrom
Netra-Srinivasan-McW:searchsorted-support
Open

Netra-Srinivasan-McW wants to merge 3 commits into
apache:mainfrom
Netra-Srinivasan-McW:searchsorted-support

Conversation

@Netra-Srinivasan-McW

@Netra-Srinivasan-McW Netra-Srinivasan-McW commented Oct 16, 2026 •

Copy link
Copy Markdown

This PR adds support for the searchsorted operator in TVM Relax, which is used in transformer models such as Perplexity AI's pplx-embed-v2-late-9b.

The following requirements have been covered:

  • C0: Check whether the operator can be decomposed into existing Relax operators.
  • C1: Introduce legalization rules to enable compilation on CPU.
  • C2: Ensure the operator compiles on CUDA.

Tests have been added to validate the operator's behavior and compilation across the supported targets, covering both static and dynamic shapes.

Closes #20739

@Netra-Srinivasan-McW

Copy link
Copy Markdown
Author

@tqchen @tlopex Could you review this PR and provide your feedback. Thanks.

@Netra-Srinivasan-McW

Copy link
Copy Markdown
Author

Hi @mshr-h, @yzhliu, @tqchen, @tlopex,
Could you please take a look at this PR when you get a chance. Thanks.

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

Missing searchsorted rank and batch-shape validation allows incompatible inputs to reach CPU code that reads beyond the sequence buffer.

5 open issues
What changed in this PR

Adds PyTorch searchsorted support to TVM Relax, addressing the unsupported operator reported in #20739 for models such as pplx-embed-v2-late-9b.

Changes:

  • Introduces the Relax operator, attributes, type inference, and Python/script APIs.
  • Adds FX and exported-program conversion, CPU legalization, and GPU dispatch.
  • Adds structural and CPU numerical tests covering search options and dynamic batch shapes.
File Description
tests/​python/​relax/​test_frontend_from_fx.py Adds structural FX import coverage.
tests/​python/​relax/​test_frontend_from_exported_program.py Adds structural and numerical tests, including dynamic shapes.
src/​relax/​op/​tensor/​search.h Declares the C++ operator constructor.
src/​relax/​op/​tensor/​search.cc Registers the operator, attributes, and type inference.
python/​tvm/​relax/​transform/​legalize_ops/​search.py Adds CPU legalization through TOPI.
python/​tvm/​relax/​script/​ir_builder/​op.py Exposes R.searchsorted in Relax scripts.
python/​tvm/​relax/​op/​search.py Adds the public Python wrapper and documentation.
python/​tvm/​relax/​op/​__init__.py Exports the new operator.
python/​tvm/​relax/​frontend/​torch/​fx_translator.py Registers FX conversion.
python/​tvm/​relax/​frontend/​torch/​exported_program_translator.py Registers tensor and scalar overloads.
python/​tvm/​relax/​frontend/​torch/​base_fx_graph_translator.py Translates operands, scalar constants, and search options.
python/​tvm/​relax/​backend/​dispatch_sort_scan.py Selects CPU or GPU TOPI implementations.
include/​tvm/​relax/​op/​search.h Defines output-dtype and insertion-side attributes.

Review effort: Max

Comment thread src/relax/op/tensor/search.cc Resolved
@Netra-Srinivasan-McW

Copy link
Copy Markdown
Author

Hi @mshr-h, @yzhliu, @tqchen, @tlopex,

I have updated the PR to address all the review changes:

  • Rank & Batch Validation: Updated InferTypeSearchSorted in search.cc to reject rank-0 sequences and enforce batch/outer-dimension equality via sym::Analyzer for N>1, preventing TOPI out-of-bounds reads.
  • GPU Dispatch & Tests: Corrected operand ordering (sorted_sequence, values) and explicitly passed target to topi.gpu.searchsorted in dispatch_sort_scan.py to eliminate reliance on ambient target contexts. Added CPU/CUDA structural dispatch and N-D symbolic compilation tests in test_backend_dispatch_sort_scan.py.
  • FX Operand Resolution: Resolved node.kwargs via _retrieve_args() with input/values/self fallback in base_fx_graph_translator.py, validated side ("left"/"right").
  • Scalar Promotion & Regressions: Promoted integer sequences alongside float scalars via relax.op.astype to avoid index truncation; added Python-scalar numerical tests in test_frontend_from_exported_program.py.

Could you please take a look into the changes made when you get a chance? Thanks!

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Bug] [Relax] Unsupported function type searchsorted.Tensor

2 participants