Repository navigation
[PyTorch] FP8BS grouped GEMM test reference fix - #3722
Conversation
0d14e26 to
49df5dd
Compare
|
/te-ci pytorch |
 [Low impact] The PR appears safe to merge, with a non-blocking request to retain the mixed-format rejection test. Findings
SummaryReplaces the FP32 reference with separately quantized, split GEMMs for the FP8 block-scaling and changing-splits CUDA graph tests.
Reviews (4) · Last reviewed commit: "[PyTorch] Test grouped FP8 block-scaling..." · Reviewed by Greptile |
![greptile-apps[bot]](https://avatars.githubusercontent.com/in/867647?s=60&v=4){kind=small}
49df5dd to
2717644
Compare
|
/te-ci pytorch |
…r path MXFP8 GEMM on Blackwell accumulates at higher precision than FP8 block-scaling GEMM on Hopper. The grouped FP8 block-scaling GEMM tests compared against an FP32 reference, so they passed with the MXFP8 emulation on Blackwell but failed on Hopper. Compare against the legacy split-tensor grouped GEMM instead, consistent with the existing MXFP8 grouped GEMM tests. Also remove the FP8 block-scaling zero-size-expert and mixed-operand tests, which no other recipe has. Signed-off-by: Alp Dener <adener@nvidia.com>
…xperts Dropping the varying splits and the zero-size-expert test left gaps: the remaining tests no longer run grouped FP8 block-scaling GEMM with uneven or empty experts when the weights or outputs are per-expert lists, when accumulating, or when K is not a multiple of 128. Parametrize the grouped FP8 block-scaling GEMM test over uniform, varying and zero-expert splits, checked against the split-tensor reference. Signed-off-by: Alp Dener <adener@nvidia.com>
fe14444 to
cc9d122
Compare
| assert all(ms % 128 == 0 for ms in m_sizes) | ||
| _run_fp8bs_grouped_gemm(z, m_sizes, k, n, layout, case, weight_block_dim, accumulate, dtype) | ||
|
|
||
|
|
There was a problem hiding this comment.
The deleted test_grouped_gemm_grouped_tensor_fp8_block_scaling_mixed_operands_rejected was the only test that mixed FP8 block-scaling and MXFP8 operands. The replacement cases use matching formats, so a later change could stop rejecting mixed formats on Blackwell without failing these tests.
Keep the negative test and switch its inputs to _tile_scaled_randn; it does not depend on the old FP32 reference.
Note: If this suggestion doesn't match your team's coding style, reply to this and let me know. I'll remember it for next time!
|
/te-ci pytorch |
Include upstream NVIDIA#3722 so Hopper grouped FP8 block-scaling tests use the corrected reference. Preserve the existing MXFP8 stride-fix commits and their focused diff. Signed-off-by: bvolpato <brunocvcunha@gmail.com>
Description
FP8BS grouped GEMM tests were being compared against a reference with FP32 accumulation. FP8BS on Blackwell via MXFP8 emulation passed these tests because MXFP8 grouped GEMM on Blackwell accumulates at a higher precision than native FP8BS grouped GEMM on Hopper.
This PR changes the reference in the test to the unfused split-grouped GEMM — consistent with how native MXFP8 grouped GEMM is tested.
Type of change
Checklist: