[PIX] Fix mesh shader instrumentation

[Damyan Pepper (damyanp)]

Part 14 of 14 in the PIX instrumentation stack. It targets users/damyanp/pix-fixes-13. Its content depends on PR 2 for the declaration cleanup and the shared UAV behavior.

PIX instruments an amplification shader by expanding its payload to carry a group identifier. The mesh shader must agree on the resulting layout. When the two cannot be reconciled, the pass applies a locally derived layout that is known to disagree. The instrumented pair then disagrees about where each payload field lives.

A mesh shader whose payload is never read is skipped, so its output cannot be instrumented at all. A signed 16-bit output is zero-extended instead of sign-extended. Unused EmitIndices declarations stay in the module, and the validator refuses it. The reservation assertion checks a value that is already zero, so it does not catch a caller that asks for enough space to overwrite the offset counter.

The amplification shader reports the size and offset of the expanded payload through the expanded-payload-size and expanded-payload-offset options. The mesh shader reconstructs its payload type from that report. When reconstruction fails, the pass reports the failure and leaves the mesh payload unchanged. A mismatched pair produces records PIX cannot interpret, so there is no fallback layout.

The pass does not expand a payload that is already near the size limit.

If the PIX coordinator does not forward the layout of the amplification shader, the mesh shader leaves its payload unchanged. That is an integration item for PIX.

Assisted-by: Copilot

This changes only the PIX instrumentation, so it needs no release note.

---

_{Stack created with GitHub Stacks CLI • Give Feedback 💬}

[Copilot]

Pull request overview

Fixes PIX mesh-shader instrumentation so amplification and mesh shaders can share an expanded payload layout.

Changes:

• Reports and consumes expanded payload size and offset.
• Adds payload limits, alignment fixes, signed 16-bit handling, and declaration cleanup.
• Adds regression and validation coverage.

Reviewed changes

Copilot reviewed 10 out of 10 changed files in this pull request and generated 1 comment.

Show a summary per file

File	Description
utils/hct/hctdb.py	Registers payload-layout pass options.
tools/clang/unittests/HLSL/PixTest.cpp	Adds PIX instrumentation regression tests.
tools/clang/test/HLSLFileCheck/pix/SynthesizedPayloadTypeShapes.hlsl	Tests synthesized payload layouts.
tools/clang/test/HLSLFileCheck/pix/SynthesizedPayloadTypeRejectsUnusableLayout.hlsl	Tests invalid-layout rejection.
tools/clang/test/HLSLFileCheck/pix/MeshShaderWithoutEmitIndicesLeavesNoDeadDeclaration.hlsl	Verifies declaration cleanup.
tools/clang/test/HLSLFileCheck/pix/MeshPayloadExpansionAgreesWithAmplificationShader.hlsl	Verifies AS/MS layout agreement.
tools/clang/test/HLSLFileCheck/pix/MeshOutputSignedInt16IsSignExtended.hlsl	Verifies signed 16-bit extension.
tools/clang/test/HLSLFileCheck/pix/AddThreadIdWhenMSPayloadIsUnused.hlsl	Updates unused-payload expectations.
lib/DxilPIXPasses/DxilPIXMeshShaderOutputInstrumentation.cpp	Reconstructs payload layouts and fixes output instrumentation.
lib/DxilPIXPasses/DxilPIXAddTidToAmplificationShaderPayload.cpp	Reports layout and enforces payload constraints.

---

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.