Refactor build pipeline to separate test jobs by hoyosjs · Pull Request #5875 · dotnet/diagnostics

hoyosjs · 2026-06-11T11:32:33Z

This PR changes diagnostics CI from "build and test in every leg" to "build once, then run separate test legs from published artifacts".

New state:

the build stage produces the binaries/debuggees/test assets once
the test stage depends on those build jobs and runs tests from downloaded artifacts - it also installs the appropriate SDKs
tools stay on net8, but unit tests move to net10 to avoid installing more runtimes.
a couple of overly noisy MSBuild messages are downgraded

How this PR fixes it

diagnostics.yml and eng/pipelines/build.yml now model build and test as separate stages/jobs. Build legs publish TestArtifacts_*; test-only legs download them and run test.cmd/test.sh.
Build legs now publish bin/**, obj/**, and test/** so test legs can run from prebuilt outputs.
eng/build.ps1 / eng/build.sh explicitly install test runtimes on -skipmanaged legs and pass SkipTestArtifactsBuild=true so debuggees are not rebuilt there.
eng/TraversalOverrides.targets is imported after the Traversal SDK and clears TestDependsOn when SkipTestArtifactsBuild=true. That is the key fix that actually stops implicit build-before-test.
src/tests/dirs.proj skips BuildDebuggees on those prebuilt test legs.
Test TFMs now use $(NetCoreAppTestTargetFramework) (net10.0) while tool TFMs remain $(NetCoreAppMinTargetFramework) (net8.0), which lets us drop the extra net8 test runtime install from global.json.
global.json keeps an x86 runtime, but now at net10, so Arcade can launch an x86 host on the x86 leg.
EventLogsPipelineUnitTests replaces reader.EndOfStream assertions with ReadToEndAsync() checks so those tests remain stable on the newer runtime.
Logging cleanup:
- eng/AuxMsbuildFiles/SdkPackOverrides.targets: High -> Low (from Change message importance from High to Low in debuggee build #5869)
- src/tests/SOS.UnitTests/Debuggees/Directory.Build.targets: skip message lowered

- Publish full test artifacts from build-only legs and consume them in test-only legs with -skipmanaged\n- Gate test artifact publish on buildOnly to avoid duplicate or invalid publish work\n- Ensure test runtimes/sdk install in skipmanaged test legs via explicit InstallRuntimes build step\n- Keep tools on net8.0 while moving unit tests to net10.0\n- Remove global.json tools.runtimes dependency and fix net10 analyzer warnings in EventLogsPipelineUnitTests\n\nCo-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Build legs already compile all debuggee TFMs and install test runtimes; test legs were redundantly rebuilding both. Publish artifacts/dotnet-test in the TestArtifacts package so test legs download SDK+runtimes and skip InstallRuntimes and BuildDebuggees via SkipTestArtifactsBuild. buildOnly still builds everything. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

CopyFiles/DownloadPipelineArtifact dropped exec bits on the bundled dotnet-test host, breaking every test that launches a debuggee. Install runtimes locally on test legs (cheap, correct perms) and only skip the expensive debuggee rebuild via SkipTestArtifactsBuild. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

The clrmd 4.1 bump in main (dotnet#5913) carries the .NET 8 fix, so the unit tests can run on net10 again. Since the live SDK already ships the net10 runtime, the net8 shared runtime no longer needs to be installed via global.json's tools.runtimes node. Reverts the temporary 'Keep tests on net8' workaround. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

The x86 test legs failed because Arcade launches the xUnit runner via $(DotNetRoot)x86/dotnet.exe and silently falls back to the x64 host when that path is missing. Without an x86 runtime in .dotnet/x86, the runner ran 64-bit, so the DbgShim/SOS harness resolved the native binary directory from the process architecture as Windows_NT.x64.Release and failed to load the x86 dbgshim.dll/SOS bits. Restore the tools.runtimes dotnet/x86 entry (pointed at net10) so Arcade provisions the x86 host the test runner needs. dotnet-test only provides the arch-matched runtime for debuggees, not the runner host itself. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

- Include obj/** in the published TestArtifacts so the test-only legs have the MSBuild intermediate outputs. Without them the incremental up-to-date check misses and CoreCompile/Csc re-runs for product and test projects on the test leg even though bin/** was downloaded. - Test-only legs now invoke test.cmd/test.sh, which already encode '-test -skipmanaged -skipnative' and do not prepend the inert '-restore -build' that build.cmd/build.sh add. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

The Microsoft.Build.Traversal SDK hardcodes TestDependsOn=Build, so every traversal's Test target (build.proj, src/dirs.proj, src/tests/dirs.proj) rebuilds its entire subgraph before running tests, regardless of whether -build was passed. On the test-only CI legs the product and test binaries are downloaded as prebuilt TestArtifacts; rebuilding them is wasted work, and because pipeline artifact downloads do not preserve file timestamps the incremental check fails and it becomes a full recompile. Override TestDependsOn (empty) via CustomAfterTraversalTargets when the managed build was skipped (SkipTestArtifactsBuild=true, set by build.ps1/ build.sh for -skipmanaged). The Test target still invokes Test on each referenced project, so tests run directly against the downloaded binaries. Validated locally: test leg binlog shows Csc=0, CoreCompile not invoked, RunTests executed on all test projects. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Lower the importance of two diagnostic messages that were emitted at High importance (shown even at minimal verbosity) and added noise to every build: - SdkPackOverrides.targets: OverrideTestingVersions message -> Low (folds in dotnet#5869). - Debuggees Directory.Build.targets: interpreter-test skip message -> default importance. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Copilot

Pull request overview

This PR refactors the CI pipeline to separate build and test jobs by publishing full build outputs as “TestArtifacts” and enabling test-only legs to download and run tests without rebuilding. It also updates unit test projects to target a newer test TFM and adjusts build/test scripts to support “download artifacts then test” flows.

Changes:

Move unit tests to $(NetCoreAppTestTargetFramework) (introduced as net10.0) while keeping product/tool TFMs on the minimum supported TFM.
Add “test-only” CI legs that download TestArtifacts_* and run tests via test.cmd/test.sh, skipping debuggee rebuilds.
Add MSBuild traversal overrides to drop the traversal Test -> Build dependency when running on prebuilt test-artifact legs.

Reviewed changes

Copilot reviewed 18 out of 18 changed files in this pull request and generated 5 comments.

Show a summary per file

File	Description
src/tests/SOS.UnitTests/SOS.UnitTests.csproj	Switch unit test TFM to `$(NetCoreAppTestTargetFramework)`.
src/tests/SOS.UnitTests/Debuggees/Directory.Build.targets	Reduce verbosity of skip message for interpreter debuggees.
src/tests/Microsoft.SymbolStore.UnitTests/Microsoft.SymbolStore.UnitTests.csproj	Switch unit test TFM to `$(NetCoreAppTestTargetFramework)`.
src/tests/Microsoft.FileFormats.UnitTests/Microsoft.FileFormats.UnitTests.csproj	Switch unit test TFM to `$(NetCoreAppTestTargetFramework)`.
src/tests/Microsoft.Diagnostics.Monitoring.EventPipe/EventLogsPipelineUnitTests.cs	Tighten “read all expected logs” assertion by reading remaining content.
src/tests/Microsoft.Diagnostics.DebugServices.UnitTests/Microsoft.Diagnostics.DebugServices.UnitTests.csproj	Switch unit test TFM to `$(NetCoreAppTestTargetFramework)`.
src/tests/dotnet-trace/DotnetTrace.UnitTests.csproj	Switch unit test TFM to `$(NetCoreAppTestTargetFramework)`.
src/tests/dotnet-counters/DotnetCounters.UnitTests.csproj	Switch unit test TFM to `$(NetCoreAppTestTargetFramework)`.
src/tests/dirs.proj	Add `SkipTestArtifactsBuild` gating to avoid rebuilding debuggees on test-only legs.
src/tests/DbgShim.UnitTests/DbgShim.UnitTests.csproj	Switch unit test TFM to `$(NetCoreAppTestTargetFramework)`.
global.json	Adjust installed runtimes list (removes x64 runtime entry; updates x86 runtime to `$(MicrosoftNETCoreApp100Version)`).
eng/TraversalOverrides.targets	New: overrides traversal `TestDependsOn` when `SkipTestArtifactsBuild=true` to prevent rebuilds.
eng/pipelines/build.yml	Add `publishTestArtifacts` support + test-only artifact download flow (issues noted in PR comments).
eng/build.sh	On `-test -skipmanaged`, install runtimes explicitly and set `SkipTestArtifactsBuild=true`.
eng/build.ps1	Same as build.sh for Windows: install runtimes explicitly and set `SkipTestArtifactsBuild`.
eng/AuxMsbuildFiles/SdkPackOverrides.targets	Reduce message verbosity for overridden runtime test versions.
Directory.Build.props	Add `NetCoreAppTestTargetFramework` and hook traversal override targets via `CustomAfterTraversalTargets`.
diagnostics.yml	Split into build stage (build-only + publish test artifacts) and test stage (test-only legs consuming artifacts).

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

hoyosjs · 2026-07-08T22:53:27Z

/ba-g test failure is marked as known

steveisok

LGTM

hoyosjs force-pushed the juhoyosa/separate-test-jobs branch from 1b595cb to e722ead Compare June 26, 2026 23:17

hoyosjs and others added 6 commits June 29, 2026 17:23

Refactor build pipeline to separate test jobs

31f61e6

Keep tests on net8

3b05a95

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

hoyosjs force-pushed the juhoyosa/separate-test-jobs branch from fb04a2a to 17e2280 Compare June 30, 2026 00:40

hoyosjs and others added 4 commits June 29, 2026 20:16

hoyosjs marked this pull request as ready for review June 30, 2026 23:15

hoyosjs requested a review from a team as a code owner June 30, 2026 23:15

Copilot AI review requested due to automatic review settings June 30, 2026 23:15

Copilot started reviewing on behalf of hoyosjs June 30, 2026 23:15 View session

Copilot AI reviewed Jun 30, 2026

View reviewed changes

Comment thread eng/pipelines/build.yml

Comment thread eng/pipelines/build.yml

Comment thread eng/pipelines/build.yml

Comment thread eng/pipelines/build.yml

Comment thread eng/pipelines/build.yml

build-analysis Bot mentioned this pull request Jul 1, 2026

[Tests] LogsPipelineUnitTests TestLogsPipeline fails with Assert.Equal() Failure #5659

Open

hoyosjs enabled auto-merge (squash) July 8, 2026 22:53

max-charlamb approved these changes Jul 9, 2026

View reviewed changes

Comment thread Directory.Build.props

steveisok approved these changes Jul 9, 2026

View reviewed changes

hoyosjs disabled auto-merge July 10, 2026 21:30

hoyosjs merged commit 01841e2 into dotnet:main Jul 10, 2026
23 of 25 checks passed

hoyosjs deleted the juhoyosa/separate-test-jobs branch July 10, 2026 21:32

dotnet-maestro Bot mentioned this pull request Jul 11, 2026

[main] Source code updates from dotnet/diagnostics dotnet/dotnet#7668

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Refactor build pipeline to separate test jobs#5875

Refactor build pipeline to separate test jobs#5875
hoyosjs merged 10 commits into
dotnet:mainfrom
hoyosjs:juhoyosa/separate-test-jobs

hoyosjs commented Jun 11, 2026 •

edited

Loading

Uh oh!

Copilot AI left a comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

hoyosjs commented Jul 8, 2026

Uh oh!

Uh oh!

steveisok left a comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

Uh oh!

Conversation

hoyosjs commented Jun 11, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

How this PR fixes it

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

hoyosjs commented Jul 8, 2026

Uh oh!

Uh oh!

steveisok left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

hoyosjs commented Jun 11, 2026 •

edited

Loading