test(agent): cover policy management end to end - #1981
Benoît Cortier (CBenoit) wants to merge 13 commits into
Conversation
|
Implementation notes:
Note Human-tuned, LLM-assisted content. |
There was a problem hiding this comment.
🟡 Changes recommended
The standard-user scenario runs both client and Agent under the same restricted identity, leaving the production cross-identity authorization boundary untested.
Once you've addressed the issues Copilot identified, you can request another Copilot review.
Pull request overview
Adds end-to-end Agent policy-management coverage across restricted and LocalSystem execution modes.
Changes:
- Expands lifecycle, authorization, persistence, and audit tests.
- Adds protected unelevated test orchestration.
- Runs identity and feature-gated route tests in Windows CI.
Documentation-specific review was not applicable.
File summaries
| File | Description |
|---|---|
crates/agent-policy-tester/src/windows.rs |
Adds identity modes and policy lifecycle scenarios. |
crates/agent-policy-tester/run-unelevated.ps1 |
Runs the tester with a restricted token. |
crates/agent-policy-tester/run-as-system.ps1 |
Selects elevated test mode. |
crates/agent-policy-tester/Cargo.toml |
Adds Windows identity dependencies. |
Cargo.lock |
Records dependency updates. |
.github/workflows/ci.yml |
Runs both identities and route tests. |
Review details
- Files reviewed: 5/6 changed files
- Comments generated: 1
- Review effort level: Balanced
💡 Configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
There was a problem hiding this comment.
🟡 Changes recommended
The psexec -l client runs at Low integrity and may be denied by the named pipe before testing the intended authorization response.
Once you've addressed the issues Copilot identified, you can request another Copilot review.
Review details
- Files reviewed: 5/6 changed files
- Comments generated: 1
- Review effort level: Balanced
There was a problem hiding this comment.
🟡 Changes recommended
Failed readiness validation can leave LocalSystem test processes running into later CI steps.
Once you've addressed the issues Copilot identified, you can request another Copilot review.
Review details
- Files reviewed: 5/6 changed files
- Comments generated: 1
- Review effort level: Balanced
3867fa3 to
3cce637
Compare
911345a to
0e6d7c5
Compare
Exercise policy management through a protected standard-user client and a LocalSystem client without weakening executable or path checks. Cover validation and write denial, managed Create, Update, Repair, stale conflicts, confirmed overwrite, restart persistence, durable managed authority, and observable audit outcomes. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Run the protected policy tester under a restricted standard-user token before the existing LocalSystem lifecycle. Keep feature-gated route authorization tests in the same Windows job so the development signature feature cannot hide them from the default workspace suite. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Keep the Agent and test server under LocalSystem while a distinct restricted process exercises the named-pipe management endpoints. Coordinate readiness and shutdown through a protected, read-only test directory so authorization regressions cannot pass by inspecting the server token. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Treat PsExec's detached-process PID as diagnostic output and use bounded, validated readiness as the authoritative launch result. Publish readiness atomically, preserve launch diagnostics, and keep shutdown and cleanup idempotent so orchestration errors remain actionable. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Launch policy authorization requests from a unique temporary standard-user account instead of a Low-integrity PsExec token. Require the exact account SID and Medium mandatory integrity level, keep credentials out of arguments and logs, and remove the account and profile after the bounded run. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Own LocalSystem test-server shutdown as soon as detached launch is attempted, even when readiness validation fails. Signal the unique protected stop marker with a direct fallback, preserve the original scenario error, and bound status collection before cleanup. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Cover the current policy contract, receipt invalidation, warnings, and interrupted Repair recovery. Exercise actual installer conversion and managed authority under the hosted LocalSystem gate without accepting skipped privileged tests or incomplete server completion status. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Pass the restored package root explicitly to the LocalSystem test invocation so generated imports discover the test SDK and produce its required privileged-test result. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Preserve the restored NuGet imports and the product Agent filename when running the privileged installer lifecycle tests. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Run migration lifecycle probes through the staged trusted test client instead of the hosted .NET test process, whose executable image cannot be retained by the package broker. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Remove installer migration and retired-policy compatibility coverage while preserving the canonical policy management lifecycle and its security boundaries. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Exercise the final canonical policy rule shapes and validation semantics through the end-to-end policy management harness. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
0e6d7c5 to
486c4ea
Compare
There was a problem hiding this comment.
🔵 Needs a closer look
Three moderate test-coverage gaps must be addressed before approval.
Review details
Suppressed comments (3)
crates/agent-policy-tester/run-unelevated.ps1:350
- The new self-test does not exercise the cleanup guarantee it claims to cover.
$launchAttemptedis assignedtrueimmediately before eachWait-ServerReadinesscall and is only tested against itself; no code invokes the surroundingfinallyor verifies thatSignal/status cleanup runs after failed readiness. A regression that removes or relocates$serverLaunchAttempted = $truecould therefore passSelfTest. Refactor the orchestration into a testable function with a fake launcher, or add an integration assertion that failed readiness still signals and observes server shutdown.
$launchAttempted = $true
Set-Content -LiteralPath $ready -Value (
@{
Nonce = "wrong-nonce"
PipeName = "\\.\pipe\test"
ServerPid = 100
ServerSid = "S-1-5-18"
AgentPid = 200
AgentSid = "S-1-5-18"
} | ConvertTo-Json -Compress
)
try {
Wait-ServerReadiness -Path $ready -ServerStatusPath $status -ExpectedNonce "nonce" `
-TimeoutMilliseconds 50 -LaunchValue 6256 -LaunchDiagnostics "started detached process" | Out-Null
throw "Mismatched-readiness simulation unexpectedly succeeded"
} catch {
if ($_ -notmatch "readiness nonce mismatch" -or -not $launchAttempted) {
crates/agent-policy-tester/src/windows.rs:921
- The standard-user leg only exercises
/v1/policy/managementand/v1/policy/validate; it never callsGET /v1/policy, even though the policy-management contract keeps reads available to an authenticated standard user. A regression that makes active-policy reads administrator-only would therefore pass this scenario. Add a GET assertion here (the policy is missing in this harness, so expect HTTP 404).
let management = policy_management_by_pipe(pipe_name).await?;
ensure!(management["State"] == "Missing", "expected a missing policy");
crates/agent-policy-tester/src/windows.rs:1129
- This
ConfirmOverwriterequest usescurrent_tokenfrom the stale-conflict response, so the token is already fresh. It never verifies the contract that a stale token remains rejected whenConflictHandlingisConfirmOverwrite; a regression that turns this mode into an unconditional stale-token bypass would still pass. Preservestale_token, submit it withConfirmOverwrite, and assert409/StalePolicyStoreTokenbefore retrying withcurrent_token.
let confirmed = replace_policy_response(
&agent,
"Update",
"ConfirmOverwrite",
current_token.clone(),
- Files reviewed: 5/6 changed files
- Comments generated: 0 new
- Review effort level: Lite (auto)
Note
Copilot is running an experiment and ran this review at Lite.
Exercise standard-user read access, stale overwrite rejection, and LocalSystem shutdown verification after failed readiness. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
|
Addressed the exact-head Copilot review coverage gaps in
No inline review threads were created by the review to resolve. Note Human-tuned, LLM-assisted content. |
There was a problem hiding this comment.
🔵 Needs a closer look
The extensive multi-process Windows identity orchestration and security-boundary coverage require final human validation.
Review details
- Files reviewed: 5/6 changed files
- Comments generated: 0 new
- Review effort level: Lite (auto)
Note
Copilot is running an experiment and ran this review at Lite.
Exercises Agent policy management through protected standard-user and LocalSystem clients without weakening executable, ACL, path, or process-identity checks.
Covers read and validation behavior, standard-user write denial, Create, Update, Repair, stale conflict rejection, exact ConfirmOverwrite, restart persistence, durable managed authority, and observable audit outcomes. The Windows CI job runs both identities and the feature-gated route authorization tests.
Local PsExec execution could not install PSEXESVC, so the identity scenarios are delegated to CI; local builds, workspace tests, Clippy, formatting, runner parsing, and range validation pass.
Extracted from the production-shaped E2E support in #1963.