[AIGTWY-4876] Test managed config CUJ1

[JTSIV1]

Add CUJ1 tests that configure Claude and Codex from one published policy and verify their model catalogs, defaults, inference headers, and traces. Run six agent tasks to check the selected models and follow-up requests.

[github-actions]

UG review

The visible changes add live managed Claude/Codex configuration, inference, and trace checks plus shared TUI boot handling. The request-count assertion does not establish that tool-result follow-ups were exercised.

Major

• Assert actual tool-result follow-ups — tests/e2e_cuj/test_ug_managed_sentinel.py:95: can we assert a marked tool_result/function_call_output follow-up? retries or auxiliary POSTs also satisfy this count without verifying tool-result forwarding.

Some patches were unavailable or truncated to fit the review context. Treat this as a partial review.

Automated advisory review of cae93eac2763 using Lilly's UG review rubric. It does not approve or block this PR.

[lilly-luo]

can you ensure Test Claude's Opus, Sonnet, and Haiku aliases each complete a task on the matching actual model is captured?

[david-siqi-liu]

Reviewed against main's tests/e2e_cuj/AGENTS.md (#1000) and the CUJ 1 plan in the UG Configure E2E doc. Lilly's earlier comments about splitting the single test still apply and aren't repeated here.

P0

1. tests/integration/utils/terminal.py ~317: the "Select model" render wait now runs only when model_visible is None, so a caller's callback replaces the gate instead of adding to it. test_ug_claude_model_discovery.py cases 07 and 09 (lines 106, 124) pass lambda _text: session.claude_gateway_cache_ready(), which checks only the disk cache, so when the cache is already warm the screen is captured right after /model, before the picker renders. Keeping the render wait and adding the callback as a second condition avoids the flake.
2. Built on the #971 scaffold with its own tests/e2e_cuj/conftest.py (session / live_session, sys.path imports). main's #1000 conftest defines cuj and requires --confcutdir=tests/e2e_cuj, so the two conftests conflict (add/add), and keeping either one breaks the other's tests. The rebase means moving onto cuj and the package-relative helpers. That also brings main's MANAGED_PATHS preflight, which this conftest lacks; the teardown runs a terminal ug revert but never checks that the machine-wide files are gone.
3. test_ug_managed_sentinel.py ~87: _assert_inference_requests needs at least two marked HTTP 200 requests with payload["model"] == expected, and FileTask.assert_completed checks only that some answer contains the file value. Neither ties the model to the native completed turn, so a final turn on a different model still passes. SessionEvidence(...).completed(task) plus canonical_model on the turn models closes it.

P1

• Coverage against the CUJ 1 plan: "each agent's model picker shows exactly the configured models" is checked as "every configured model is on screen" in the Claude TUI (other rows aren't rejected), with exactness only from the settings file. "Neither agent ever sends the other's agent header value" is implied by each request matching its own value, with no explicit check.

No E2E CUJs job has run on this stack yet (#1004 -> #991 -> #992), so none of this has been exercised live.

[david-siqi-liu]

Correction: I rechecked against a stale local copy of this branch earlier. At 99097bd and now cae93ea, P0 2 and P0 3 are fixed: the stack uses main's cuj conftest with package-relative helpers, and _assert_native_model ties the expected model to the native completed turn via SessionEvidence and canonical_model. The picker render gate (P0 1) is fixed too, and #1004 merges cleanly onto current main. No remaining blockers from my side.

[david-siqi-liu]

No remaining blockers: the picker render gate, the move onto main's cuj scaffold, and the native completed-turn model checks are all in.