Skip to content

[AIGTWY-4880] Add native budget and model evidence for CUJ5 - #915

Closed
andy-xu-db wants to merge 1 commit into
andy-xu-db/stack/andy/aigtwy-4880-budget-fixturefrom
andy-xu-db/stack/andy/aigtwy-4880-budget-evidence
Closed

andy-xu-db wants to merge 1 commit into
andy-xu-db/stack/andy/aigtwy-4880-budget-fixturefrom
andy-xu-db/stack/andy/aigtwy-4880-budget-evidence

Conversation

@andy-xu-db

@andy-xu-db andy-xu-db commented Sep 30, 2026 •

Copy link
Copy Markdown
Collaborator

🥞 Stacked PR


Summary

Add independent evidence helpers for CUJ5's budget-driven smart-default journeys. They verify the selected native-agent process, completed task request model, displayed budget values, and recommendation.

Changes

  • Compare the launch budget panel and ug usage with backend spend, limit, percentage, and meter values using their display rounding.
  • Identify real Claude and Codex processes without treating launcher arguments as native processes.
  • Correlate each completed file task with the request model recorded in its root native Claude or Codex session, requiring Codex's matching turn context to precede completion.

Configuration

The concrete CUJ5 target is declared by the workflow in PR #911. PR #913 forwards that metadata to pytest and keeps it out of native agent environments.

Verification

The complete stack passed 119 focused tests. Evidence cases cover terminal formatting, native process identification, root-session filtering, model spelling normalization, and rejection of missing, mismatched, or later Codex turn contexts. The seven-case live run verified completed-task model correlation for both native agents.

@andy-xu-db
andy-xu-db force-pushed the andy-xu-db/stack/andy/aigtwy-4880-budget-fixture branch from 2a43691 to 8ab0906 Compare September 30, 2026 20:16
@andy-xu-db
andy-xu-db force-pushed the andy-xu-db/stack/andy/aigtwy-4880-budget-evidence branch from 3d11500 to 3757625 Compare September 30, 2026 20:16
@andy-xu-db
andy-xu-db force-pushed the andy-xu-db/stack/andy/aigtwy-4880-budget-fixture branch from 8ab0906 to bae3286 Compare September 30, 2026 20:26
@andy-xu-db
andy-xu-db force-pushed the andy-xu-db/stack/andy/aigtwy-4880-budget-evidence branch from 3757625 to a9beae9 Compare September 30, 2026 20:26
@andy-xu-db
andy-xu-db force-pushed the andy-xu-db/stack/andy/aigtwy-4880-budget-fixture branch from bae3286 to 79a23dd Compare September 30, 2026 20:39
@andy-xu-db
andy-xu-db force-pushed the andy-xu-db/stack/andy/aigtwy-4880-budget-evidence branch from a9beae9 to edf1288 Compare September 30, 2026 20:39
@andy-xu-db
andy-xu-db force-pushed the andy-xu-db/stack/andy/aigtwy-4880-budget-fixture branch from 79a23dd to 7b6553e Compare September 30, 2026 20:49
@andy-xu-db
andy-xu-db force-pushed the andy-xu-db/stack/andy/aigtwy-4880-budget-evidence branch from edf1288 to caddcd4 Compare September 30, 2026 20:49
@andy-xu-db
andy-xu-db force-pushed the andy-xu-db/stack/andy/aigtwy-4880-budget-fixture branch from 7b6553e to ef0dfb4 Compare October 1, 2026 14:38
@andy-xu-db
andy-xu-db force-pushed the andy-xu-db/stack/andy/aigtwy-4880-budget-evidence branch 2 times, most recently from 4ff6751 to 989bf6d Compare October 1, 2026 15:05
@andy-xu-db
andy-xu-db force-pushed the andy-xu-db/stack/andy/aigtwy-4880-budget-fixture branch from ef0dfb4 to 3830958 Compare October 1, 2026 15:05
@andy-xu-db
andy-xu-db force-pushed the andy-xu-db/stack/andy/aigtwy-4880-budget-evidence branch from 989bf6d to bf245ba Compare October 1, 2026 15:09
@andy-xu-db
andy-xu-db force-pushed the andy-xu-db/stack/andy/aigtwy-4880-budget-fixture branch from 3830958 to b2c5c7c Compare October 1, 2026 15:09
@andy-xu-db
andy-xu-db force-pushed the andy-xu-db/stack/andy/aigtwy-4880-budget-evidence branch from bf245ba to bf3e3ac Compare October 1, 2026 16:24
@andy-xu-db
andy-xu-db force-pushed the andy-xu-db/stack/andy/aigtwy-4880-budget-fixture branch 2 times, most recently from 5f28dfc to 131af08 Compare October 1, 2026 16:29
@andy-xu-db
andy-xu-db force-pushed the andy-xu-db/stack/andy/aigtwy-4880-budget-evidence branch 2 times, most recently from d1505fc to 5e656bd Compare October 1, 2026 16:37
@andy-xu-db
andy-xu-db force-pushed the andy-xu-db/stack/andy/aigtwy-4880-budget-fixture branch from 131af08 to 5a8abeb Compare October 1, 2026 16:37
@andy-xu-db
andy-xu-db force-pushed the andy-xu-db/stack/andy/aigtwy-4880-budget-evidence branch from 5e656bd to 715d107 Compare October 1, 2026 17:17
@andy-xu-db
andy-xu-db force-pushed the andy-xu-db/stack/andy/aigtwy-4880-budget-fixture branch from 5a8abeb to 9e8519f Compare October 1, 2026 17:17
@andy-xu-db
andy-xu-db force-pushed the andy-xu-db/stack/andy/aigtwy-4880-budget-evidence branch from 715d107 to b890d33 Compare October 1, 2026 18:51
@andy-xu-db
andy-xu-db force-pushed the andy-xu-db/stack/andy/aigtwy-4880-budget-fixture branch from 9e8519f to 3ffdb7f Compare October 1, 2026 18:51
@andy-xu-db andy-xu-db changed the title [AIGTWY-4880] Add native budget and model evidence helpers [AIGTWY-4880] Add native budget and model evidence for CUJ5 Oct 1, 2026
@andy-xu-db
andy-xu-db force-pushed the andy-xu-db/stack/andy/aigtwy-4880-budget-fixture branch from 3ffdb7f to c7ccaa6 Compare October 1, 2026 18:56
@andy-xu-db
andy-xu-db force-pushed the andy-xu-db/stack/andy/aigtwy-4880-budget-evidence branch from b890d33 to d0e1889 Compare October 1, 2026 18:56
@andy-xu-db
andy-xu-db force-pushed the andy-xu-db/stack/andy/aigtwy-4880-budget-fixture branch from d320e1d to 07a1c94 Compare October 1, 2026 20:24
@andy-xu-db
andy-xu-db force-pushed the andy-xu-db/stack/andy/aigtwy-4880-budget-evidence branch from b9ad28e to 2546e7c Compare October 1, 2026 20:24
@andy-xu-db
andy-xu-db force-pushed the andy-xu-db/stack/andy/aigtwy-4880-budget-fixture branch from 07a1c94 to b47bd5b Compare October 1, 2026 20:29
@andy-xu-db
andy-xu-db force-pushed the andy-xu-db/stack/andy/aigtwy-4880-budget-evidence branch from 2546e7c to 568d966 Compare October 1, 2026 20:29
@andy-xu-db
andy-xu-db force-pushed the andy-xu-db/stack/andy/aigtwy-4880-budget-fixture branch from b47bd5b to b3c3946 Compare October 1, 2026 20:33
@andy-xu-db
andy-xu-db force-pushed the andy-xu-db/stack/andy/aigtwy-4880-budget-evidence branch 2 times, most recently from f15cef7 to fb45b7f Compare October 1, 2026 20:38
@andy-xu-db
andy-xu-db force-pushed the andy-xu-db/stack/andy/aigtwy-4880-budget-fixture branch from b3c3946 to ff50a7b Compare October 1, 2026 20:38
@andy-xu-db
andy-xu-db force-pushed the andy-xu-db/stack/andy/aigtwy-4880-budget-evidence branch from fb45b7f to 34c8692 Compare October 1, 2026 20:43
@andy-xu-db
andy-xu-db force-pushed the andy-xu-db/stack/andy/aigtwy-4880-budget-fixture branch from ff50a7b to bf72d55 Compare October 1, 2026 20:43
@andy-xu-db
andy-xu-db force-pushed the andy-xu-db/stack/andy/aigtwy-4880-budget-evidence branch from 34c8692 to 5df19cb Compare October 1, 2026 20:47
@andy-xu-db
andy-xu-db force-pushed the andy-xu-db/stack/andy/aigtwy-4880-budget-fixture branch from bf72d55 to fa66168 Compare October 1, 2026 20:47
@andy-xu-db
andy-xu-db force-pushed the andy-xu-db/stack/andy/aigtwy-4880-budget-evidence branch from 5df19cb to 6a96c46 Compare October 1, 2026 20:57
@andy-xu-db
andy-xu-db force-pushed the andy-xu-db/stack/andy/aigtwy-4880-budget-fixture branch 2 times, most recently from b07e258 to 6eaabe9 Compare October 2, 2026 13:36
@andy-xu-db
andy-xu-db force-pushed the andy-xu-db/stack/andy/aigtwy-4880-budget-evidence branch 4 times, most recently from 158b51f to 0fcc784 Compare October 2, 2026 14:04
@andy-xu-db
andy-xu-db force-pushed the andy-xu-db/stack/andy/aigtwy-4880-budget-fixture branch from 6eaabe9 to cf56ee8 Compare October 2, 2026 14:26
@andy-xu-db
andy-xu-db force-pushed the andy-xu-db/stack/andy/aigtwy-4880-budget-evidence branch 2 times, most recently from 98540a1 to 6126184 Compare October 2, 2026 15:42
@andy-xu-db
andy-xu-db force-pushed the andy-xu-db/stack/andy/aigtwy-4880-budget-fixture branch 2 times, most recently from 56fa5f9 to c848559 Compare October 2, 2026 16:49
@andy-xu-db
andy-xu-db force-pushed the andy-xu-db/stack/andy/aigtwy-4880-budget-evidence branch from 6126184 to 24833d7 Compare October 2, 2026 16:49
@andy-xu-db
andy-xu-db force-pushed the andy-xu-db/stack/andy/aigtwy-4880-budget-evidence branch from 24833d7 to 3ecec0e Compare October 2, 2026 17:03
@andy-xu-db
andy-xu-db force-pushed the andy-xu-db/stack/andy/aigtwy-4880-budget-fixture branch from c848559 to 6c4f77a Compare October 2, 2026 17:03
@andy-xu-db

Copy link
Copy Markdown
Collaborator Author

Consolidated into #911, which now contains the complete CUJ5 integration test change as one commit against main.

@andy-xu-db andy-xu-db closed this Oct 2, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant