Skip to content

ci: update release docs during code freeze - #681

Open
balasaajay wants to merge 1 commit into
NVIDIA-NeMo:mainfrom
balasaajay:ci/code-freeze-release-docs
Open

balasaajay wants to merge 1 commit into
NVIDIA-NeMo:mainfrom
balasaajay:ci/code-freeze-release-docs

Conversation

@balasaajay

@balasaajay balasaajay commented Oct 27, 2026 •

Copy link
Copy Markdown
Contributor

Summary

Validation

  • actionlint passed for the caller and shared workflow.
  • Shared updater tests passed (6 tests), including retry idempotence and Bridge label cleanup.

Dependency: merge NVIDIA-NeMo/FW-CI-templates#609 before this caller PR. The current release docs are covered by #680.

@copy-pr-bot

copy-pr-bot Bot commented Oct 27, 2026

Copy link
Copy Markdown

Auto-sync is disabled for draft pull requests in this repository. Workflows must be run manually.

Contributors can view more details about this message here.

@balasaajay
balasaajay marked this pull request as ready for review October 27, 2026 22:56
@balasaajay

Copy link
Copy Markdown
Contributor Author

/ok to test 43f1af9

@balasaajay

Copy link
Copy Markdown
Contributor Author

/review-approve

Copy link
Copy Markdown
Contributor

⚠️ balasaajay is not in any eligible team (run_team, run_maintainers, automation) — you don't have permission to trigger formal reviews. Posting review as comment only.

Copy link
Copy Markdown
Contributor

Advisory review — 1 finding
PR: #681
Head: 43f1af9050d210e385643f25d18d92db503cf674

P2 — Preserve the prior release’s picker entry across freezes
.github/workflows/release-freeze.yml:80–82
The new job derives the picker solely from the newly frozen branch’s docs/versions1.json, but updates that file only on the release branch. Main still lists 0.11.0 as latest, while r0.12.0 contains the newer 0.12.0 entry. Running the exact added expression for the next freeze (0.13.0, matching main’s package version) produces:
nightly, 0.13.0, 0.11.0, 0.10.0, 0.9.0, 0.8.0, 0.5.0
0.12.0 is omitted. The release publisher copies this generated picker over the shared parent versions1.json, so publishing the next release would remove the previous release from the picker.
Correction: Maintain a canonical picker inventory on main, or merge the previous release’s inventory before applying this transformation. Keep main’s Sphinx/project version as nightly.
Acceptance check: Freeze 0.13.0 using the current inventories; verify that 0.12.0 remains at its numbered URL, only 0.13.0 is preferred, and a second run changes nothing.

Checks: Three release checks are failing. In run 40598543397, attempt 1, the wheel files built successfully, but the subsequent package import failed at nemo_run/core/execution/lepton.py:30:
ModuleNotFoundError: No module named 'leptonai.api.v1'
The log installed leptonai==0.47.1; the unchanged dependency declaration permits it. publish-wheel then failed because its wheel artifact was absent, and release-summary reported those failures. These import/dependency paths are unchanged from current main, so they are not findings against this PR. Unit tests, installation matrix, lint, formatting, and CodeQL passed.

Coverage and limits: Inspected the complete diff, surrounding source, pinned workflow dependencies, publishing consumers, tests, and existing feedback. Completed the publication/check inspection directly after one delegated inspection ended without an assessment. No live freeze or publication was executed; standalone jq was unavailable, so expression checks used gh --jq. Branch-ruleset inspection was denied, and the required prose-guide file was absent. No GitHub approval, comment, or formal review was submitted.

Copy link
Copy Markdown
Contributor

The workflow published the advisory review on PR #681. The reviewed head is unchanged, and no formal approval was submitted. Review work is complete; no further action is pending.

@balasaajay

Copy link
Copy Markdown
Contributor Author

/ok to test 749dc4b

@balasaajay

Copy link
Copy Markdown
Contributor Author

/review-approve

Copy link
Copy Markdown
Contributor

Follow-up advisory review — 1 unresolved P2; no additional eligible findings
PR: #681
Head: 749dc4bcc0b7aced589511ea700bee99e94e0d3e

P2 — Preserve the previous release in the version picker
.github/workflows/release-freeze.yml:38 — shared updater enabled by this line.
Moving the update into the shared workflow does not resolve the inventory loss. The pinned updater reads only the newly frozen branch’s docs/versions1.json, inserts the new version, and commits only to that release branch. It neither imports the previous release’s inventory nor updates main’s inventory.

The current head and freshly fetched main still omit 0.12.0; r0.12.0 contains it. For the next freeze, the updater therefore produces:
nightly, 0.13.0, 0.11.0, 0.10.0, 0.9.0, 0.8.0, 0.5.0
The publisher replaces the shared parent picker with this file, removing 0.12.0 from the selector. Evidence is in the pinned _code_freeze.yml:299–322 and publisher action.yml:251–258.

Correction: Keep a canonical picker inventory on main, or merge the previous release’s inventory before applying the shared updater. Keep main’s Sphinx/project version as nightly.
Acceptance check: Starting from the current main and r0.12.0 inventories, verify that freezing 0.13.0 retains 0.12.0 at its numbered URL, has exactly one preferred entry, and is unchanged on rerun. The added shared tests cover preservation of entries already present, not this cross-branch inventory gap.

Dependency: FW-CI-templates#609 remains open at the exact pinned commit. Merge that dependency before this caller PR, as its description specifies.

Current CI: Run 40602052040, attempt 1, failed after successfully building the wheel. Its subsequent package import reports:

ModuleNotFoundError: No module named 'leptonai.api.v1'```
The run installed `leptonai==0.47.1`; `publish-wheel` then failed on the missing artifact, and `release-summary` reported the failures. These imports and dependency declarations are unchanged from current main, so this is separate from the code-freeze change. Unit tests, installation matrix, CodeQL, lint, and formatting passed; no checks remain pending.

*Limits:* Follow-up inspection covered the changed caller, pinned updater and tests, execution contract, and publishing consumers. No live freeze/publication was executed. Shared tests were read, not run; the workstation has no `python3`. No formal approval was submitted.

@svcnvidia-nemo-ci svcnvidia-nemo-ci left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Advisory review: no merge-blocking defect was independently validated from the mounted source.

Inspected the complete freeze caller change against base and traced local version metadata and publishing consumers. The configured documentation root matches the existing picker URLs and publication target. Nightly publication continues to disable picker updates.

The earlier picker-loss report is not verified fixed. The current inventory still omits 0.12.0, while the package version is 0.13.0. However, the pinned shared updater, publisher implementation, shared regression tests, and r0.12.0 tree are not mounted. Consequently, the reported transformation and publication outcome could not be independently confirmed; historical comments alone do not establish them.

Limitations: this completes the available-source inspection, not validation of the external dependency. Its merge status and reported test results were not verified, and no tests or workflows were executed. Two delegated assessments were unavailable; their caller-contract and publication-routing topics were inspected directly.

Review generated by codex. This was a full review. Request a full review with /review reset=true.

@balasaajay
balasaajay force-pushed the ci/code-freeze-release-docs branch from 749dc4b to f4835f9 Compare October 28, 2026 02:05
@balasaajay

Copy link
Copy Markdown
Contributor Author

/ok to test f4835f9

@balasaajay

Copy link
Copy Markdown
Contributor Author

/review-approve

@balasaajay
balasaajay force-pushed the ci/code-freeze-release-docs branch from f4835f9 to 8ff3345 Compare October 28, 2026 02:06
@balasaajay

Copy link
Copy Markdown
Contributor Author

/ok to test 8ff3345

@balasaajay

Copy link
Copy Markdown
Contributor Author

/review-approve

Copy link
Copy Markdown
Contributor

💬 A replacement inspection workstation has been allocated. The follow-up review will resume when it is ready; the latest synchronized PR head has not yet been inspected.

@svcnvidia-nemo-ci svcnvidia-nemo-ci left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Follow-up review complete: no in-scope merge-blocking defect was independently validated.

Inspected the freeze caller against the mounted base and traced documentation metadata and publication consumers. The configured documentation root matches the existing picker URLs and publication path. Nightly publication continues to disable picker updates.

The earlier picker-loss concern is not verified fixed. The local inventory still omits 0.12.0, but the external updater, publisher implementation, shared regression tests, and prior-release inventory are not mounted. Those local facts alone do not establish the alleged transformation or publication failure.

Limitations: the previously reviewed tree is unavailable, so this follow-up cannot independently establish an inter-review regression. The pinned dependency’s input contract, implementation, and merge status were not verified. All three delegated topics were completed and consolidated; no tests or workflows were executed.

Review generated by codex. This was a findings-only review. Request a full review with /review reset=true.

Copy link
Copy Markdown
Contributor

Follow-up advisory review — 1 unresolved P2; no additional eligible findings
PR: #681
Reviewed head: 8ff3345b50c18bc012d74f6b2b372986d8dea574

P2 — Preserve the previous release in the version picker
.github/workflows/release-freeze.yml:38
The new shared-workflow pin still does not preserve entries that exist only on the preceding release branch. Its updater reads the freeze checkout’s docs/versions1.json, adds the new version, and commits only to the release branch (_code_freeze.yml@bea3a002…:241–279).

Current main and this PR head omit 0.12.0, while r0.12.0 includes it. With the current 0.13.0 package version, the next freeze would therefore generate:
nightly, 0.13.0, 0.11.0, 0.10.0, 0.9.0, 0.8.0, 0.5.0
The publisher copies that file over the shared parent picker (publish-docs/action.yml@v0.94.1:251–258), so publishing the next release would remove 0.12.0 from the selector. The merged docs PR #680 targets r0.12.0; it does not repair main’s inventory.

Fix: Maintain the canonical picker inventory on main, or merge the previous release’s inventory before updating it. Keep main’s Sphinx/project version as nightly.
Acceptance check: Freeze 0.13.0 from the current inventories; assert that 0.12.0 remains at its numbered URL, exactly one entry is preferred, and rerunning leaves the files unchanged. The shared tests cover entries already present, not this cross-branch gap.

Pin changes: Moving the docs steps into branch creation fixes their ordering relative to the dependent jobs, and the updater now overwrites stale display names. Neither change addresses the missing inventory. FW-CI-templates#609 remains open at the pinned commit; its stated merge dependency remains outstanding.

Current CI: Run 40619017922, attempt 1, failed after building the wheel and source archive. The subsequent import failed at nemo_run/core/execution/lepton.py:30:
ModuleNotFoundError: No module named 'leptonai.api.v1'
The run selected leptonai==0.47.1. publish-wheel then lacked its artifact, and release-summary reported those failures. The failing source and dependency declaration are unchanged from freshly fetched main, so this is not introduced by this PR. Unit tests, installation matrix, CodeQL, lint, and formatting passed; no checks remain pending.

Validation limits: Inspected the complete caller diff, exact shared pin, tests, publishing consumers, feedback, and current failure logs. No tests, freeze, or publication were executed. No formal approval was submitted.

Signed-off-by: Ajay Balasa <abalasa@nvidia.com>
@balasaajay

Copy link
Copy Markdown
Contributor Author

/ok to test 3829434

@balasaajay

Copy link
Copy Markdown
Contributor Author

/review-approve

@svcnvidia-nemo-ci svcnvidia-nemo-ci left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Follow-up review complete: no eligible merge-blocking defect was independently validated.

Inspected the freeze caller, including the updated shared-workflow pin and documentation opt-in. Dry-run and release input forwarding remain intact. Traced documentation metadata and publication consumers; nightly publication continues to disable picker updates.

The earlier picker-loss concern is not verified fixed. The local inventory still omits 0.12.0, but the pinned updater, external publisher implementation, shared regression tests, and preceding release inventory are unavailable. The local omission alone does not establish the reported freeze-and-publication failure.

Limitations: the previous reviewed tree is unavailable, preventing an independent inter-review source comparison. The shared dependency’s input contract, implementation, and merge status were not verified. All three delegated inspections were completed and consolidated; no tests or workflows were executed.

Review generated by codex. This was a findings-only review. Request a full review with /review reset=true.

Copy link
Copy Markdown
Contributor

Follow-up advisory review — 1 unresolved P2; no additional eligible findings
PR: #681
Head: 38294343112bf7a94e26980bd7823b33adfb26c4

P2 — Preserve the previous release in the version picker
.github/workflows/release-freeze.yml:38
The new update-release-docs: true interface correctly enables the updater, but the pinned implementation still reads only the freeze checkout’s inventory (_code_freeze.yml@9804c761…:220–257). It does not merge entries from the preceding release or maintain main’s inventory.

Freshly fetched main and this head omit 0.12.0; r0.12.0 includes it. For the next 0.13.0 freeze, the updater would therefore produce:
nightly, 0.13.0, 0.11.0, 0.10.0, 0.9.0, 0.8.0, 0.5.0
The release publisher copies this file over the shared parent picker (publish-docs/action.yml@v0.94.1:251–258), so publishing the next release would remove 0.12.0 from the selector.

Fix: Maintain a canonical picker inventory on main, or merge the previous release’s inventory before transforming it. Keep main’s Sphinx/project version as nightly.
Acceptance check: Freeze 0.13.0 from the current inventories; verify that 0.12.0 remains at its numbered URL, exactly one entry is preferred, and a second update changes nothing. The shared tests preserve entries already present but do not cover this cross-branch gap.

Interface change: Inferring the root and latest-URL convention from Run’s existing manifest is compatible with its publishing configuration. Boolean typing, dry-run guards, environment bindings, working directories, and failure propagation have no additional validated defect. FW-CI-templates#609 remains open at the pinned commit; its stated merge dependency is outstanding.

Current CI: Run 40626100840, attempt 1, failed after building the wheel and source archive. The subsequent import failed at nemo_run/core/execution/lepton.py:30:
ModuleNotFoundError: No module named 'leptonai.api.v1'
The run selected leptonai==0.47.1; publish-wheel then lacked its artifact, and release-summary reported the failures. The relevant source and dependency declarations are unchanged from current main, so this is separate from the code-freeze change. Unit tests, installation matrix, CodeQL, lint, and formatting passed; no checks remain pending.

Limits: Inspected the complete caller delta, pinned updater and tests, execution contract, publishing consumers, feedback, and current failure logs. No tests, freeze, or publication were executed. No formal approval was submitted.

This branch had an error being deployed

1 failed deployment
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants