Skip to content

feat(builtins): add aspect ci warming task - #1063

Closed
gregmagolan wants to merge 3 commits into
mainfrom
aspect_ci_warming_task
Closed

feat(builtins): add aspect ci warming task#1063
gregmagolan wants to merge 3 commits into
mainfrom
aspect_ci_warming_task

Conversation

@gregmagolan

@gregmagolan gregmagolan commented May 5, 2026

Copy link
Copy Markdown
Member

Summary

Adds a new built-in AXL task: aspect ci warming.

  1. Cleans ${ASPECT__STORAGE_PATH}/{bazel,output,caches/repository} (matching rosetta's WarmingTaskRef cleanup).
  2. Runs bazel build --nobuild against the given targets so the repository, output, and bazel caches under the ephemeral mount are populated.
  3. When ASPECT_WORKFLOWS_RUNNER is set, also invokes /etc/aspect/workflows/bin/warming_archive to upload the populated caches to the warming bucket. Off-runner this step is skipped with a stderr warning; the populate step still runs so the task is useful for local testing.

This collapses the customer-facing GHA warming workflow to:

jobs:
    warming-archive:
        name: Aspect Workflows Warming
        runs-on: [self-hosted, aspect-workflows, aspect-default]
        steps:
            - uses: actions/checkout@v4
            - name: Warming Task
              run: aspect ci warming

(Replacing the prior 5-step flow that called configure_workflows_env, agent_health_check, rosetta run warming, and warming_archive separately.)

  • Default targets //...; positional args override.
  • Honors BazelTrait flags / startup_flags contributions so warming sees the same --config=... etc. as real builds.
  • Cleanup is no-op when ASPECT__STORAGE_PATH is unset.

This is the new-task half of the warming-refactor plan. Sibling PRs:

The three pieces are runtime-coupled but compile-independent. Either PR here can land first.

Test plan

Tested locally on macOS aarch64.

  • cargo build -p aspect-cli — clean
  • aspect ci warming --help — CLI surface exposes the task under group ci
  • aspect ci warming //... in a Bazel example workspace — cleanup is no-op (no ASPECT__STORAGE_PATH), bazel build --nobuild runs, exits 0 with 0 total actions, then prints the ASPECT_WORKFLOWS_RUNNER is not set; skipping the warming archive upload step. warning to stderr
  • Pre-seeded ASPECT__STORAGE_PATH=/tmp/warmtest with bazel/, output/, caches/repository/ subdirs and a sentinel file → after aspect ci warming, all three subdirs are removed
  • ASPECT_WORKFLOWS_RUNNER=1 ASPECT_WORKFLOWS_RUNNER_STORAGE_PATH=/tmp/warmtest aspect ci warming //... — task progresses past the bazel populate step and attempts to spawn /etc/aspect/workflows/bin/warming_archive (fails on a laptop because the binary isn't there, which confirms the call site is wired)
  • End-to-end on a staging Workflows runner — depends on feat: route launcher and AXL caches via env vars #1062 + the silo PR

🤖 Generated with Claude Code

@aspect-workflows

aspect-workflows Bot commented May 5, 2026

Copy link
Copy Markdown

✨ Aspect Workflows Tasks

📅 Mon May 25 23:08:52 UTC 2026

❌ 2 failed tasks

  • ❌ buildifier (buildifier-gha-debug) · ⏱ 24.9s · ✨ Aspect · 🐙 GitHub Actions · ☑️ Check
    💬 failed in diff · Format failed (1 file needs format)
  • ❌ buildifier (buildifier-gha-ephemeral) · ⏱ 55.8s · 🐙 GitHub Actions · ☑️ Check
    💬 failed in diff · Format failed (1 file needs format)

⚠️ 2 flagged tasks

  • ⚠️ delivery (delivery-gha) · ⏱ 19.3s · ✨ Aspect · 🐙 GitHub Actions · ☑️ Check
    💬 Delivery complete (2 warn · 1 skipped)
  • ⚠️ delivery (delivery-gha-debug) · ⏱ 38.1s · ✨ Aspect · 🐙 GitHub Actions · ☑️ Check
    💬 Delivery complete (2 warn · 1 skipped)

✅ 14 successful tasks

  • ✅ build (build-gha) · ⏱ 32.2s · ✨ Aspect · 🐙 GitHub Actions · ☑️ Check
    💬 Bazel build complete (158 built)
  • ✅ build (build-gha-debug) · ⏱ 3m 4s · ✨ Aspect · 🐙 GitHub Actions · ☑️ Check
    💬 Bazel build complete (158 built)
  • ✅ format (format-gha) · ⏱ 15.7s · ✨ Aspect · 🐙 GitHub Actions · ☑️ Check
    💬 Format complete (clean)
  • ✅ format (format-gha-debug) · ⏱ 34.8s · ✨ Aspect · 🐙 GitHub Actions · ☑️ Check
    💬 Format complete (clean)
  • ✅ format (format-gha-ephemeral) · ⏱ 2m 26s · 🐙 GitHub Actions · ☑️ Check
    💬 Format complete (clean)
  • ✅ gazelle (gazelle-from-source-gha) · ⏱ 18.7s · ✨ Aspect · 🐙 GitHub Actions · ☑️ Check
    💬 Gazelle complete (clean)
  • ✅ gazelle (gazelle-from-source-gha-debug) · ⏱ 46.4s · ✨ Aspect · 🐙 GitHub Actions · ☑️ Check
    💬 Gazelle complete (clean)
  • ✅ gazelle (gazelle-gha) · ⏱ 16.9s · ✨ Aspect · 🐙 GitHub Actions · ☑️ Check
    💬 Gazelle complete (clean)
  • ✅ gazelle (gazelle-gha-debug) · ⏱ 20.7s · ✨ Aspect · 🐙 GitHub Actions · ☑️ Check
    💬 Gazelle complete (clean)
  • ✅ gazelle (gazelle-gha-ephemeral) · ⏱ 57.7s · 🐙 GitHub Actions · ☑️ Check
    💬 Gazelle complete (clean)
  • ✅ lint (lint-gha) · ⏱ 15.2s · ✨ Aspect · 🐙 GitHub Actions · ☑️ Check
    💬 Lint complete (clean)
  • ✅ lint (lint-gha-debug) · ⏱ 24.8s · ✨ Aspect · 🐙 GitHub Actions · ☑️ Check
    💬 Lint complete (clean)
  • ✅ test (test-gha) · ⏱ 21.7s · ✨ Aspect · 🐙 GitHub Actions · ☑️ Check
    💬 Bazel test complete (25/25 passed · 25 cached)
  • ✅ test (test-gha-debug) · ⏱ 1m 47s · ✨ Aspect · 🐙 GitHub Actions · ☑️ Check
    💬 Bazel test complete (25/25 passed · 25 cached)

🔁 Reproduce

❌ buildifier (buildifier-gha-debug · buildifier-gha-ephemeral)

aspect buildifier --on-change=fail --base-ref=origin/main

⚠️ delivery (delivery-gha · delivery-gha-debug)

aspect delivery --mode=always --track-state=false --dry-run=true

CI ran --mode=selective; --mode=always is the off-runner equivalent (selective change detection needs the Aspect Workflows delivery state backend).

🛠️ Fix

❌ buildifier (buildifier-gha-debug · buildifier-gha-ephemeral)

aspect buildifier --on-change=silent -- crates/aspect-cli/src/builtins/aspect/warming.axl

⏱ Last updated Mon May 25 23:12:55 UTC 2026 · 📊 GitHub API quota 80/15,000 (1% used, resets in 58m)
🚀 Powered by Aspect CLI (v0.0.0-dev)  |  Aspect Build · X · LinkedIn · YouTube

@gregmagolan
gregmagolan force-pushed the aspect_ci_warming_task branch from 1369346 to 9cbc9dd Compare May 5, 2026 18:14
@gregmagolan
gregmagolan marked this pull request as ready for review May 5, 2026 18:44
@gregmagolan
gregmagolan requested a review from thesayyn May 5, 2026 18:44

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 573c2b0010

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".


def _impl(ctx: TaskContext) -> int:
on_workflows_runner = bool(ctx.std.env.var("ASPECT_WORKFLOWS_RUNNER"))
mount = ctx.std.env.var("ASPECT__STORAGE_PATH")

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Badge Use runner storage env var for warming cleanup

The cleanup step reads ASPECT__STORAGE_PATH, but the runner environment model in lib/environment.axl is keyed off ASPECT_WORKFLOWS_RUNNER_STORAGE_PATH (with defaults derived from it). On Workflows runners where only the documented ASPECT_WORKFLOWS_RUNNER_* variables are present, mount is empty, so the pre-warming cleanup is skipped and stale cache contents can be re-archived by warming_archive, defeating the task’s “clean then repopulate” behavior.

Useful? React with 👍 / 👎.

@gregmagolan
gregmagolan force-pushed the aspect_ci_warming_task branch 3 times, most recently from 3b26ee6 to cf6ab03 Compare May 9, 2026 17:58
@gregmagolan
gregmagolan force-pushed the aspect_ci_warming_task branch 2 times, most recently from 767c490 to 6c8fdfd Compare May 20, 2026 01:06
@gregmagolan
gregmagolan marked this pull request as draft May 20, 2026 01:36
@gregmagolan
gregmagolan force-pushed the aspect_ci_warming_task branch from 6c8fdfd to 312b9f0 Compare May 25, 2026 08:34
gregmagolan and others added 3 commits May 25, 2026 16:06
Cleans `${ASPECT__STORAGE_PATH}/{bazel,output,caches/repository}` —
matching rosetta's WarmingTaskRef cleanup — then runs
`bazel build --nobuild` against the given targets so the repository,
output, and bazel caches under the ephemeral mount are populated and
ready for an Aspect Workflows warming archive to capture.

Default targets `//...`; positional args override. Honors BazelTrait
flags / startup_flags contributions so warming sees the same
`--config=...` etc. as real builds.

Pairs with #1062 (env-var routing of the launcher / AXL caches into
the same ephemeral mount) and a forthcoming silo PR that adds these
subtrees to the warming archive.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Closes the loop on the customer-facing warming workflow. After the
bazel cache populate step succeeds, when ASPECT_WORKFLOWS_RUNNER is
set the task now invokes /etc/aspect/workflows/bin/warming_archive to
upload the populated caches to the warming bucket — so the customer's
GitHub Actions workflow drops to:

    steps:
      - uses: actions/checkout@v4
      - name: Warming Task
        run: aspect ci warming

Off a Workflows runner the archive step is skipped with a warning to
stderr; the populate step still runs so the task is useful for local
testing.
@gregmagolan
gregmagolan force-pushed the aspect_ci_warming_task branch from 312b9f0 to aba0355 Compare May 25, 2026 23:06
gregmagolan added a commit that referenced this pull request Jun 27, 2026
Folds the `aspect ci warming` task (originally proposed in #1063) into the `ci`
command group alongside `aspect ci bazelrc`, so the whole group lands in one
PR. Cleans prior Bazel state under ${ASPECT__STORAGE_PATH}, runs
`bazel build --nobuild` against the given targets to populate the repository,
output, and bazel caches, and — on an Aspect Workflows runner — invokes
/etc/aspect/workflows/bin/warming_archive to upload them to the warming bucket.

Carried over verbatim from #1063 except the `BazelTrait` load, rewritten to the
canonical `@aspect//traits.axl` form required by the public/private load lint
that landed on main after that branch was cut.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@gregmagolan

Copy link
Copy Markdown
Member Author

Superseded by #1294, which lands the whole aspect ci command group (bazelrc + warming) in one PR. The warming.axl from this branch is carried over there verbatim, except the BazelTrait load rewritten to the canonical @aspect//traits.axl form required by the public/private load lint that landed after this branch was cut. Closing in favor of the consolidated PR.

gregmagolan added a commit that referenced this pull request Jun 27, 2026
The task carried over from #1063 predated the current buildifier config;
normalize blank lines and trailing commas so the buildifier CI check passes.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
gregmagolan added a commit that referenced this pull request Jun 27, 2026
…#1294)

Introduces the `aspect ci` command group — the CI-runner-facing commands
that the Aspect Workflows integrations call. This PR lands the whole
group in one shot: `aspect ci bazelrc` and `aspect ci warming` (the
latter folded in from #1063, which this supersedes).

## `aspect ci bazelrc`

The in-CLI successor to `rosetta bazelrc`. On an Aspect Workflows CI
runner, `aspect <task>` already routes Bazel through the runner's remote
cache, repository cache, and per-runner output paths (via the
`workflows` feature + `get_bazelrc_flags`). A bare `bazel build` started
from the same job gets none of that. This command writes those same
flags to a Bazel rc file so vanilla `bazel` invocations on the runner
pick them up too.

- Reuses `get_bazelrc_flags` — the single source of truth the
`workflows` feature uses — so the generated rc can't drift from what
`aspect <task>` injects.
- Everything lands under one `common` section plus the runner's
`startup` flags; version-gated `(flag, constraint)` tuples are resolved
against the Bazel version detected on the runner (assumed-latest when
undetectable, matching the runtime's `constraint_matches`).
- Defaults to `~/.bazelrc` (the first user rc Bazel loads), overridable
with `--output`. File output only — no stdout mode — and it refuses to
run off a Workflows runner.
- Unit tests cover the constraint gating, flag resolution, and rc
rendering (`aspect dev test-bazelrc`).

## `aspect ci warming`

Pre-populates Bazel caches on a CI runner: cleans prior Bazel state
under `${ASPECT__STORAGE_PATH}`, runs `bazel build --nobuild` against
the given targets, and — on an Aspect Workflows runner — invokes
`/etc/aspect/workflows/bin/warming_archive` to upload the populated
caches to the warming bucket. Carried over from #1063; the only change
is the `BazelTrait` load rewritten to the canonical
`@aspect//traits.axl` form required by the public/private load lint that
landed after that branch was cut.

The CI integrations (the setup-aspect GitHub Action, Buildkite plugin,
CircleCI orb, GitLab component) call `aspect ci bazelrc` instead of
`rosetta bazelrc` once `rosetta` is gone from the runner image.

---

### Changes are visible to end-users: yes

- Searched for relevant documentation and updated as needed: yes
(docsite PR switches the integrations from `rosetta bazelrc` to `aspect
ci bazelrc`)
- Breaking change (forces users to change their own code or config): no
- Suggested release notes appear below: yes

- Added the `aspect ci` command group: `aspect ci bazelrc` writes a
Bazel rc (default `~/.bazelrc`) that routes raw `bazel` calls on an
Aspect Workflows CI runner through the runner's caches — the in-CLI
replacement for `rosetta bazelrc` — and `aspect ci warming`
pre-populates and archives the runner's Bazel caches.

### Test plan

- New test cases added (`bazelrc_test.axl`: version-constraint gating,
mixed plain/tuple flag resolution, rc rendering; `aspect dev
test-bazelrc`).
- Manual testing:
- `aspect ci bazelrc` off a runner exits non-zero with a clear message
and writes nothing; on a runner (or faked via
`ASPECT_WORKFLOWS_RUNNER=1`) it writes the `startup`/`common` flags with
version-gated flags resolved against the local `bazel --version`.
- `aspect ci warming --help` and group registration verified; both
subcommands appear under `aspect ci`.

---------

Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant