From c25cafa149fe5b47e11d26855bd31c4040e21885 Mon Sep 17 00:00:00 2001 From: mergify-ci-bot Date: Mon, 16 Nov 2026 18:06:25 +0000 Subject: [PATCH] chore: sync the public files from 2026.11.10 Copied from the CLI's source at 2026.11.10 by ci-bot's public-releases-sync. Edit these files in the CLI's source, not here: the next release overwrites them. --- README.md | 20 +- skills/mergify-ci/SKILL.md | 78 ++++--- skills/mergify-config/SKILL.md | 11 +- skills/mergify-events/SKILL.md | 9 + skills/mergify-merge-protections/SKILL.md | 11 +- skills/mergify-merge-queue/SKILL.md | 9 + skills/mergify-test-engine-setup/SKILL.md | 69 +++--- .../references/containers.md | 12 +- .../references/github-actions.md | 215 ++++++++++-------- .../references/jest.md | 9 +- .../references/minitest.md | 11 +- .../references/mocha.md | 9 +- .../references/playwright.md | 7 +- .../references/pytest.md | 20 +- .../references/rspec.md | 14 +- .../references/verify.md | 39 ++-- .../references/vitest.md | 12 +- 17 files changed, 334 insertions(+), 221 deletions(-) diff --git a/README.md b/README.md index a659d396..1a354ba3 100644 --- a/README.md +++ b/README.md @@ -19,7 +19,8 @@ mergify ci junit-process report.xml # upload test results to Test Engine laptop or a CI runner and go. - **One command to sign in.** `mergify auth login` approves the CLI in your browser and keeps the credential in your OS keychain; `MERGIFY_TOKEN` or - `--token` for scripting. + `--token` for scripting. In a GitHub Actions job granted `id-token: write`, + `mergify ci` needs neither: it exchanges the job's ID token. - **Built for pipelines.** Logs to stderr, structured `--json` output on read commands, and stable [exit codes](#exit-codes) for scripts and runbooks. - **Cross-platform.** Linux (x86_64 + aarch64, glibc 2.28 or newer), macOS @@ -95,12 +96,15 @@ Mergify API is dropping GitHub tokens, and a command with none of the credential above fails and says how to get one. Run `mergify auth login`, or set `MERGIFY_TOKEN` in CI. -`mergify ci` skips step 3. Those endpoints require an organization -application key, which is what `MERGIFY_TOKEN` holds in a CI job; the -per-user credential `mergify auth login` mints is refused there by design. -With an application key in `MERGIFY_TOKEN`, `mergify auth status` names the -key, its permissions and when it expires: a refused request never says which -permission it lacked. +`mergify ci` skips step 3: its endpoints take a CI credential, and refuse the +per-user one `mergify auth login` mints by design. In a GitHub Actions job +granted `id-token: write`, with neither `--token` nor `MERGIFY_TOKEN`, +`mergify ci junit-process` and `mergify ci scopes-send` exchange the job's ID +token for a short-lived Mergify token, once a trust policy of the organization +allows the job (`mergify oidc`, below). On any other CI, `MERGIFY_TOKEN` holds +an organization application key with the `ci` scope. With an application key +in `MERGIFY_TOKEN`, `mergify auth status` names the key, its permissions and +when it expires: a refused request never says which permission it lacked. `mergify stack` also calls the **GitHub API** directly, and resolves that token separately (`--token`, `MERGIFY_TOKEN`, `GITHUB_TOKEN`, `gh auth @@ -220,7 +224,7 @@ These are accepted on every command: | Variable | Effect | | --- | --- | -| `MERGIFY_TOKEN` | API token. Takes precedence over a stored `mergify auth login` credential. | +| `MERGIFY_TOKEN` | API token. Takes precedence over a stored `mergify auth login` credential. Unset in a GitHub Actions job granted `id-token: write`, `mergify ci` exchanges the job's ID token instead. | | `GITHUB_TOKEN` | Not a Mergify API credential. Still the GitHub token `mergify stack` uses. | | `GITHUB_REPOSITORY` | Default `owner/repo` when `--repository` is omitted. | | `MERGIFY_API_URL` | API base URL (default `https://api.mergify.com`). | diff --git a/skills/mergify-ci/SKILL.md b/skills/mergify-ci/SKILL.md index f0469272..80838750 100644 --- a/skills/mergify-ci/SKILL.md +++ b/skills/mergify-ci/SKILL.md @@ -41,9 +41,11 @@ CLI calls, the two lists share only the quarantine list. | `ci` | What a CI job does: trace upload, `scopes-send`, quarantine *evaluation* (`junit-process`) and the quarantine *list*. | | `admin` | Reading test health, the quarantine *list*, and mutating the quarantine. Not trace upload, not `scopes-send`. | -Trace upload and `scopes-send` take a `ci` key only: an `admin` key or a -user credential gets a `403`. Every other endpoint below that refuses a `ci` -key accepts a user credential (`mergify auth login`) instead. `GITHUB_TOKEN` +Trace upload and `scopes-send` take only a `ci` key, or on GitHub Actions the +job's ID token ([below](#github-actions-jobs-need-no-key-oidc)): an `admin` +key or a user credential gets a `403`. Every other endpoint below that refuses a `ci` +key accepts a user credential (`mergify auth login`, which your user runs: see +[below](#when-a-command-says-there-is-no-mergify-credential)) instead. `GITHUB_TOKEN` inside GitHub Actions is not a Mergify credential at all, so do not reach for it to clear one of these `403`s. @@ -72,8 +74,8 @@ trades its GitHub ID token for a short-lived token that does what a `ci` key does, on its own repository only, when a trust policy of the organization matches the job: `ci junit-process`, `ci scopes-send` and the test framework plugins (2026.11.2 or later) do it themselves. A `MERGIFY_TOKEN` that is set -always wins. Writing a trust policy needs the Enterprise plan, and an -organization admin or CI Admin: see [OIDC Trust Policies](#oidc-trust-policies-mergify-oidc). +always wins. Writing a trust policy needs an organization admin or CI Admin: +see [OIDC Trust Policies](#oidc-trust-policies-mergify-oidc). Fork pull requests get no ID token, as they get no secret. Jobs outside GitHub Actions keep a `ci` key. @@ -88,7 +90,7 @@ mergify ci junit-process \ path/to/junit-results.xml ``` -The key is read from `MERGIFY_TOKEN`: set it in the job's environment from a secret rather than passing `--token`, which would put it in the process list. +On GitHub Actions, a job granted `id-token: write` needs no key (see [GitHub Actions jobs need no key](#github-actions-jobs-need-no-key-oidc)). On any other CI, the key is read from `MERGIFY_TOKEN`: set it in the job's environment from a secret rather than passing `--token`, which would put it in the process list. `FILES` can be individual paths or quoted glob patterns (e.g. `'reports/**/*.xml'`). Always quote the pattern so Mergify expands it rather than the shell — this is the recommended approach for large, sharded test suites. @@ -110,21 +112,27 @@ The key is read from `MERGIFY_TOKEN`: set it in the job's environment from a sec 5. Exits with code 0 if all failures are quarantined, code 1 if any are blocking 6. If `--test-exit-code` is non-zero but no test failures are found, exits with code 1 (silent failure detection) 7. Upload failures never affect the exit code (Mergify-side trouble must not break CI), but they are surfaced: on GitHub Actions the command emits an `::error::` annotation when the upload is rejected (refused by the schema check, or HTTP 4xx except 408/429, e.g. a token without access to the repository, or Test Engine turned off for it: the annotation then gives Mergify's reason) or a `::warning::` annotation for transient errors (5xx, 408, 429, network), and appends `test_results_upload=success|rejected|failed` to `$GITHUB_OUTPUT` so workflows can detect dead ingest programmatically +8. With no `MERGIFY_TOKEN`, a job whose ID token Mergify refuses (no trust policy matches it yet) or does not exchange carries on without Mergify: nothing is uploaded and no test is quarantined, so it exits as points 5 and 6 say with every failure blocking. The report says `MERGIFY_TOKEN is not set, skipping the test results upload and the quarantine check.`, followed by Mergify's reason when it refused, and so does a `::warning::` annotation on GitHub Actions; `$GITHUB_OUTPUT` gets `test_results_upload=rejected`. A Mergify that cannot answer the exchange is an outage instead, reported as in point 7 **GitHub Actions example:** ```yaml -- name: Run tests - id: tests - run: pytest --junitxml=results.xml || echo "exit_code=$?" >> "$GITHUB_OUTPUT" - -- name: Process test results - if: always() - run: | - mergify ci junit-process \ - --test-exit-code ${{ steps.tests.outputs.exit_code || 0 }} \ - results.xml - env: - MERGIFY_TOKEN: ${{ secrets.MERGIFY_TOKEN }} +jobs: + test: + permissions: + id-token: write + contents: read + steps: + # ... + - name: Run tests + id: tests + run: pytest --junitxml=results.xml || echo "exit_code=$?" >> "$GITHUB_OUTPUT" + + - name: Process test results + if: always() && (!github.event.pull_request || github.event.pull_request.head.repo.full_name == github.repository) + run: | + mergify ci junit-process \ + --test-exit-code ${{ steps.tests.outputs.exit_code || 0 }} \ + results.xml ``` ## Git References (`git-refs`) @@ -199,8 +207,8 @@ Sends scopes tied to a pull request to the Mergify API. Used when scopes are det The report is addressed by the pull request's **head SHA**, so it says which revision it was computed for and a result computed for an older head cannot be taken for the current one. The head is detected from the CI environment (GitHub Actions event payload, `BUILDKITE_COMMIT`, `CIRCLE_SHA1`, or a Jenkins pull request build's checkout); pass `--head-sha` to name it explicitly wherever it is not detected: a GitHub Actions event with no `pull_request` payload (`push`, `workflow_run`, ...), any other CI. A Jenkins multibranch pull request build needs no flag: the number comes from `CHANGE_ID`, and the head is the commit checked out (`GIT_COMMIT`, else `HEAD`) or, when the GitHub Branch Source plugin merges the pull request into its target branch, the first parent of the merge Jenkins made. No head is detected where git cannot read the checkout, such as a container without its `.git`, nor on an agent that merges with JGit, whose merge Jenkins does not make under its own name: pass `--head-sha` with the pull request's head there, or switch the source's strategy to "The current pull request revision". When no head SHA can be resolved, the command reports against the pull request number alone and warns: that route is deprecated and will be removed. A Mergify deployment that predates the commit endpoint (on-premise before 8.18.0) gets the same pull request route without a warning. -Needs a `ci` key — an `admin` key or a user credential gets a `403`. See -[Authentication](#authentication). +Needs a `ci` key, or on GitHub Actions the job's ID token; an `admin` key or a +user credential gets a `403`. See [Authentication](#authentication). ```bash # Send specific scopes @@ -464,18 +472,16 @@ A GitHub Actions job gets a Mergify token for its ID token when one trust policy of its organization matches it, and a policy matches when each of its lines does: repositories, workflows, events and environments. Policies only allow, with no order and no deny, so a narrow one cannot hold back a broad one. +OIDC is in Preview, open to every organization: it may still change. Every `mergify oidc` command takes the organization with `--org`, else the owner of the current repository, and the credential of one of its admins or CI Admins (`mergify auth login`), or an application key with the OIDC trust policies permission, and the repositories one to name repositories (an -`admin` key has both); a `ci` key gets a `403`. Creating or editing a policy -needs a plan that includes CI authentication with OIDC: without it Mergify -answers HTTP 402, and the jobs keep a `ci` key. Listing, deleting and the -exchanges never need it. Every command takes `--json`, which prints what -Mergify answered; `exchanges list --json` puts the exchanges under -`exchanges`, beside `since` and `truncated` (true when `--limit` left some -out). +`admin` key has both); a `ci` key gets a `403`. Every command takes `--json`, +which prints what Mergify answered; `exchanges list --json` puts the exchanges +under `exchanges`, beside `since` and `truncated` (true when `--limit` left +some out). ```bash # Every policy, line by line. @@ -559,15 +565,16 @@ This command is useful in CI workflows that need to know whether the current run ```yaml jobs: test: + permissions: + id-token: write + contents: read steps: - uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1 - name: Run tests run: pytest --junitxml=results.xml - name: Upload and evaluate - if: always() + if: always() && (!github.event.pull_request || github.event.pull_request.head.repo.full_name == github.repository) run: mergify ci junit-process results.xml - env: - MERGIFY_TOKEN: ${{ secrets.MERGIFY_TOKEN }} ``` ### Selective testing with scopes @@ -590,6 +597,17 @@ jobs: - run: pytest backend/ ``` +## When a command says there is no Mergify credential + +On your user's machine, `no Mergify credential found` from `tests show`, `tests runs`, `tests quarantines` or `oidc` means neither `--token`, `MERGIFY_TOKEN` nor a stored login gave the CLI a credential. Stop and ask your user to log in: + +- Don't run `mergify auth login` yourself. It prints a URL and a code, then waits until a person approves it in a browser: you cannot finish it, and the tool call hangs. +- In Claude Code, ask them to type `! mergify auth login` at the prompt. Elsewhere, ask them to run `mergify auth login` in their own terminal. +- Wait until they say it's done, then run the same command again. +- Don't read another source instead (CI logs, the GitHub API) without asking them first. + +`ci junit-process` and `ci scopes-send` say something else when they find no credential: they need an application key in `MERGIFY_TOKEN` (or OIDC on GitHub Actions), which logging in does not provide. + ## When a command or flag is missing If `mergify` rejects a command or flag this skill names, the CLI is probably older than the skill: follow the `mergify-cli-upgrade` skill. diff --git a/skills/mergify-config/SKILL.md b/skills/mergify-config/SKILL.md index 2a14dd94..5d297b24 100644 --- a/skills/mergify-config/SKILL.md +++ b/skills/mergify-config/SKILL.md @@ -68,7 +68,7 @@ mergify config simulate https://github.com/owner/repo/pull/123 - `PULL_REQUEST_URL` -- Full GitHub URL of the pull request to simulate against **Options:** -- `--token` / `-t` (env: `MERGIFY_TOKEN`) -- Mergify credential. Falls back to the credential `mergify auth login` stored. The CLI no longer falls back to `GITHUB_TOKEN` or `gh auth token`. +- `--token` / `-t` (env: `MERGIFY_TOKEN`) -- Mergify credential. Falls back to the credential `mergify auth login` stored (your user runs it, see [below](#when-a-command-says-there-is-no-mergify-credential)). The CLI no longer falls back to `GITHUB_TOKEN` or `gh auth token`. - `--api-url` / `-u` (env: `MERGIFY_API_URL`) -- Mergify API URL (default: `https://api.mergify.com`) **Output:** Shows a title and detailed Markdown summary of what actions Mergify would take on the PR with the local configuration. @@ -185,6 +185,15 @@ jobs: run: mergify config validate ``` +## When a command says there is no Mergify credential + +`no Mergify credential found` (from `simulate`, which asks the Mergify API) means neither `--token`, `MERGIFY_TOKEN` nor a stored login gave the CLI a credential. Stop and ask your user to log in: + +- Don't run `mergify auth login` yourself. It prints a URL and a code, then waits until a person approves it in a browser: you cannot finish it, and the tool call hangs. +- In Claude Code, ask them to type `! mergify auth login` at the prompt. Elsewhere, ask them to run `mergify auth login` in their own terminal. +- Wait until they say it's done, then run the same command again. +- Don't guess what Mergify would do instead, or read another source, without asking them first. + ## When a command or flag is missing If `mergify` rejects a command or flag this skill names, the CLI is probably older than the skill: follow the `mergify-cli-upgrade` skill. diff --git a/skills/mergify-events/SKILL.md b/skills/mergify-events/SKILL.md index d0dc5d04..7e557db9 100644 --- a/skills/mergify-events/SKILL.md +++ b/skills/mergify-events/SKILL.md @@ -67,6 +67,15 @@ The window is echoed so an empty `events` is self-describing. Filter with `jq` o - **"Why was this PR dequeued?"** — `mergify queue show ` (the `mergify-merge-queue` skill) is the dedicated answer: it renders the last leave event with the reason, the failing checks' job URLs, and the head-SHA staleness check. `mergify events` is for the *whole* trail or for non-queue events. - **Raw API access** (CLI not installed, or a token refused the log): `GET /v1/repos/{owner}/{repo}/logs` — same data; always pass `received_from` (the API silently defaults to 1 day) and keep the span ≤ 93 days. +## When a command says there is no Mergify credential + +`no Mergify credential found` means neither `--token`, `MERGIFY_TOKEN` nor a stored login gave the CLI a credential. Stop and ask your user to log in: + +- Don't run `mergify auth login` yourself. It prints a URL and a code, then waits until a person approves it in a browser: you cannot finish it, and the tool call hangs. +- In Claude Code, ask them to type `! mergify auth login` at the prompt. Elsewhere, ask them to run `mergify auth login` in their own terminal. +- Wait until they say it's done, then run the same command again. +- Don't look for the answer somewhere else (the GitHub API, the PR's comments) without asking them first. + ## When a command or flag is missing If `mergify` rejects a command or flag this skill names, the CLI is probably older than the skill: follow the `mergify-cli-upgrade` skill. diff --git a/skills/mergify-merge-protections/SKILL.md b/skills/mergify-merge-protections/SKILL.md index 33fd6d0c..22aa76ee 100644 --- a/skills/mergify-merge-protections/SKILL.md +++ b/skills/mergify-merge-protections/SKILL.md @@ -107,7 +107,7 @@ mergify freeze delete FREEZE_ID # Delete a freeze ## Authentication -All commands require a Mergify credential. Run `mergify auth login` once, or: +All commands require a Mergify credential: the one `mergify auth login` stores (your user runs it, see [below](#when-a-command-says-there-is-no-mergify-credential)), or: - `--token` / `-t` (env: `MERGIFY_TOKEN`) -- Mergify credential. Falls back to the credential `mergify auth login` stored. The CLI no longer falls back to `GITHUB_TOKEN` or `gh auth token`. - `--repository` / `-r` -- Repository full name (auto-detected from git remote) - `--api-url` / `-u` (env: `MERGIFY_API_URL`) -- Mergify API URL (default: `https://api.mergify.com`) @@ -224,6 +224,15 @@ mergify freeze create \ -e "label=security" ``` +## When a command says there is no Mergify credential + +`no Mergify credential found` means neither `--token`, `MERGIFY_TOKEN` nor a stored login gave the CLI a credential. Stop and ask your user to log in: + +- Don't run `mergify auth login` yourself. It prints a URL and a code, then waits until a person approves it in a browser: you cannot finish it, and the tool call hangs. +- In Claude Code, ask them to type `! mergify auth login` at the prompt. Elsewhere, ask them to run `mergify auth login` in their own terminal. +- Wait until they say it's done, then run the same command again. +- Don't do it some other way, or read another source, without asking them first. + ## When a command or flag is missing If `mergify` rejects a command or flag this skill names, the CLI is probably older than the skill: follow the `mergify-cli-upgrade` skill. diff --git a/skills/mergify-merge-queue/SKILL.md b/skills/mergify-merge-queue/SKILL.md index 0c448d0d..2b2ffcf9 100644 --- a/skills/mergify-merge-queue/SKILL.md +++ b/skills/mergify-merge-queue/SKILL.md @@ -420,6 +420,15 @@ mergify queue unpause - Check for failing batches that trigger bisection: `mergify queue status` - Bisecting batches test PRs individually, which is slower than batch merging +## When a command says there is no Mergify credential + +`no Mergify credential found` means neither `--token`, `MERGIFY_TOKEN` nor a stored login gave the CLI a credential. Stop and ask your user to log in: + +- Don't run `mergify auth login` yourself. It prints a URL and a code, then waits until a person approves it in a browser: you cannot finish it, and the tool call hangs. +- In Claude Code, ask them to type `! mergify auth login` at the prompt. Elsewhere, ask them to run `mergify auth login` in their own terminal. +- Wait until they say it's done, then run the same command again. +- Don't look for the answer somewhere else (the GitHub API, the PR's comments) without asking them first. + ## When a command or flag is missing If `mergify` rejects a command or flag this skill names, the CLI is probably older than the skill: follow the `mergify-cli-upgrade` skill. diff --git a/skills/mergify-test-engine-setup/SKILL.md b/skills/mergify-test-engine-setup/SKILL.md index 36a9f08e..36d3cca0 100644 --- a/skills/mergify-test-engine-setup/SKILL.md +++ b/skills/mergify-test-engine-setup/SKILL.md @@ -97,10 +97,10 @@ Node.js, **ask the user** before changing it, and set up everything else first. The framework and CI references give the exact steps. The rules that apply everywhere: -- **Decide first how the jobs reach Mergify** (step 8): OIDC or a key. With - OIDC, a GitHub Actions job is granted `id-token: write` and gets no - `MERGIFY_TOKEN`: leave out the `MERGIFY_TOKEN` line a framework reference - gives, and follow the OIDC section of `references/github-actions.md`. +- **How the jobs reach Mergify** (step 8): on GitHub Actions, with their ID + token (OIDC): each job is granted `id-token: write` and gets no + `MERGIFY_TOKEN` (`references/github-actions.md`). On another CI, or when + step 8 picks a key, with `MERGIFY_TOKEN`. - **`MERGIFY_TOKEN` comes from the CI's secret store**, never from a file. Never ask for its value, never write a token anywhere. - **The plugins report only when `CI` is `true`** (or their enable variable is @@ -117,11 +117,11 @@ everywhere: environments); - per called workflow, when reusable workflows share a job id; - never per shard: shards of one suite share one name. -- **Without a token** (fork pull requests; Dependabot's until its own secret is - stored), the plugins upload nothing and the tests run as usual. The JUnit - upload step needs a guard (`references/github-actions.md`). On Jenkins a - missing credential fails the stage (`references/jenkins.md`). With OIDC, - fork pull requests get no ID token either, and Dependabot's get one. +- **Without a credential** (pull requests from forks get no ID token and no + secret; with a key, Dependabot's get none until its own secret is stored), + the plugins upload nothing and the tests run as usual. The JUnit upload step + needs a guard (`references/github-actions.md`). On Jenkins a missing + credential fails the stage (`references/jenkins.md`). ## 6. Keep everything else as it was @@ -144,18 +144,22 @@ written. Check that the CI files still parse. ## 8. How the jobs reach Mergify -**On GitHub Actions, prefer OIDC.** A job granted `id-token: write` with no +**On GitHub Actions, OIDC.** A job granted `id-token: write` with no `MERGIFY_TOKEN` trades its GitHub ID token for a short-lived Mergify token, limited to its repository, when a trust policy of the organization matches -it. Writing a policy needs the Enterprise plan and an organization admin or -CI Admin. Decide with the `mergify` CLI (`mergify-ci` skill, OIDC Trust -Policies), signed in as the user (`mergify auth status`; never log in for -them): +it: no secret to store, Dependabot's included. The plugins and `mergify ci` +need release 2026.11.2 or later (raise an older pin or lock), and +`mergifyio/gha-mergify-ci` v30 or later. +Until a policy matches, Mergify refuses the job's ID token: the plugins upload +nothing, and the JUnit upload step fails the job. An organization admin or CI +Admin writes the policy. Check with the `mergify` CLI (`mergify-ci` skill, OIDC +Trust Policies), signed in as the user (`mergify auth status`; never log in +for them): 1. `mergify oidc trust-policies list --org ` shows each policy line by line, its repositories by name. A policy whose lines cover every job you changed (the repository, the events that start them, their workflows and - environments): use OIDC. + environments): nothing to write. 2. No such policy: **ask the user before writing one**, since it decides which jobs of the organization get a token. Propose the narrowest that covers the jobs, usually @@ -163,23 +167,22 @@ them): (CI events, any workflow, any environment), or `mergify oidc trust-policies update --org --repository …` to add the repository to an existing one (list every repository it already - holds: the flags replace the line). With their go, run it. HTTP 402: the - organization's plan does not include it, use a key. HTTP 403: they are - neither an admin nor a CI Admin: give them the command for one, or use a - key. + holds: the flags replace the line). With their go, run it. HTTP 403: they + are neither an admin nor a CI Admin: give them the command to hand to one. 3. Without the CLI, or when it cannot authenticate or read the policies, ask - the user whether the organization trusts its GitHub Actions jobs with OIDC. - If they do not know, use a key, and tell them OIDC replaces it on the - Enterprise plan. - -With OIDC, the plugins and `mergify ci` need release 2026.11.2 or later, and -`mergifyio/gha-mergify-ci` its latest major. Then there is no secret to -store, Dependabot's included. - -**The key**, on any other CI or without OIDC: the user creates a Mergify -application key with the `ci` scope in the Mergify dashboard and stores it as -`MERGIFY_TOKEN`. Before telling them it is missing, -check what already exists (GitHub: repository secrets with + the user whether a trust policy of the organization covers the + repository's GitHub Actions jobs. If none does, or they do not know, give + them the `create` command above for an organization admin or CI Admin. + +While no policy covers the jobs, open the pull request as a draft, and say in +it that it leaves draft once the policy exists, with the command: its JUnit +upload jobs fail until then. A user who would rather not wait uses a key. + +**A key instead**, on any other CI, and on GitHub Actions when the user asks +for one rather than a trust policy: the user creates a Mergify application +key with the `ci` scope in the Mergify dashboard and stores it as +`MERGIFY_TOKEN`. Before telling them it is missing, check what already exists +(GitHub: repository secrets with `gh secret list --repo /`, organization secrets with `gh api repos///actions/organization-secrets --jq '.secrets[].name'`, Dependabot's with `gh secret list --app dependabot --repo /`). @@ -197,8 +200,8 @@ Without it, uploads are refused while CI stays green. ## 9. Open the pull request One pull request with the change. Say per job what changed and why, which jobs -cannot report and why, and what the user still has to do (the key or the trust -policy, a Node.js upgrade they did not approve). Describe the lockfile change in words. Follow the +cannot report and why, and what the user still has to do (the trust policy or +the key, a Node.js upgrade they did not approve). Describe the lockfile change in words. Follow the repository's conventions for the commit message and the pull request. ## 10. Check that results arrive diff --git a/skills/mergify-test-engine-setup/references/containers.md b/skills/mergify-test-engine-setup/references/containers.md index bb4eaf28..f1e3b264 100644 --- a/skills/mergify-test-engine-setup/references/containers.md +++ b/skills/mergify-test-engine-setup/references/containers.md @@ -1,9 +1,11 @@ # Tests inside tox, Docker or Docker Compose tox, `docker run` and Docker Compose start the tests in an environment that -does not inherit the CI job's variables. Pass through `MERGIFY_TOKEN`, `CI`, -`MERGIFY_TEST_JOB_NAME` if you set it, and every variable the plugin reads to -identify the run. tox's `pass_env` accepts globs (`GITHUB_*`); Docker Compose +does not inherit the CI job's variables. Pass through the credential (on +GitHub Actions, `ACTIONS_ID_TOKEN_REQUEST_URL` and +`ACTIONS_ID_TOKEN_REQUEST_TOKEN`, which GitHub sets in a job granted +`id-token: write`; `MERGIFY_TOKEN` with a key), `CI`, `MERGIFY_TEST_JOB_NAME` +if you set it, and every variable the plugin reads to identify the run. tox's `pass_env` accepts globs (`GITHUB_*`); Docker Compose (`environment:`), `docker run -e` and the Buildkite Docker Compose plugin need each name: @@ -25,10 +27,6 @@ Jenkins: JENKINS_URL BUILD_ID BUILD_URL JOB_NAME BRANCH_NAME NODE_NAME GIT_URL - Pass them by name (`- GITHUB_SHA` in a Compose `environment:` list, or `-e GITHUB_SHA`), never by writing a value into a file. -- **On GitHub Actions with OIDC** (no `MERGIFY_TOKEN`), pass - `ACTIONS_ID_TOKEN_REQUEST_URL` and `ACTIONS_ID_TOKEN_REQUEST_TOKEN` instead: - GitHub sets them in a job granted `id-token: write`, and without them the - plugin has no ID token to exchange. - **On GitHub Actions in a container**, also mount the file `GITHUB_EVENT_PATH` points to, at the same path: without it the run is recorded against the merge commit instead of the pull request's head commit. diff --git a/skills/mergify-test-engine-setup/references/github-actions.md b/skills/mergify-test-engine-setup/references/github-actions.md index a0f5b5b8..694e11b2 100644 --- a/skills/mergify-test-engine-setup/references/github-actions.md +++ b/skills/mergify-test-engine-setup/references/github-actions.md @@ -1,51 +1,74 @@ # GitHub Actions -A job reaches Mergify with its ID token (OIDC) or with the `MERGIFY_TOKEN` -secret: SKILL.md, step 8, decides which. The sections below give the key's -steps; with OIDC, follow [OIDC](#oidc) instead wherever they pass the token. +A job reaches Mergify with its ID token (OIDC), with no secret to store. When +SKILL.md, step 8, picks a key instead, [With a key](#with-a-key) says what +changes. -## Plugin jobs (pytest, RSpec, Vitest, Playwright) +## Grant the ID token -With a key, add the token to the step that runs the tests. `CI` is already set. +Grant `id-token: write` on each job you change, never on the whole workflow, +which would change the other jobs too. ```yaml -- name: Run tests - env: - MERGIFY_TOKEN: ${{ secrets.MERGIFY_TOKEN }} - run: pytest +jobs: + test: + permissions: + id-token: write + contents: read ``` +- A job's `permissions` block sets every permission it does not list to + `none`, and replaces the workflow's own block. List what the job had before: + the workflow's `permissions` entries if it has some, else what its steps use + (`contents: read` for `actions/checkout`, `packages: read` for a `container:` + or `services:` image from GHCR, `pull-requests: write` or `checks: write` for + a step that comments or reports). +- Any code the job runs can then ask GitHub for an ID token, for any audience. + When the repository's workflows already trust a cloud with their ID tokens + (`aws-actions/configure-aws-credentials`, `google-github-actions/auth`, + `azure/login`), say in the pull request that a cloud role trusting this + repository's pull request or branch tokens now trusts the test jobs too. + +The plugins and `mergify ci` (2026.11.2 or later) and +`mergifyio/gha-mergify-ci` (v30 or later) exchange the ID token themselves: no +`MERGIFY_TOKEN`, no `token` input, no `--token`. A set `MERGIFY_TOKEN` wins +over the exchange; an empty one counts as unset. + +## Plugin jobs (pytest, RSpec, Vitest, Playwright) + +Nothing to add on the step that runs the tests: `CI` is already set. + - **Matrix**: every leg shares one job id, so set a name per leg from the matrix values on the same step: `MERGIFY_TEST_JOB_NAME: test (${{ matrix.python-version }}, ${{ matrix.os }})`. - **Two test runs in one job**: a distinct `MERGIFY_TEST_JOB_NAME` on each step. - **Shards**: one name for all shards. -- **Pull requests from forks and Dependabot** get no Actions secret (Dependabot - has its own store, see "The key"): the plugins upload nothing and the job is - unaffected. No guard needed. +- **Pull requests from forks** get no ID token: the plugins upload nothing and + the job is unaffected. No guard needed. Dependabot's pull requests get one. ## JUnit upload jobs Give the test step an `id`, and upload after it with -`mergifyio/gha-mergify-ci` (latest major: `v27`; check +`mergifyio/gha-mergify-ci` (latest major: `v30`; check `gh release list -R Mergifyio/gha-mergify-ci --limit 1` for a newer one, and -pin it to a commit SHA if the repository pins its actions). With a key: +pin it to a commit SHA if the repository pins its actions): ```yaml jobs: test: - env: - HAS_MERGIFY_TOKEN: ${{ secrets.MERGIFY_TOKEN != '' }} + permissions: + id-token: write + contents: read steps: # ... - name: Run tests id: tests run: npm test - - name: Mergify CI Upload - if: (success() || failure()) && env.HAS_MERGIFY_TOKEN == 'true' && hashFiles('junit.xml') != '' - uses: mergifyio/gha-mergify-ci@v27 + - name: Mergify Test Engine Upload + id: upload + if: (success() || failure()) && (!github.event.pull_request || github.event.pull_request.head.repo.full_name == github.repository) && hashFiles('junit.xml') != '' + uses: mergifyio/gha-mergify-ci@v30 with: - token: ${{ secrets.MERGIFY_TOKEN }} report_path: junit.xml test_step_outcome: ${{ steps.tests.outcome }} ``` @@ -54,8 +77,11 @@ jobs: whole condition, parentheses included: `&&` binds tighter than `||`. `hashFiles` (same path as `report_path`) skips the upload when an earlier step failed before the report was written. -- **The upload step fails without a token** (it exits 8 on pull requests from - forks and from Dependabot): the `HAS_MERGIFY_TOKEN` guard skips it there. +- **The upload step fails without a credential**: it exits 8 on pull requests + from forks, which get no ID token. The second condition skips it on a pull + request whose head is in another repository; `head.repo.fork` would also + skip every pull request of a repository that is itself a fork. The step also + fails while no trust policy matches the job (SKILL.md, step 8). - `test_step_outcome` lets Mergify detect a runner that crashed while the report looks clean. - **Matrix**: add `job_name: ` to the action's inputs. @@ -67,7 +93,7 @@ jobs: ```yaml - name: Fail when tests failed and no report was uploaded - if: steps.tests.outcome == 'failure' && env.HAS_MERGIFY_TOKEN != 'true' + if: steps.tests.outcome == 'failure' && steps.upload.outcome == 'skipped' run: exit 1 ``` @@ -75,89 +101,94 @@ jobs: ## Reusable workflows -A workflow called with `uses: ./.github/workflows/.yml` gets no secret -unless the caller passes it: `secrets: inherit`, or - -```yaml - secrets: - MERGIFY_TOKEN: ${{ secrets.MERGIFY_TOKEN }} -``` - -Passed by name, the called workflow must declare it, or the run fails: - -```yaml -on: - workflow_call: - secrets: - MERGIFY_TOKEN: - required: false -``` +A workflow called with `uses: ./.github/workflows/.yml` gets at most the +permissions its caller grants: grant `id-token: write` on the calling job, and +list it in a called job that declares its own `permissions`. Pass no secret. A called workflow reports under its caller's workflow name: two called workflows with the same job id (often `test`) collide, so set a distinct `MERGIFY_TEST_JOB_NAME` (or `job_name`) in each. -## OIDC +## Tests in tox or a container + +Pass `ACTIONS_ID_TOKEN_REQUEST_URL` and `ACTIONS_ID_TOKEN_REQUEST_TOKEN` +through (`containers.md`): without them the plugin has no ID token to +exchange. + +## A refused job + +A job no trust policy matches prints Mergify's reason after +`Error: Mergify refused the job's ID token:`, naming its exchange +(`mergify-ci` skill, OIDC Trust Policies, says how to read it). -The job trades its GitHub ID token for a Mergify token, with no secret: +## With a key -- **Grant `id-token: write`** to the job (or the whole workflow). A - `permissions` block sets every permission it does not list to `none`: keep - the ones the job's other steps need, `contents: read` for - `actions/checkout` at least. +The job reads the `MERGIFY_TOKEN` secret instead, and needs no +`id-token: write`. + +- **Plugin jobs**: add the token to the step that runs the tests. + + ```yaml + - name: Run tests + env: + MERGIFY_TOKEN: ${{ secrets.MERGIFY_TOKEN }} + run: pytest + ``` + + Pull requests from forks and Dependabot get no Actions secret (Dependabot + has its own store, below): the plugins upload nothing and the job is + unaffected. +- **JUnit upload jobs**: pass the token, and skip the upload where there is + none in place of the same-repository condition (the step exits 8 without + one, on pull requests from forks and from Dependabot). The step that fails + the job under `continue-on-error` stays as above: ```yaml jobs: test: - permissions: - id-token: write - contents: read + env: + HAS_MERGIFY_TOKEN: ${{ secrets.MERGIFY_TOKEN != '' }} + steps: + # ... + - name: Mergify Test Engine Upload + id: upload + if: (success() || failure()) && env.HAS_MERGIFY_TOKEN == 'true' && hashFiles('junit.xml') != '' + uses: mergifyio/gha-mergify-ci@v30 + with: + token: ${{ secrets.MERGIFY_TOKEN }} + report_path: junit.xml + test_step_outcome: ${{ steps.tests.outcome }} + ``` + +- **Reusable workflows** get no secret unless the caller passes it: + `secrets: inherit`, or + + ```yaml + secrets: + MERGIFY_TOKEN: ${{ secrets.MERGIFY_TOKEN }} ``` -- **No `MERGIFY_TOKEN`** on the test step, no `token` input on - `mergifyio/gha-mergify-ci`, no `--token`: the plugins and `mergify ci` - (2026.11.2 or later) and `mergifyio/gha-mergify-ci` (its latest major) - exchange the ID token themselves. A set `MERGIFY_TOKEN` wins over the - exchange; an empty one counts as unset. -- **JUnit upload**: fork pull requests get no ID token, so skip the upload - there in place of the `HAS_MERGIFY_TOKEN` guard (drop that `env` too): + Passed by name, the called workflow must declare it, or the run fails: ```yaml - - name: Mergify CI Upload - if: (success() || failure()) && !github.event.pull_request.head.repo.fork && hashFiles('junit.xml') != '' - uses: mergifyio/gha-mergify-ci@ - with: - report_path: junit.xml - test_step_outcome: ${{ steps.tests.outcome }} + on: + workflow_call: + secrets: + MERGIFY_TOKEN: + required: false ``` - With `continue-on-error` on the test step, the step that fails the job - runs on `steps.tests.outcome == 'failure' && github.event.pull_request.head.repo.fork`. -- **Reusable workflows** get at most what their caller grants: grant - `id-token: write` on the calling job, and list it in a called job that - declares its own `permissions`. Pass no secret. -- **Tests in tox or a container**: pass `ACTIONS_ID_TOKEN_REQUEST_URL` and - `ACTIONS_ID_TOKEN_REQUEST_TOKEN` through instead of `MERGIFY_TOKEN` - (`containers.md`). -- **Dependabot's pull requests** get an ID token: no Dependabot secret. -- **A refused job** prints Mergify's reason after - `Error: Mergify refused the job's ID token:`, naming its exchange - (`mergify-ci` skill, OIDC Trust Policies, says how to read it). - -## Workflow linters - -Some zizmor versions and personas report `secrets-outside-env` on the new -`secrets.MERGIFY_TOKEN` line. Run the repository's own zizmor configuration on -the changed workflow; if it reports the finding, follow the convention the -repository already uses for it (a deployment environment on the job, or an -inline `# zizmor: ignore[secrets-outside-env]`). Keep actionlint green too. - -## The key - -Only without OIDC. `gh secret set MERGIFY_TOKEN --repo /` (prompts for the -value). An organization secret covers every repository: check -`gh api repos///actions/organization-secrets --jq '.secrets[].name'` -before saying it is missing. Dependabot sees neither: when -`.github/dependabot.yml` exists, also give -`gh secret set MERGIFY_TOKEN --app dependabot --repo /` -unless `gh secret list --app dependabot` already lists it. +- **Workflow linters**: some zizmor versions and personas report + `secrets-outside-env` on the new `secrets.MERGIFY_TOKEN` line. Run the + repository's own zizmor configuration on the changed workflow; if it reports + the finding, follow the convention the repository already uses for it (a + deployment environment on the job, or an inline + `# zizmor: ignore[secrets-outside-env]`). Keep actionlint green too. +- **Storing it**: `gh secret set MERGIFY_TOKEN --repo /` + (prompts for the value). An organization secret covers every repository: + check + `gh api repos///actions/organization-secrets --jq '.secrets[].name'` + before saying it is missing. Dependabot sees neither: when + `.github/dependabot.yml` exists, also give + `gh secret set MERGIFY_TOKEN --app dependabot --repo /` + unless `gh secret list --app dependabot` already lists it. diff --git a/skills/mergify-test-engine-setup/references/jest.md b/skills/mergify-test-engine-setup/references/jest.md index 0afd1ba3..af1fee47 100644 --- a/skills/mergify-test-engine-setup/references/jest.md +++ b/skills/mergify-test-engine-setup/references/jest.md @@ -64,18 +64,19 @@ export default withMergify(config); ## CI -- GitHub Actions: `MERGIFY_TOKEN: ${{ secrets.MERGIFY_TOKEN }}` on the step - that runs Jest. +- GitHub Actions: nothing to add on the step that runs Jest, the job's + `id-token: write` is enough (`github-actions.md`). With a key, + `MERGIFY_TOKEN: ${{ secrets.MERGIFY_TOKEN }}` on the step. - Matrix: `MERGIFY_TEST_JOB_NAME` per leg. `--shard` legs share one name. ## Variables | Variable | Purpose | |---|---| -| `MERGIFY_TOKEN` | the `ci` application key | +| `MERGIFY_TOKEN` | the `ci` application key; unset on GitHub Actions with OIDC | | `MERGIFY_TEST_JOB_NAME` | the job name to report under | | `MERGIFY_CI_ENABLE` | report even when `CI` is not set | | `MERGIFY_CI_DEBUG` | print the spans instead of uploading them | Check: `MERGIFY_CI_ENABLE=true MERGIFY_CI_DEBUG=true npx jest` prints the spans -and a `Mergify CI` summary. +and a `Mergify Test Engine` summary (`Mergify CI` in earlier releases). diff --git a/skills/mergify-test-engine-setup/references/minitest.md b/skills/mergify-test-engine-setup/references/minitest.md index 06dff3f9..7486896b 100644 --- a/skills/mergify-test-engine-setup/references/minitest.md +++ b/skills/mergify-test-engine-setup/references/minitest.md @@ -48,18 +48,21 @@ require 'minitest/mergify' if RUBY_ENGINE == 'ruby' && RUBY_VERSION >= '3.1' # ## CI -- GitHub Actions: `MERGIFY_TOKEN: ${{ secrets.MERGIFY_TOKEN }}` on the step - that runs the tests (`bin/rails test`, `rake test`, …). +- GitHub Actions: nothing to add on the step that runs the tests + (`bin/rails test`, `rake test`, …), the job's `id-token: write` is enough + (`github-actions.md`). With a key, `MERGIFY_TOKEN: ${{ secrets.MERGIFY_TOKEN }}` + on the step. - Matrix: `MERGIFY_TEST_JOB_NAME` per leg. ## Variables | Variable | Purpose | |---|---| -| `MERGIFY_TOKEN` | the `ci` application key | +| `MERGIFY_TOKEN` | the `ci` application key; unset on GitHub Actions with OIDC | | `MERGIFY_TEST_JOB_NAME` | the job name to report under | | `MERGIFY_CI_ENABLE` | report even when `CI` is not set | | `MERGIFY_CI_DEBUG` | print the spans instead of sending them | Check: `MERGIFY_CI_ENABLE=true MERGIFY_CI_DEBUG=true bundle exec rake test` -(or `bin/rails test`) prints the spans and a `Mergify CI` summary. +(or `bin/rails test`) prints the spans and a `# Mergify Test Engine:` summary +(`# Mergify CI:` in earlier releases). diff --git a/skills/mergify-test-engine-setup/references/mocha.md b/skills/mergify-test-engine-setup/references/mocha.md index 012b3450..c539799e 100644 --- a/skills/mergify-test-engine-setup/references/mocha.md +++ b/skills/mergify-test-engine-setup/references/mocha.md @@ -59,8 +59,9 @@ or with `--require @mergifyio/mocha` on the command line. Mocha merges the ## CI -- GitHub Actions: `MERGIFY_TOKEN: ${{ secrets.MERGIFY_TOKEN }}` on the step - that runs Mocha. +- GitHub Actions: nothing to add on the step that runs Mocha, the job's + `id-token: write` is enough (`github-actions.md`). With a key, + `MERGIFY_TOKEN: ${{ secrets.MERGIFY_TOKEN }}` on the step. - Matrix: `MERGIFY_TEST_JOB_NAME` per leg; a job that splits its files between several Mocha runs gives each run its own name. @@ -68,10 +69,10 @@ or with `--require @mergifyio/mocha` on the command line. Mocha merges the | Variable | Purpose | |---|---| -| `MERGIFY_TOKEN` | the `ci` application key | +| `MERGIFY_TOKEN` | the `ci` application key; unset on GitHub Actions with OIDC | | `MERGIFY_TEST_JOB_NAME` | the job name to report under | | `MERGIFY_CI_ENABLE` | report even when `CI` is not set | | `MERGIFY_CI_DEBUG` | print the spans instead of uploading them | Check: `MERGIFY_CI_ENABLE=true MERGIFY_CI_DEBUG=true npx mocha` prints the -spans and a `Mergify CI` summary. +spans and a `Mergify Test Engine` summary (`Mergify CI` in earlier releases). diff --git a/skills/mergify-test-engine-setup/references/playwright.md b/skills/mergify-test-engine-setup/references/playwright.md index b1f796c0..b60a786c 100644 --- a/skills/mergify-test-engine-setup/references/playwright.md +++ b/skills/mergify-test-engine-setup/references/playwright.md @@ -52,8 +52,9 @@ step, results upload but quarantined tests still fail the run. ## CI -- GitHub Actions: `MERGIFY_TOKEN: ${{ secrets.MERGIFY_TOKEN }}` on the step - that runs Playwright. +- GitHub Actions: nothing to add on the step that runs Playwright, the job's + `id-token: write` is enough (`github-actions.md`). With a key, + `MERGIFY_TOKEN: ${{ secrets.MERGIFY_TOKEN }}` on the step. - Matrix: `MERGIFY_TEST_JOB_NAME` per environment leg. `--shard` legs share one name: never put the shard index in it. - **Several projects** (browsers) in one config share a test identity by @@ -74,7 +75,7 @@ step, results upload but quarantined tests still fail the run. | Variable | Purpose | |---|---| -| `MERGIFY_TOKEN` | the `ci` application key | +| `MERGIFY_TOKEN` | the `ci` application key; unset on GitHub Actions with OIDC | | `MERGIFY_TEST_JOB_NAME` | the job name to report under | | `MERGIFY_CI_ENABLE` | report even when `CI` is not set (was `PLAYWRIGHT_MERGIFY_ENABLE`) | | `MERGIFY_PLAYWRIGHT_INCLUDE_PROJECT_IN_TEST_NAME` | prefix test names with the project (was `PLAYWRIGHT_MERGIFY_INCLUDE_PROJECT_IN_TEST_NAME`) | diff --git a/skills/mergify-test-engine-setup/references/pytest.md b/skills/mergify-test-engine-setup/references/pytest.md index a64ab971..ef1d0cd6 100644 --- a/skills/mergify-test-engine-setup/references/pytest.md +++ b/skills/mergify-test-engine-setup/references/pytest.md @@ -8,7 +8,9 @@ configuration change. It uploads at the end of the session. - **CPython only.** Releases from 2026.10.6 need Python 3.10 or later; 2026.10.5 is the last one for 3.8 and 3.9. When the project supports Python below 3.10, require `pytest-mergify>=2026.10.5` and let the resolver pick per - interpreter. `pytest-mergify` ships compiled wheels and no source + interpreter. 2026.10.5 cannot exchange an ID token: on GitHub Actions with + OIDC, the legs on Python 3.8 or 3.9 report nothing (say so in the pull + request). `pytest-mergify` ships compiled wheels and no source distribution: Linux (x86_64, aarch64; glibc 2.28 or newer, and musl), macOS (arm64; x86_64 as best effort, not tested before a release), Windows (x86_64). There is no wheel for **PyPy** or **free-threaded** CPython @@ -39,8 +41,9 @@ Add it where the project's test dependencies live, with its tool: ## CI -- GitHub Actions: `MERGIFY_TOKEN: ${{ secrets.MERGIFY_TOKEN }}` in the `env` of - the step that runs pytest (see `github-actions.md`). +- GitHub Actions: nothing to add on the step that runs pytest, the job's + `id-token: write` is enough (`github-actions.md`). With a key, + `MERGIFY_TOKEN: ${{ secrets.MERGIFY_TOKEN }}` in the step's `env`. - Matrix: `MERGIFY_TEST_JOB_NAME` per leg from the matrix values. - One job running pytest twice (for example with and without optional dependencies): a distinct `MERGIFY_TEST_JOB_NAME` on each step. @@ -64,11 +67,13 @@ pass_env = MERGIFY_* GITHUB_* RUNNER_* + ACTIONS_ID_TOKEN_REQUEST_* ``` Use the same keys as the project (`deps`, `dependency_groups`, `extras`; old -projects use `passenv`). On another CI, replace `GITHUB_*` and `RUNNER_*` with -that CI's variables (`containers.md`). When one `tox` call runs several +projects use `passenv`). `ACTIONS_ID_TOKEN_REQUEST_*` lets the plugin ask for +the job's ID token. On another CI, replace `GITHUB_*`, `RUNNER_*` and +`ACTIONS_ID_TOKEN_REQUEST_*` with that CI's variables (`containers.md`). When one `tox` call runs several environments (`py312-crypto`, `py312-nocrypto`), give each its own name: ```ini @@ -80,11 +85,12 @@ set_env = | Variable | Purpose | |---|---| -| `MERGIFY_TOKEN` | the `ci` application key (from the CI secret store) | +| `MERGIFY_TOKEN` | the `ci` application key (from the CI secret store); unset on GitHub Actions with OIDC | | `MERGIFY_TEST_JOB_NAME` | the job name to report under | | `MERGIFY_CI_ENABLE` | report even when `CI` is not set (was `PYTEST_MERGIFY_ENABLE`) | | `MERGIFY_CI_DEBUG` | print the spans instead of uploading them (was `PYTEST_MERGIFY_DEBUG`) | Check: `MERGIFY_CI_ENABLE=true MERGIFY_CI_DEBUG=true pytest …` prints a -`Mergify CI` section (on 2026.10.5, use the old names). Compare with +`Mergify Test Engine` section (`Mergify CI` in earlier releases; on 2026.10.5, +use the old names). Compare with `pytest -p no:pytest_mergify …` for the run without the plugin. diff --git a/skills/mergify-test-engine-setup/references/rspec.md b/skills/mergify-test-engine-setup/references/rspec.md index 22daccaa..7545f9a2 100644 --- a/skills/mergify-test-engine-setup/references/rspec.md +++ b/skills/mergify-test-engine-setup/references/rspec.md @@ -48,23 +48,25 @@ the pull request which legs cannot report. ## CI -- GitHub Actions: `MERGIFY_TOKEN: ${{ secrets.MERGIFY_TOKEN }}` in the `env` of - the step that runs RSpec. +- GitHub Actions: nothing to add on the step that runs RSpec, the job's + `id-token: write` is enough (`github-actions.md`). With a key, + `MERGIFY_TOKEN: ${{ secrets.MERGIFY_TOKEN }}` in the step's `env`. - Matrix: `MERGIFY_TEST_JOB_NAME` per leg. Shards of one suite (a `ci_node_index` matrix, Knapsack, `parallel_tests`): keep one name for all shards; the default (the job id) already does. - No `continue-on-error`: quarantine is applied inside RSpec. -- An empty or missing `MERGIFY_TOKEN` (fork pull requests) makes the gem warn - that the token is not set; tests are not affected. +- A job with no credential (a pull request from a fork gets no ID token and no + secret) makes the gem warn that the token is not set; tests are not + affected. ## Variables | Variable | Purpose | |---|---| -| `MERGIFY_TOKEN` | the `ci` application key | +| `MERGIFY_TOKEN` | the `ci` application key; unset on GitHub Actions with OIDC | | `MERGIFY_TEST_JOB_NAME` | the job name to report under | | `MERGIFY_CI_ENABLE` | report even when `CI` is not set (was `RSPEC_MERGIFY_ENABLE`) | | `MERGIFY_CI_DEBUG` | print the spans instead of uploading them (was `RSPEC_MERGIFY_DEBUG`) | Check: `MERGIFY_CI_ENABLE=true MERGIFY_CI_DEBUG=true bundle exec rspec` -prints a `Mergify CI:` section. +prints a `Mergify Test Engine:` section (`Mergify CI:` in earlier releases). diff --git a/skills/mergify-test-engine-setup/references/verify.md b/skills/mergify-test-engine-setup/references/verify.md index b7e40037..7ac964f9 100644 --- a/skills/mergify-test-engine-setup/references/verify.md +++ b/skills/mergify-test-engine-setup/references/verify.md @@ -10,9 +10,13 @@ by job, for exactly the jobs you changed. entry per job name they arrived under, on every CI provider, without reading any CI log. It is the check to use whenever the `mergify` CLI is installed (2026.10.8 or later) and can authenticate: `MERGIFY_TOKEN` (any application -key), or the credential `mergify auth login` stored. If the command is missing or refuses to -authenticate, do not install or log in for the user: read the CI logs instead -(below), and give the user the command to run themselves. +key), or the credential `mergify auth login` stored. If the command is missing, +do not install it for the user: read the CI logs instead (below), and give the +user the command to run themselves. If it says `no Mergify credential found`, +do not run `mergify auth login` yourself either: it waits until a person +approves it in a browser. Ask the user to run it (in Claude Code, by typing +`! mergify auth login` at the prompt) and run `mergify tests runs` again once +they say it's done, or read the CI logs instead if they would rather not log in. ```bash mergify tests runs --sha "$(git rev-parse HEAD)" --json # HEAD = the commit you pushed @@ -99,8 +103,10 @@ done jq -r '.jobs[] | [.pipeline_name, .job_name, .test_results] | @tsv' /tmp/mergify-runs.json ``` -- A run on a pull request from a branch of the same repository receives the - repository's secrets, so the upload is real. +- A run on a pull request from a branch of the same repository gets an ID + token and the repository's secrets, so the upload is real once a trust + policy matches the job (a refused exchange only warns in a plugin job: read + the log, SKILL.md step 8). - If you cannot finish (rate limit, a job still going after about 20 minutes), report the remaining jobs as **not verified yet**, with the commands above for the user to run. @@ -152,8 +158,9 @@ gh api --allow-escape-sequences repos///actions/jobs/ it is installed and authenticated; otherwise give the user the traces below to look for. -From 2026.10.6, every plugin ends its run with a `Mergify CI` section (RSpec: -`Mergify CI:`) holding the same lines: +Every plugin ends its run with a `Mergify Test Engine` section (RSpec: +`Mergify Test Engine:`, Minitest: `# Mergify Test Engine:`; earlier releases, +from 2026.10.6, say `Mergify CI` there) holding the same lines: | Line | Meaning | |---|---| @@ -190,15 +197,15 @@ What to do on an error: details, or `test_results_ingestion: not_enabled`): Test Engine is off for the repository. The user turns it on in the Mergify dashboard (the refusal links the page), then re-runs the job. -- **No token** (`MERGIFY_TOKEN is not set`; before 2026.10.6 `No token - configured` or `WARNING: MERGIFY_TOKEN is not set`): the secret is not - stored yet, or not passed to that step. Check the step's `env`; if it is - there, the user stores `MERGIFY_TOKEN` and re-runs the job. A job meant to - use OIDC that prints only this found no ID token: it lacks `id-token: write` - (for a reusable workflow, check the calling job), it runs in a container - that was not passed `ACTIONS_ID_TOKEN_REQUEST_URL` and - `ACTIONS_ID_TOKEN_REQUEST_TOKEN`, its plugin is older than 2026.11.2, or the - run is for a fork's pull request. +- **No credential** (`MERGIFY_TOKEN is not set`; before 2026.10.6 `No token + configured` or `WARNING: MERGIFY_TOKEN is not set`): on GitHub Actions, the + job found no ID token: it lacks `id-token: write` (for a reusable workflow, + check the calling job), it runs in a container that was not passed + `ACTIONS_ID_TOKEN_REQUEST_URL` and `ACTIONS_ID_TOKEN_REQUEST_TOKEN`, its + plugin is older than 2026.11.2, or the run is for a fork's pull request. + With a key, the secret is not stored yet, or not passed to that step: check + the step's `env`; if it is there, the user stores `MERGIFY_TOKEN` and + re-runs the job. - **ID token refused** (`… could not be exchanged for a Mergify token.` then `Error: Mergify refused the job's ID token: …`): ` has no trust policy` or `No trust policy of matches this job`, then diff --git a/skills/mergify-test-engine-setup/references/vitest.md b/skills/mergify-test-engine-setup/references/vitest.md index 3ac14dbd..55ee7f48 100644 --- a/skills/mergify-test-engine-setup/references/vitest.md +++ b/skills/mergify-test-engine-setup/references/vitest.md @@ -54,13 +54,14 @@ export default defineConfig({ browser mode and in custom pools (Cloudflare's Workers pool), and a `runner` the configuration sets keeps running underneath. A custom `runner` that overrides `onBeforeRunTask` without calling `super` loses it: the - `Mergify CI` summary then names the project. Do not change that runner; say + `Mergify Test Engine` summary then names the project. Do not change that runner; say so in the pull request and in your final answer. ## CI -- GitHub Actions: `MERGIFY_TOKEN: ${{ secrets.MERGIFY_TOKEN }}` on the step - that runs Vitest. +- GitHub Actions: nothing to add on the step that runs Vitest, the job's + `id-token: write` is enough (`github-actions.md`). With a key, + `MERGIFY_TOKEN: ${{ secrets.MERGIFY_TOKEN }}` on the step. - Matrix: `MERGIFY_TEST_JOB_NAME` per leg (for example `test (node ${{ matrix.node-version }})`). `--shard` legs share one name. @@ -68,10 +69,11 @@ export default defineConfig({ | Variable | Purpose | |---|---| -| `MERGIFY_TOKEN` | the `ci` application key | +| `MERGIFY_TOKEN` | the `ci` application key; unset on GitHub Actions with OIDC | | `MERGIFY_TEST_JOB_NAME` | the job name to report under | | `MERGIFY_CI_ENABLE` | report even when `CI` is not set (was `VITEST_MERGIFY_ENABLE`) | | `MERGIFY_CI_DEBUG` | print the spans instead of uploading them | Check: `MERGIFY_CI_ENABLE=true MERGIFY_CI_DEBUG=true npx vitest run` prints -the spans and a `Mergify CI` summary. +the spans and a `Mergify Test Engine` summary (`Mergify CI` in earlier +releases).