Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 6 additions & 2 deletions .github/boring-cyborg.yml
Original file line number Diff line number Diff line change
Expand Up @@ -59,16 +59,20 @@ labelPRBasedOnFilePath:
- "**/*.Rd"

verifyTitles:
titleRegexp: '^(WIP:?\s*)?GH-[0-9]+|^MINOR: '
titleRegexp: '^GH-[0-9]+|^MINOR: '
alwaysUsePrTitle: true
validateEitherPrOrSingleCommitTitle: false
statusTitle: "PR Title"
successMessage: "PR title format is valid"
failureMessage: "Title must start with 'GH-<issue>: [Component]' or 'MINOR: [Component]'"

insertIssueLinkInPrDescription:
descriptionIssuePlaceholderRegexp: '^\* GitHub Issue: (.*)$'
descriptionIssuePlaceholderRegexp: '^\* GitHub Issue: (?!Refs )(.*)$'
matchers:
closedIssue:
targetBranchFilter: '^main$'
titleIssueIdRegexp: '^GH-([0-9]+)'
descriptionIssueLink: 'Closes #${1}'
Comment on lines +70 to +75

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do we want to test this config? I can do on my fork where I already configured boring-cyborg app.

githubIssue:
titleIssueIdRegexp: '^(?:WIP:?\s*)?GH-([0-9]+)'
descriptionIssueLink: '#${1}'
Expand Down
94 changes: 94 additions & 0 deletions .github/workflows/dev_pr_merged.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,94 @@
# Licensed to the Apache Software Foundation (ASF) under one
# or more contributor license agreements. See the NOTICE file
# distributed with this work for additional information
# regarding copyright ownership. The ASF licenses this file
# to you under the Apache License, Version 2.0 (the
# "License"); you may not use this file except in compliance
# with the License. You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing,
# software distributed under the License is distributed on an
# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
# KIND, either express or implied. See the License for the
# specific language governing permissions and limitations
# under the License.

name: Dev PR merged

on:
push:
branches:
- main

permissions:
contents: read
issues: write

jobs:
update:
name: Update issue
if: github.repository == 'apache/arrow'
runs-on: ubuntu-latest
steps:
# GitHub closes the issue of a "GH-XXX: ..." pull request on merge
# because of "* GitHub Issue: Closes #XXX" in the pull request
# description. This sets what GitHub doesn't set: the assignee
# and the milestone of the issue.
- name: Update issue
env:
AUTHOR: ${{ github.event.head_commit.author.username }}
GH_REPO: ${{ github.repository }}
GH_TOKEN: ${{ github.token }}
MESSAGE: ${{ github.event.head_commit.message }}
run: |
set -euo pipefail

title=${MESSAGE%%$'\n'*}
if [[ ! "${title}" =~ ^GH-([0-9]+) ]]; then
echo "No GitHub issue in the commit title: ${title}"
exit 0
fi
issue_number=${BASH_REMATCH[1]}
issue_path="repos/${GH_REPO}/issues/${issue_number}"
issue=$(gh api "${issue_path}")
if [ "$(jq 'has("pull_request")' <<< "${issue}")" = "true" ]; then
echo "#${issue_number} is a pull request, not an issue."
exit 0
fi
# The issue is open if the pull request description didn't
# have "Closes #XXX", for example for a partial fix.
if [ "$(jq -r .state <<< "${issue}")" != "closed" ]; then
echo "::notice::#${issue_number} is still open. It isn't updated."
exit 0
fi

# Assign the issue to the commit author, who is the pull
# request author for a squash merge, if it isn't assigned to
# anyone. GitHub ignores this silently if the author can't be
# assigned.
if [ -n "${AUTHOR}" ] && \
[ "$(jq '.assignees | length' <<< "${issue}")" -eq 0 ]; then
gh api "${issue_path}/assignees" -f "assignees[]=${AUTHOR}" > /dev/null
fi

# Keep the milestone of the issue. If the issue doesn't have
# a milestone, use the next major release that doesn't have a
# maintenance branch yet like dev/merge_arrow_pr.py does.
if [ "$(jq '.milestone' <<< "${issue}")" = "null" ]; then
maintenance_versions=$(
gh api --paginate "repos/${GH_REPO}/git/matching-refs/heads/maint-" \
--jq '.[].ref | ltrimstr("refs/heads/maint-")')
major_versions=$(
gh api --paginate "repos/${GH_REPO}/milestones?per_page=100" \
--jq '.[] | select(.title | test("^[0-9]+\\.0\\.0$")) | "\(.title):\(.number)"' |
sort -V)
for major_version in ${major_versions}; do
if ! grep -qxF "${major_version%:*}" <<< "${maintenance_versions}"; then
gh api -X PATCH "${issue_path}" \
-F "milestone=${major_version#*:}" > /dev/null
break
fi
done
fi
36 changes: 34 additions & 2 deletions dev/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -33,8 +33,40 @@ committer.

## How to Merge a Pull Request

Please don't merge PRs using the GitHub Web interface. Instead, run
the following command:
Use the "Squash and merge" button of the GitHub Web interface. Before
you merge, check that:

* The pull request title is what you want as the commit title:
`GH-XXX: [Component] Title` or `MINOR: [Component] Title`. The
"PR Title" status of the Boring Cyborg app
(`.github/boring-cyborg.yml`) fails unless the title starts with
`GH-XXX` or `MINOR: ` but it doesn't block the merge.
* The last line of the pull request description is
`* GitHub Issue: Closes #XXX`. The Boring Cyborg app writes it for a
pull request to `main` and GitHub closes the issue on merge because
of it. If the pull request fixes only a part of the issue, change
the line to `* GitHub Issue: Refs #XXX` before you merge: the issue
stays open. If the line doesn't say `Closes #XXX`, for example
because the app is down, change it or close the issue yourself.
* The pull request description is what you want as the commit
message. Remove leftovers of the pull request template in the
merge dialog if needed. You should remove CI directives such as
`[skip ci]`: GitHub doesn't run the workflow below for a commit
with them.

After the merge, the "Dev PR merged" GitHub Actions workflow reads
`GH-XXX` from the commit title. If GitHub closed the issue, the
workflow assigns it to the pull request author if it isn't assigned to
anyone and sets its milestone. The existing issue milestone is
retained: set it before merging if the pull request isn't targeted for
the next major release. If the issue doesn't have a milestone, the next
major release is used. This is only done for pull requests to `main`.

### Merge script

The merge script is still available for now. It additionally adds

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
The merge script is still available for now. It additionally adds
The merge script is still available for now but using the merge button is encouraged. It additionally adds

`Authored-by:`, `Co-authored-by:` and `Signed-off-by:` trailers to the
commit message. Run the following command:

```bash
dev/merge_arrow_pr.sh
Expand Down
33 changes: 29 additions & 4 deletions dev/release/post-04-website.sh
Original file line number Diff line number Diff line change
Expand Up @@ -75,10 +75,36 @@ rough_n_development_months=$((
git_tag=apache-arrow-${version}
git_range=apache-arrow-${previous_version}..${git_tag}

committers_command_line="git shortlog -sn --group=trailer:signed-off-by ${git_range}"
contributors_command_line="git shortlog -sn ${git_range}"

committers=$(${committers_command_line})
# Count the pull requests of the commits in ${git_range} by who merged
# them. We ask GitHub because commits merged with the merge button
# don't have "Signed-off-by:" trailers. A failed request stops the
# script: we must not publish a partial count.
list_committers() {

@rok rok Nov 18, 2026 •

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

New committer count proposal @raulcd

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is extremely hard to follow.

As the script already requires archery and uses it to talk to GitHub (archery release changelog generate), what about an archery release committers command? Something like this in dev/archery/archery/release/cli.py:
Have not tested the following, it's just claude:

# needs `import collections`, `import re` and `from git import Repo`
@release.command('committers')
@click.argument('previous_version')
@click.argument('version')
@click.pass_obj
def release_committers(obj, previous_version, version):
    """Count the pull requests of a release by who merged them."""
    repo = Repo(obj['repo'])
    github_repo = obj['issue_tracker'].github_repo
    commit_range = f"apache-arrow-{previous_version}..apache-arrow-{version}"

    counts = collections.Counter()
    for commit in repo.iter_commits(commit_range):
        match = re.search(r" \(#(\d+)\)$", commit.summary)
        if match is None:
            # Release commits and direct pushes have no PR number.
            continue
        pr_number = int(match.group(1))
        merged_by = github_repo.get_pull(pr_number).merged_by
        if merged_by is None:
            raise click.ClickException(f"#{pr_number} wasn't merged on GitHub")
        counts[merged_by.name or merged_by.login] += 1

    for name, count in sorted(counts.items(), key=lambda c: (-c[1], c[0])):
        click.echo(f"{count:6d}\t{name}")

and in post-04-website.sh:

committers=$(archery release committers ${previous_version} ${version})

If you prefer to keep it in bash, I think splitting it into named steps and dropping the batching would already help a lot:

pr_numbers() {
  git log --pretty=%s ${git_range} | sed -n -E 's/^.* \(#([0-9]+)\)$/\1/p'
}

merger() {
  gh api "repos/apache/arrow/pulls/$1" --jq '.merged_by.login // error("not merged")'
}

list_committers() {
  local number
  for number in $(pr_numbers); do
    merger "${number}" || return 1
  done | sort | uniq -c | sort -k1,1nr -k2
}

That one only shows logins, not names, getting names would need an extra call per user.

git log --pretty=%s ${git_range} |
sed -n -E 's/^.* \(#([0-9]+)\)$/\1/p' |
xargs -n 50 |
while read -r -a numbers; do
local query="query {repository(owner: \"apache\", name: \"arrow\") {"
local number
for number in "${numbers[@]}"; do
query+=" pr${number}: pullRequest(number: ${number})"
query+=" {mergedBy {login ... on User {name}}}"
done
query+="}}"
gh api graphql -f query="${query}" \
--jq '.data.repository[].mergedBy | select(. != null) |
if (.name // "") == "" then .login else .name end' ||
exit 1
done |
sort |
uniq -c |
sort -k1,1nr -k2 |
awk '{count = $1; sub(/^ *[0-9]+ /, ""); printf "%6d\t%s\n", count, $0}'
}

committers=$(set -o pipefail; list_committers)
contributors=$(${contributors_command_line})

n_commits=$(git log --pretty=oneline ${git_range} | wc -l)
Expand Down Expand Up @@ -151,8 +177,7 @@ cat <<ANNOUNCE >> "${announce_file}"

The following Apache committers merged contributed patches to the repository.

\`\`\`console
$ ${committers_command_line}
\`\`\`text
ANNOUNCE

echo "${committers}" >> "${announce_file}"
Expand Down
1 change: 1 addition & 0 deletions docs/source/developers/continuous_integration/overview.rst
Original file line number Diff line number Diff line change
Expand Up @@ -53,6 +53,7 @@ The ``.yml`` files in ``.github/workflows`` are workflows which are run on GitHu
- ``@github-actions crossbow submit ...`` - runs the specified Crossbow command
- ``@github-actions rebase`` - rebases the PR onto the main branch
- ``dev.yml`` - runs any time there is activity on a PR, or a PR is merged; it runs the linter and tests that the PR can be merged
- ``dev_pr_merged.yml`` - runs any time a PR is merged to the main branch; assigns the GitHub issue that the PR closed to the PR author if needed and sets its milestone

PR housekeeping (component labels, PR title validation and the GitHub issue link in the PR description) is handled by the `Boring Cyborg <https://github.com/kaxil/boring-cyborg>`_ GitHub App, configured in ``.github/boring-cyborg.yml``.

Expand Down
14 changes: 9 additions & 5 deletions docs/source/developers/overview.rst
Original file line number Diff line number Diff line change
Expand Up @@ -133,8 +133,13 @@ in the end. To make the review process smooth for everyone, try to

When tests are passing and the pull request has been approved by the interested
parties, a `committer <https://arrow.apache.org/committers/>`_
will merge the pull request. This is done with a
**command-line utility that does a squash merge**.
will merge the pull request. This is done with a **squash merge**.

The ``* GitHub Issue:`` line at the end of the pull request description is
filled in automatically from the pull request title. For a pull request to
``main`` it says ``Closes #XXX`` and GitHub closes the issue when the pull
request is merged. If your pull request fixes only a part of the issue, change
the line to ``* GitHub Issue: Refs #XXX`` so that the issue stays open.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should we add this as a comment on the PR template?


.. dropdown:: Details on squash merge
:animate: fade-in-slide-down
Expand All @@ -147,9 +152,8 @@ will merge the pull request. This is done with a
cherry-pick individual patches onto a maintenance branch.

Your pull request will appear in the GitHub interface to have been "merged".
In the commit message of that commit, the merge tool adds the pull request
description, a link back to the pull request, and attribution to the contributor
and any co-authors.
The commit message includes the pull request description, a link to the
pull request, and attribution to the contributor and any co-authors.

.. _ai-generated-code:

Expand Down
5 changes: 3 additions & 2 deletions docs/source/developers/release.rst
Original file line number Diff line number Diff line change
Expand Up @@ -105,7 +105,7 @@ assigned.
archery release curate <version>

Ensure a major version milestone for a follow up release is created on GitHub. This will
automatically be used by our merge script as the new version for issues closed when
automatically be used on merging pull requests as the new version for issues closed when
the maintenance branch is created.

Patch Releases
Expand Down Expand Up @@ -400,7 +400,8 @@ Be sure to go through on the following checklist:
:class-title: sd-fs-5
:class-container: sd-shadow-md

Add a release note for the new version to our website and update the latest release information:
Add a release note for the new version to our website and update the latest release information.
The script needs an authenticated ``gh`` to ask GitHub who merged the pull requests of the release:

.. code-block:: Bash

Expand Down
Loading