* [PATCH v3 0/8] Update release.sh
@ 2025-02-11 17:26 Andrey Albershteyn
2025-02-11 17:26 ` [PATCH v3 1/8] release.sh: add signing and fix outdated commands Andrey Albershteyn
` (7 more replies)
0 siblings, 8 replies; 17+ messages in thread
From: Andrey Albershteyn @ 2025-02-11 17:26 UTC (permalink / raw)
To: linux-xfs; +Cc: Darrick J. Wong, Andrey Albershteyn
To: linux-xfs@vger.kernel.org
The release.sh seems to be last updated in 2015. Every one seems to have
their own script to release. This patchset updates release.sh to do the
basic stuff as updating version files, committing them, creating tag,
creating release tarball, uploading tarball (optional), generate
for-next announce.
--
Andrey
---
Changes in v3:
- Add -f to generate for-next ANNOUNCE email
- Update for-next when new released is pushed
- Link to v2: https://lore.kernel.org/r/20250122-update-release-v2-0-d01529db3aa5@kernel.org
Changes in v2:
- Added git-contributors.py from Darrick
- A bit better/shorter commands for maintainer to run
- Link to v1: https://lore.kernel.org/r/20250110-update-release-v1-0-61e40b8ffbac@kernel.org
---
Andrey Albershteyn (8):
release.sh: add signing and fix outdated commands
release.sh: add --kup to upload release tarball to kernel.org
release.sh: update version files make commit optional
release.sh: generate ANNOUNCE email
Add git-contributors script to notify about merges
git-contributors: make revspec required and shebang fix
release.sh: use git-contributors to --cc contributors
release.sh: add -f to generate for-next update email
release.sh | 185 +++++++++++++++++++++++++++++++++++++++++++---
tools/git-contributors.py | 93 +++++++++++++++++++++++
2 files changed, 266 insertions(+), 12 deletions(-)
---
base-commit: eff7226942a59fc78e8ecd7577657c30ed0cf9a8
change-id: 20241111-update-release-13ef17c008a5
Best regards,
--
Andrey Albershteyn <aalbersh@kernel.org>
^ permalink raw reply [flat|nested] 17+ messages in thread
* [PATCH v3 1/8] release.sh: add signing and fix outdated commands
2025-02-11 17:26 [PATCH v3 0/8] Update release.sh Andrey Albershteyn
@ 2025-02-11 17:26 ` Andrey Albershteyn
2025-02-11 17:26 ` [PATCH v3 2/8] release.sh: add --kup to upload release tarball to kernel.org Andrey Albershteyn
` (6 subsequent siblings)
7 siblings, 0 replies; 17+ messages in thread
From: Andrey Albershteyn @ 2025-02-11 17:26 UTC (permalink / raw)
To: linux-xfs; +Cc: Darrick J. Wong, Andrey Albershteyn
Reviewed-by: Darrick J. Wong <djwong@kernel.org>
Signed-off-by: Andrey Albershteyn <aalbersh@kernel.org>
---
release.sh | 29 ++++++++++++++++++++++++-----
1 file changed, 24 insertions(+), 5 deletions(-)
diff --git a/release.sh b/release.sh
index 577257a354d442e1cc0a2b9381b11ffbe2f64a71..b15ed610082f34928827ab0547db944cf559cef4 100755
--- a/release.sh
+++ b/release.sh
@@ -9,6 +9,8 @@
# configure.ac (with new version string)
# debian/changelog (with new release entry, only for release version)
+set -e
+
. ./VERSION
version=${PKG_MAJOR}.${PKG_MINOR}.${PKG_REVISION}
@@ -16,21 +18,38 @@ date=`date +"%-d %B %Y"`
echo "Cleaning up"
make realclean
+rm -rf "xfsprogs-${version}.tar" \
+ "xfsprogs-${version}.tar.gz" \
+ "xfsprogs-${version}.tar.asc" \
+ "xfsprogs-${version}.tar.sign"
echo "Updating CHANGES"
sed -e "s/${version}.*/${version} (${date})/" doc/CHANGES > doc/CHANGES.tmp && \
mv doc/CHANGES.tmp doc/CHANGES
echo "Commiting CHANGES update to git"
-git commit -a -m "${version} release"
+git commit --all --signoff --message="xfsprogs: Release v${version}
+
+Update all the necessary files for a v${version} release."
echo "Tagging git repository"
-git tag -a -m "${version} release" v${version}
+git tag --annotate --sign --message="Release v${version}" v${version}
echo "Making source tarball"
make dist
+gunzip -k "xfsprogs-${version}.tar.gz"
-#echo "Sign the source tarball"
-#gpg --detach-sign xfsprogs-${version}.tar.gz
+echo "Sign the source tarball"
+gpg \
+ --detach-sign \
+ --armor \
+ "xfsprogs-${version}.tar"
-echo "Done. Please remember to push out tags using \"git push --tags\""
+echo "Verify signature"
+gpg \
+ --verify \
+ "xfsprogs-${version}.tar.asc"
+
+mv "xfsprogs-${version}.tar.asc" "xfsprogs-${version}.tar.sign"
+
+echo "Done. Please remember to push out tags using \"git push origin v${version}\""
--
2.47.2
^ permalink raw reply related [flat|nested] 17+ messages in thread
* [PATCH v3 2/8] release.sh: add --kup to upload release tarball to kernel.org
2025-02-11 17:26 [PATCH v3 0/8] Update release.sh Andrey Albershteyn
2025-02-11 17:26 ` [PATCH v3 1/8] release.sh: add signing and fix outdated commands Andrey Albershteyn
@ 2025-02-11 17:26 ` Andrey Albershteyn
2025-02-11 17:26 ` [PATCH v3 3/8] release.sh: update version files make commit optional Andrey Albershteyn
` (5 subsequent siblings)
7 siblings, 0 replies; 17+ messages in thread
From: Andrey Albershteyn @ 2025-02-11 17:26 UTC (permalink / raw)
To: linux-xfs; +Cc: Darrick J. Wong, Andrey Albershteyn
Add kup support so that the maintainer can push the newly formed
release tarballs to kernel.org.
Reviewed-by: Darrick J. Wong <djwong@kernel.org>
Signed-off-by: Andrey Albershteyn <aalbersh@kernel.org>
---
release.sh | 31 +++++++++++++++++++++++++++++++
1 file changed, 31 insertions(+)
diff --git a/release.sh b/release.sh
index b15ed610082f34928827ab0547db944cf559cef4..b036c3241b3f67bfb2435398e6a17ea4c6a6eebe 100755
--- a/release.sh
+++ b/release.sh
@@ -16,6 +16,30 @@ set -e
version=${PKG_MAJOR}.${PKG_MINOR}.${PKG_REVISION}
date=`date +"%-d %B %Y"`
+KUP=0
+
+help() {
+ echo "$(basename) - create xfsprogs release"
+ printf "\t[--kup|-k] upload final tarball with KUP\n"
+}
+
+while [ $# -gt 0 ]; do
+ case "$1" in
+ --kup|-k)
+ KUP=1
+ ;;
+ --help|-h)
+ help
+ exit 0
+ ;;
+ *)
+ >&2 printf "Error: Invalid argument\n"
+ exit 1
+ ;;
+ esac
+ shift
+done
+
echo "Cleaning up"
make realclean
rm -rf "xfsprogs-${version}.tar" \
@@ -52,4 +76,11 @@ gpg \
mv "xfsprogs-${version}.tar.asc" "xfsprogs-${version}.tar.sign"
+if [ $KUP -eq 1 ]; then
+ kup put \
+ xfsprogs-${version}.tar.gz \
+ xfsprogs-${version}.tar.sign \
+ pub/linux/utils/fs/xfs/xfsprogs/
+fi;
+
echo "Done. Please remember to push out tags using \"git push origin v${version}\""
--
2.47.2
^ permalink raw reply related [flat|nested] 17+ messages in thread
* [PATCH v3 3/8] release.sh: update version files make commit optional
2025-02-11 17:26 [PATCH v3 0/8] Update release.sh Andrey Albershteyn
2025-02-11 17:26 ` [PATCH v3 1/8] release.sh: add signing and fix outdated commands Andrey Albershteyn
2025-02-11 17:26 ` [PATCH v3 2/8] release.sh: add --kup to upload release tarball to kernel.org Andrey Albershteyn
@ 2025-02-11 17:26 ` Andrey Albershteyn
2025-02-11 17:26 ` [PATCH v3 4/8] release.sh: generate ANNOUNCE email Andrey Albershteyn
` (4 subsequent siblings)
7 siblings, 0 replies; 17+ messages in thread
From: Andrey Albershteyn @ 2025-02-11 17:26 UTC (permalink / raw)
To: linux-xfs; +Cc: Darrick J. Wong, Andrey Albershteyn
Based on ./VERSION script updates all other files. For
./doc/changelog script asks maintainer to fill it manually as not
all changes goes into changelog.
--no-commit|-n flag is handy when something got into the version commit
and need to be changed manually. Then ./release.sh -c will use fixed
history
Reviewed-by: Darrick J. Wong <djwong@kernel.org>
Signed-off-by: Andrey Albershteyn <aalbersh@kernel.org>
---
release.sh | 75 ++++++++++++++++++++++++++++++++++++++++++++++++--------------
1 file changed, 58 insertions(+), 17 deletions(-)
diff --git a/release.sh b/release.sh
index b036c3241b3f67bfb2435398e6a17ea4c6a6eebe..c3f74bc8dc587a40d867dd22e1e4c6e4aabb8997 100755
--- a/release.sh
+++ b/release.sh
@@ -11,16 +11,33 @@
set -e
-. ./VERSION
-
-version=${PKG_MAJOR}.${PKG_MINOR}.${PKG_REVISION}
-date=`date +"%-d %B %Y"`
-
KUP=0
+COMMIT=1
help() {
echo "$(basename) - create xfsprogs release"
printf "\t[--kup|-k] upload final tarball with KUP\n"
+ printf "\t[--no-commit|-n] don't create release commit\n"
+}
+
+update_version() {
+ echo "Updating version files"
+ # doc/CHANGES
+ header="xfsprogs-${version} ($(date +'%d %b %Y'))"
+ sed -i "1s/^/$header\n\t<TODO list user affecting changes>\n\n/" doc/CHANGES
+ $EDITOR doc/CHANGES
+
+ # ./configure.ac
+ CONF_AC="AC_INIT([xfsprogs],[${version}],[linux-xfs@vger.kernel.org])"
+ sed -i "s/^AC_INIT.*/$CONF_AC/" ./configure.ac
+
+ # ./debian/changelog
+ sed -i "1s/^/\n/" ./debian/changelog
+ sed -i "1s/^/ -- Nathan Scott <nathans@debian.org> `date -R`\n/" ./debian/changelog
+ sed -i "1s/^/\n/" ./debian/changelog
+ sed -i "1s/^/ * New upstream release\n/" ./debian/changelog
+ sed -i "1s/^/\n/" ./debian/changelog
+ sed -i "1s/^/xfsprogs (${version}-1) unstable; urgency=low\n/" ./debian/changelog
}
while [ $# -gt 0 ]; do
@@ -28,6 +45,9 @@ while [ $# -gt 0 ]; do
--kup|-k)
KUP=1
;;
+ --no-commit|-n)
+ COMMIT=0
+ ;;
--help|-h)
help
exit 0
@@ -40,6 +60,36 @@ while [ $# -gt 0 ]; do
shift
done
+if [ -z "$EDITOR" ]; then
+ EDITOR=$(command -v vi)
+fi
+
+if [ $COMMIT -eq 1 ]; then
+ if git diff --exit-code ./VERSION > /dev/null; then
+ $EDITOR ./VERSION
+ fi
+fi
+
+. ./VERSION
+
+version=${PKG_MAJOR}.${PKG_MINOR}.${PKG_REVISION}
+date=`date +"%-d %B %Y"`
+
+if [ $COMMIT -eq 1 ]; then
+ update_version
+
+ git diff --color=always | less -r
+ [[ "$(read -e -p 'All good? [Y/n]> '; echo $REPLY)" == [Nn]* ]] && exit 0
+
+ echo "Commiting new version update to git"
+ git commit --all --signoff --message="xfsprogs: Release v${version}
+
+Update all the necessary files for a v${version} release."
+
+ echo "Tagging git repository"
+ git tag --annotate --sign --message="Release v${version}" v${version}
+fi
+
echo "Cleaning up"
make realclean
rm -rf "xfsprogs-${version}.tar" \
@@ -47,17 +97,6 @@ rm -rf "xfsprogs-${version}.tar" \
"xfsprogs-${version}.tar.asc" \
"xfsprogs-${version}.tar.sign"
-echo "Updating CHANGES"
-sed -e "s/${version}.*/${version} (${date})/" doc/CHANGES > doc/CHANGES.tmp && \
- mv doc/CHANGES.tmp doc/CHANGES
-
-echo "Commiting CHANGES update to git"
-git commit --all --signoff --message="xfsprogs: Release v${version}
-
-Update all the necessary files for a v${version} release."
-
-echo "Tagging git repository"
-git tag --annotate --sign --message="Release v${version}" v${version}
echo "Making source tarball"
make dist
@@ -83,4 +122,6 @@ if [ $KUP -eq 1 ]; then
pub/linux/utils/fs/xfs/xfsprogs/
fi;
-echo "Done. Please remember to push out tags using \"git push origin v${version}\""
+echo ""
+echo "Done. Please remember to push out tags and the branch."
+printf "\tgit push origin v${version} master:master master:for-next\n"
--
2.47.2
^ permalink raw reply related [flat|nested] 17+ messages in thread
* [PATCH v3 4/8] release.sh: generate ANNOUNCE email
2025-02-11 17:26 [PATCH v3 0/8] Update release.sh Andrey Albershteyn
` (2 preceding siblings ...)
2025-02-11 17:26 ` [PATCH v3 3/8] release.sh: update version files make commit optional Andrey Albershteyn
@ 2025-02-11 17:26 ` Andrey Albershteyn
2025-02-11 17:26 ` [PATCH v3 5/8] Add git-contributors script to notify about merges Andrey Albershteyn
` (3 subsequent siblings)
7 siblings, 0 replies; 17+ messages in thread
From: Andrey Albershteyn @ 2025-02-11 17:26 UTC (permalink / raw)
To: linux-xfs; +Cc: Darrick J. Wong, Andrey Albershteyn
Reviewed-by: Darrick J. Wong <djwong@kernel.org>
Signed-off-by: Andrey Albershteyn <aalbersh@kernel.org>
---
release.sh | 43 +++++++++++++++++++++++++++++++++++++++++++
1 file changed, 43 insertions(+)
diff --git a/release.sh b/release.sh
index c3f74bc8dc587a40d867dd22e1e4c6e4aabb8997..a2afb98c2391c418a1b1d8537ea9f7a2f5138c1e 100755
--- a/release.sh
+++ b/release.sh
@@ -13,11 +13,13 @@ set -e
KUP=0
COMMIT=1
+LAST_HEAD=""
help() {
echo "$(basename) - create xfsprogs release"
printf "\t[--kup|-k] upload final tarball with KUP\n"
printf "\t[--no-commit|-n] don't create release commit\n"
+ printf "\t[--last-head|-l] commit of the last release\n"
}
update_version() {
@@ -48,6 +50,10 @@ while [ $# -gt 0 ]; do
--no-commit|-n)
COMMIT=0
;;
+ --last-head|-l)
+ LAST_HEAD=$2
+ shift
+ ;;
--help|-h)
help
exit 0
@@ -122,6 +128,43 @@ if [ $KUP -eq 1 ]; then
pub/linux/utils/fs/xfs/xfsprogs/
fi;
+mail_file=$(mktemp)
+if [ -n "$LAST_HEAD" ]; then
+ cat << EOF > $mail_file
+To: linux-xfs@vger.kernel.org
+Subject: [ANNOUNCE] xfsprogs $(git describe --abbrev=0) released
+
+Hi folks,
+
+The xfsprogs repository at:
+
+ git://git.kernel.org/pub/scm/fs/xfs/xfsprogs-dev.git
+
+has just been updated.
+
+Patches often get missed, so if your outstanding patches are properly reviewed
+on the list and not included in this update, please let me know.
+
+The for-next branch has also been updated to match the state of master.
+
+The new head of the master branch is commit:
+
+$(git log --oneline --format="%H" -1)
+
+New commits:
+
+$(git shortlog --format="[%h] %s" $LAST_HEAD..HEAD)
+
+Code Diffstat:
+
+$(git diff --stat --summary -C -M $LAST_HEAD..HEAD)
+EOF
+fi
+
echo ""
echo "Done. Please remember to push out tags and the branch."
printf "\tgit push origin v${version} master:master master:for-next\n"
+if [ -n "$LAST_HEAD" ]; then
+ echo "Command to send ANNOUNCE email"
+ printf "\tneomutt -H $mail_file\n"
+fi
--
2.47.2
^ permalink raw reply related [flat|nested] 17+ messages in thread
* [PATCH v3 5/8] Add git-contributors script to notify about merges
2025-02-11 17:26 [PATCH v3 0/8] Update release.sh Andrey Albershteyn
` (3 preceding siblings ...)
2025-02-11 17:26 ` [PATCH v3 4/8] release.sh: generate ANNOUNCE email Andrey Albershteyn
@ 2025-02-11 17:26 ` Andrey Albershteyn
2025-02-11 18:58 ` Darrick J. Wong
2025-02-11 17:26 ` [PATCH v3 6/8] git-contributors: make revspec required and shebang fix Andrey Albershteyn
` (2 subsequent siblings)
7 siblings, 1 reply; 17+ messages in thread
From: Andrey Albershteyn @ 2025-02-11 17:26 UTC (permalink / raw)
To: linux-xfs; +Cc: Darrick J. Wong, Andrey Albershteyn
Add python script used to collect emails over all changes merged in
the next release.
CC: Darrick J. Wong <djwong@kernel.org>
Signed-off-by: Darrick J. Wong <djwong@kernel.org>
Reviewed-by: Darrick J. Wong <djwong@kernel.org>
Signed-off-by: Andrey Albershteyn <aalbersh@kernel.org>
---
tools/git-contributors.py | 94 +++++++++++++++++++++++++++++++++++++++++++++++
1 file changed, 94 insertions(+)
diff --git a/tools/git-contributors.py b/tools/git-contributors.py
new file mode 100755
index 0000000000000000000000000000000000000000..83bbe8ce0ee1dcbd591c6d3016d553fac2a7d286
--- /dev/null
+++ b/tools/git-contributors.py
@@ -0,0 +1,94 @@
+#!/usr/bin/python3
+
+# List all contributors to a series of git commits.
+# Copyright(C) 2025 Oracle, All Rights Reserved.
+# Licensed under GPL 2.0 or later
+
+import re
+import subprocess
+import io
+import sys
+import argparse
+import email.utils
+
+DEBUG = False
+
+def backtick(args):
+ '''Generator function that yields lines of a program's stdout.'''
+ if DEBUG:
+ print(' '.join(args))
+ p = subprocess.Popen(args, stdout = subprocess.PIPE)
+ for line in io.TextIOWrapper(p.stdout, encoding="utf-8"):
+ yield line
+
+class find_developers(object):
+ def __init__(self):
+ tags = '%s|%s|%s|%s|%s|%s|%s|%s' % (
+ 'signed-off-by',
+ 'acked-by',
+ 'cc',
+ 'reviewed-by',
+ 'reported-by',
+ 'tested-by',
+ 'suggested-by',
+ 'reported-and-tested-by')
+ # some tag, a colon, a space, and everything after that
+ regex1 = r'^(%s):\s+(.+)$' % tags
+
+ self.r1 = re.compile(regex1, re.I)
+
+ def run(self, lines):
+ addr_list = []
+
+ for line in lines:
+ l = line.strip()
+
+ # emailutils can handle abominations like:
+ #
+ # Reviewed-by: Bogus J. Simpson <bogus@simpson.com>
+ # Reviewed-by: "Bogus J. Simpson" <bogus@simpson.com>
+ # Reviewed-by: bogus@simpson.com
+ # Cc: <stable@vger.kernel.org> # v6.9
+ # Tested-by: Moo Cow <foo@bar.com> # powerpc
+ m = self.r1.match(l)
+ if not m:
+ continue
+ (name, addr) = email.utils.parseaddr(m.expand(r'\g<2>'))
+
+ # This last split removes anything after a hash mark,
+ # because someone could have provided an improperly
+ # formatted email address:
+ #
+ # Cc: stable@vger.kernel.org # v6.19+
+ #
+ # emailutils doesn't seem to catch this, and I can't
+ # fully tell from RFC2822 that this isn't allowed. I
+ # think it is because dtext doesn't forbid spaces or
+ # hash marks.
+ addr_list.append(addr.split('#')[0])
+
+ return sorted(set(addr_list))
+
+def main():
+ parser = argparse.ArgumentParser(description = "List email addresses of contributors to a series of git commits.")
+ parser.add_argument("revspec", nargs = '?', default = None, \
+ help = "git revisions to process.")
+ parser.add_argument("--delimiter", type = str, default = '\n', \
+ help = "Separate each email address with this string.")
+ args = parser.parse_args()
+
+ fd = find_developers()
+ if args.revspec:
+ # read git commits from repo
+ contributors = fd.run(backtick(['git', 'log', '--pretty=medium',
+ args.revspec]))
+ else:
+ # read patch from stdin
+ contributors = fd.run(sys.stdin.readlines())
+
+ print(args.delimiter.join(sorted(contributors)))
+ return 0
+
+if __name__ == '__main__':
+ sys.exit(main())
+
--
2.47.2
^ permalink raw reply related [flat|nested] 17+ messages in thread
* [PATCH v3 6/8] git-contributors: make revspec required and shebang fix
2025-02-11 17:26 [PATCH v3 0/8] Update release.sh Andrey Albershteyn
` (4 preceding siblings ...)
2025-02-11 17:26 ` [PATCH v3 5/8] Add git-contributors script to notify about merges Andrey Albershteyn
@ 2025-02-11 17:26 ` Andrey Albershteyn
2025-02-11 18:59 ` Darrick J. Wong
2025-02-11 17:26 ` [PATCH v3 7/8] release.sh: use git-contributors to --cc contributors Andrey Albershteyn
2025-02-11 17:27 ` [PATCH v3 8/8] release.sh: add -f to generate for-next update email Andrey Albershteyn
7 siblings, 1 reply; 17+ messages in thread
From: Andrey Albershteyn @ 2025-02-11 17:26 UTC (permalink / raw)
To: linux-xfs; +Cc: Andrey Albershteyn
Without default value script will show help instead of just hanging
waiting for input on stdin.
Shebang fix for system with different python location than the
/usr/bin one.
Signed-off-by: Andrey Albershteyn <aalbersh@kernel.org>
---
tools/git-contributors.py | 5 ++---
1 file changed, 2 insertions(+), 3 deletions(-)
diff --git a/tools/git-contributors.py b/tools/git-contributors.py
index 83bbe8ce0ee1dcbd591c6d3016d553fac2a7d286..628d6d0b4d8795e10b1317fa6fc91c6b98b21f3e 100755
--- a/tools/git-contributors.py
+++ b/tools/git-contributors.py
@@ -1,4 +1,4 @@
-#!/usr/bin/python3
+#!/usr/bin/env python3
# List all contributors to a series of git commits.
# Copyright(C) 2025 Oracle, All Rights Reserved.
@@ -71,8 +71,7 @@ class find_developers(object):
def main():
parser = argparse.ArgumentParser(description = "List email addresses of contributors to a series of git commits.")
- parser.add_argument("revspec", nargs = '?', default = None, \
- help = "git revisions to process.")
+ parser.add_argument("revspec", help = "git revisions to process.")
parser.add_argument("--delimiter", type = str, default = '\n', \
help = "Separate each email address with this string.")
args = parser.parse_args()
--
2.47.2
^ permalink raw reply related [flat|nested] 17+ messages in thread
* [PATCH v3 7/8] release.sh: use git-contributors to --cc contributors
2025-02-11 17:26 [PATCH v3 0/8] Update release.sh Andrey Albershteyn
` (5 preceding siblings ...)
2025-02-11 17:26 ` [PATCH v3 6/8] git-contributors: make revspec required and shebang fix Andrey Albershteyn
@ 2025-02-11 17:26 ` Andrey Albershteyn
2025-02-11 17:58 ` Darrick J. Wong
2025-02-11 17:27 ` [PATCH v3 8/8] release.sh: add -f to generate for-next update email Andrey Albershteyn
7 siblings, 1 reply; 17+ messages in thread
From: Andrey Albershteyn @ 2025-02-11 17:26 UTC (permalink / raw)
To: linux-xfs; +Cc: Andrey Albershteyn
Signed-off-by: Andrey Albershteyn <aalbersh@kernel.org>
---
release.sh | 1 +
1 file changed, 1 insertion(+)
diff --git a/release.sh b/release.sh
index a2afb98c2391c418a1b1d8537ea9f7a2f5138c1e..3d272aebdb1fe7a3b47689b9dc129a26d6a9eb20 100755
--- a/release.sh
+++ b/release.sh
@@ -132,6 +132,7 @@ mail_file=$(mktemp)
if [ -n "$LAST_HEAD" ]; then
cat << EOF > $mail_file
To: linux-xfs@vger.kernel.org
+Cc: $(./tools/git-contributors.py $LAST_HEAD.. --delimiter ' ')
Subject: [ANNOUNCE] xfsprogs $(git describe --abbrev=0) released
Hi folks,
--
2.47.2
^ permalink raw reply related [flat|nested] 17+ messages in thread
* [PATCH v3 8/8] release.sh: add -f to generate for-next update email
2025-02-11 17:26 [PATCH v3 0/8] Update release.sh Andrey Albershteyn
` (6 preceding siblings ...)
2025-02-11 17:26 ` [PATCH v3 7/8] release.sh: use git-contributors to --cc contributors Andrey Albershteyn
@ 2025-02-11 17:27 ` Andrey Albershteyn
2025-02-11 19:09 ` Darrick J. Wong
7 siblings, 1 reply; 17+ messages in thread
From: Andrey Albershteyn @ 2025-02-11 17:27 UTC (permalink / raw)
To: linux-xfs; +Cc: Andrey Albershteyn
Add --for-next/-f to generate ANNOUNCE email for for-next branch
update. This doesn't require new commit/tarball/tags, so skip it.
Signed-off-by: Andrey Albershteyn <aalbersh@kernel.org>
---
release.sh | 92 ++++++++++++++++++++++++++++++++++++++++----------------------
1 file changed, 59 insertions(+), 33 deletions(-)
diff --git a/release.sh b/release.sh
index 3d272aebdb1fe7a3b47689b9dc129a26d6a9eb20..863c2c5e061b73e232c0bb01e879a115b6dd55bb 100755
--- a/release.sh
+++ b/release.sh
@@ -14,12 +14,14 @@ set -e
KUP=0
COMMIT=1
LAST_HEAD=""
+FOR_NEXT=0
help() {
echo "$(basename) - create xfsprogs release"
printf "\t[--kup|-k] upload final tarball with KUP\n"
printf "\t[--no-commit|-n] don't create release commit\n"
printf "\t[--last-head|-l] commit of the last release\n"
+ printf "\t[--for-next|-f] generate announce email for for-next update\n"
}
update_version() {
@@ -42,6 +44,48 @@ update_version() {
sed -i "1s/^/xfsprogs (${version}-1) unstable; urgency=low\n/" ./debian/changelog
}
+prepare_mail() {
+ branch="$1"
+ mail_file=$(mktemp)
+ if [ -n "$LAST_HEAD" ]; then
+ if [ $branch == "master" ]; then
+ reason="$(git describe --abbrev=0 $branch) released"
+ else
+ reason="for-next updated to $(git log --oneline --format="%h" -1 $branch)"
+ fi;
+ cat << EOF > $mail_file
+To: linux-xfs@vger.kernel.org
+Cc: $(./tools/git-contributors.py $LAST_HEAD..$branch --delimiter ' ')
+Subject: [ANNOUNCE] xfsprogs: $reason
+
+Hi folks,
+
+The xfsprogs $branch branch in repository at:
+
+ git://git.kernel.org/pub/scm/fs/xfs/xfsprogs-dev.git
+
+has just been updated.
+
+Patches often get missed, so if your outstanding patches are properly reviewed
+on the list and not included in this update, please let me know.
+
+The for-next branch has also been updated to match the state of master.
+
+The new head of the $branch branch is commit:
+
+$(git log --oneline --format="%H" -1 $branch)
+
+New commits:
+
+$(git shortlog --format="[%h] %s" $LAST_HEAD..$branch)
+
+Code Diffstat:
+
+$(git diff --stat --summary -C -M $LAST_HEAD..$branch)
+EOF
+ fi
+}
+
while [ $# -gt 0 ]; do
case "$1" in
--kup|-k)
@@ -54,6 +98,9 @@ while [ $# -gt 0 ]; do
LAST_HEAD=$2
shift
;;
+ --for-next|-f)
+ FOR_NEXT=1
+ ;;
--help|-h)
help
exit 0
@@ -66,6 +113,17 @@ while [ $# -gt 0 ]; do
shift
done
+if [ $FOR_NEXT -eq 1 ]; then
+ echo "Push your for-next branch:"
+ printf "\tgit push origin for-next:for-next\n"
+ prepare_mail "for-next"
+ if [ -n "$LAST_HEAD" ]; then
+ echo "Command to send ANNOUNCE email"
+ printf "\tneomutt -H $mail_file\n"
+ fi
+ exit 0
+fi
+
if [ -z "$EDITOR" ]; then
EDITOR=$(command -v vi)
fi
@@ -128,39 +186,7 @@ if [ $KUP -eq 1 ]; then
pub/linux/utils/fs/xfs/xfsprogs/
fi;
-mail_file=$(mktemp)
-if [ -n "$LAST_HEAD" ]; then
- cat << EOF > $mail_file
-To: linux-xfs@vger.kernel.org
-Cc: $(./tools/git-contributors.py $LAST_HEAD.. --delimiter ' ')
-Subject: [ANNOUNCE] xfsprogs $(git describe --abbrev=0) released
-
-Hi folks,
-
-The xfsprogs repository at:
-
- git://git.kernel.org/pub/scm/fs/xfs/xfsprogs-dev.git
-
-has just been updated.
-
-Patches often get missed, so if your outstanding patches are properly reviewed
-on the list and not included in this update, please let me know.
-
-The for-next branch has also been updated to match the state of master.
-
-The new head of the master branch is commit:
-
-$(git log --oneline --format="%H" -1)
-
-New commits:
-
-$(git shortlog --format="[%h] %s" $LAST_HEAD..HEAD)
-
-Code Diffstat:
-
-$(git diff --stat --summary -C -M $LAST_HEAD..HEAD)
-EOF
-fi
+prepare_mail "master"
echo ""
echo "Done. Please remember to push out tags and the branch."
--
2.47.2
^ permalink raw reply related [flat|nested] 17+ messages in thread
* Re: [PATCH v3 7/8] release.sh: use git-contributors to --cc contributors
2025-02-11 17:26 ` [PATCH v3 7/8] release.sh: use git-contributors to --cc contributors Andrey Albershteyn
@ 2025-02-11 17:58 ` Darrick J. Wong
0 siblings, 0 replies; 17+ messages in thread
From: Darrick J. Wong @ 2025-02-11 17:58 UTC (permalink / raw)
To: Andrey Albershteyn; +Cc: linux-xfs, Andrey Albershteyn
On Tue, Feb 11, 2025 at 06:26:59PM +0100, Andrey Albershteyn wrote:
> Signed-off-by: Andrey Albershteyn <aalbersh@kernel.org>
> ---
> release.sh | 1 +
> 1 file changed, 1 insertion(+)
>
> diff --git a/release.sh b/release.sh
> index a2afb98c2391c418a1b1d8537ea9f7a2f5138c1e..3d272aebdb1fe7a3b47689b9dc129a26d6a9eb20 100755
> --- a/release.sh
> +++ b/release.sh
> @@ -132,6 +132,7 @@ mail_file=$(mktemp)
> if [ -n "$LAST_HEAD" ]; then
> cat << EOF > $mail_file
> To: linux-xfs@vger.kernel.org
> +Cc: $(./tools/git-contributors.py $LAST_HEAD.. --delimiter ' ')
I think you want a delimiter of ',' here?
Cc: foo@bar.com, gruk@micorsoft.com
--D
> Subject: [ANNOUNCE] xfsprogs $(git describe --abbrev=0) released
>
> Hi folks,
>
> --
> 2.47.2
>
>
^ permalink raw reply [flat|nested] 17+ messages in thread
* Re: [PATCH v3 5/8] Add git-contributors script to notify about merges
2025-02-11 17:26 ` [PATCH v3 5/8] Add git-contributors script to notify about merges Andrey Albershteyn
@ 2025-02-11 18:58 ` Darrick J. Wong
2025-02-12 11:16 ` Andrey Albershteyn
0 siblings, 1 reply; 17+ messages in thread
From: Darrick J. Wong @ 2025-02-11 18:58 UTC (permalink / raw)
To: Andrey Albershteyn; +Cc: linux-xfs, Andrey Albershteyn
On Tue, Feb 11, 2025 at 06:26:57PM +0100, Andrey Albershteyn wrote:
> Add python script used to collect emails over all changes merged in
> the next release.
>
> CC: Darrick J. Wong <djwong@kernel.org>
> Signed-off-by: Darrick J. Wong <djwong@kernel.org>
> Reviewed-by: Darrick J. Wong <djwong@kernel.org>
> Signed-off-by: Andrey Albershteyn <aalbersh@kernel.org>
> ---
> tools/git-contributors.py | 94 +++++++++++++++++++++++++++++++++++++++++++++++
> 1 file changed, 94 insertions(+)
>
> diff --git a/tools/git-contributors.py b/tools/git-contributors.py
> new file mode 100755
> index 0000000000000000000000000000000000000000..83bbe8ce0ee1dcbd591c6d3016d553fac2a7d286
> --- /dev/null
> +++ b/tools/git-contributors.py
> @@ -0,0 +1,94 @@
> +#!/usr/bin/python3
> +
> +# List all contributors to a series of git commits.
> +# Copyright(C) 2025 Oracle, All Rights Reserved.
> +# Licensed under GPL 2.0 or later
> +
> +import re
> +import subprocess
> +import io
> +import sys
> +import argparse
> +import email.utils
> +
> +DEBUG = False
> +
> +def backtick(args):
> + '''Generator function that yields lines of a program's stdout.'''
> + if DEBUG:
> + print(' '.join(args))
> + p = subprocess.Popen(args, stdout = subprocess.PIPE)
> + for line in io.TextIOWrapper(p.stdout, encoding="utf-8"):
> + yield line
> +
> +class find_developers(object):
> + def __init__(self):
> + tags = '%s|%s|%s|%s|%s|%s|%s|%s' % (
> + 'signed-off-by',
> + 'acked-by',
> + 'cc',
> + 'reviewed-by',
> + 'reported-by',
> + 'tested-by',
> + 'suggested-by',
> + 'reported-and-tested-by')
> + # some tag, a colon, a space, and everything after that
> + regex1 = r'^(%s):\s+(.+)$' % tags
> +
> + self.r1 = re.compile(regex1, re.I)
> +
> + def run(self, lines):
> + addr_list = []
> +
> + for line in lines:
> + l = line.strip()
> +
> + # emailutils can handle abominations like:
> + #
> + # Reviewed-by: Bogus J. Simpson <bogus@simpson.com>
> + # Reviewed-by: "Bogus J. Simpson" <bogus@simpson.com>
> + # Reviewed-by: bogus@simpson.com
> + # Cc: <stable@vger.kernel.org> # v6.9
> + # Tested-by: Moo Cow <foo@bar.com> # powerpc
> + m = self.r1.match(l)
> + if not m:
> + continue
> + (name, addr) = email.utils.parseaddr(m.expand(r'\g<2>'))
> +
> + # This last split removes anything after a hash mark,
> + # because someone could have provided an improperly
> + # formatted email address:
> + #
> + # Cc: stable@vger.kernel.org # v6.19+
> + #
> + # emailutils doesn't seem to catch this, and I can't
> + # fully tell from RFC2822 that this isn't allowed. I
> + # think it is because dtext doesn't forbid spaces or
> + # hash marks.
> + addr_list.append(addr.split('#')[0])
I think it's the case that the canonical stable cc tag format for kernel
patches as provided by the stable kernel process rules document:
Cc: <stable@vger.kernel.org> # vX.Y
is not actually actually rfc5322 compliant, so strings like that break
Python's emailutils parsers. parseaddr() completely chokes on this, and
retuns name=='' and addr=='', because the only thing that can come after
the address portion are whitespace, EOL, or a comma followed by more
email addresses. There's definitely not supposed to be an octothorpe
followed by even more text.
In the end I let myself be nerdsniped with even more string parsing bs,
and this loop body is the result:
l = line.strip()
# First, does this line match any of the headers we
# know about?
m = self.r1.match(l)
if not m:
continue
# The split removes everything after an octothorpe
# (hash mark), because someone could have provided an
# improperly formatted email address:
#
# Cc: stable@vger.kernel.org # v6.19+
#
# This, according to my reading of RFC5322, is allowed
# because octothorpes can be part of atom text.
# However, it is interepreted as if there weren't any
# whitespace ("stable@vger.kernel.org#v6.19+"). The
# grammar allows for this form, even though this is not
# a correct Internet domain name.
#
# Worse, if you follow the format specified in the
# kernel's SubmittingPatches file:
#
# Cc: <stable@vger.kernel.org> # v6.9
#
# emailutils will not know how to parse this, and
# returns empty strings. I think this is because the
# angle-addr specification allows only whitespace
# between the closing angle bracket and the CRLF.
#
# Hack around both problems by ignoring everything
# after an octothorpe, no matter where it occurs in the
# string. If someone has one in their name or the
# email address, too bad.
a = m.expand(r'\g<2>').split('#')[0]
# emailutils can extract email addresses from headers
# that roughly follow the destination address field
# format:
#
# Reviewed-by: Bogus J. Simpson <bogus@simpson.com>
# Reviewed-by: "Bogus J. Simpson" <bogus@simpson.com>
# Reviewed-by: bogus@simpson.com
# Tested-by: Moo Cow <foo@bar.com>
#
# Use it to extract the email address, because we don't
# care about the display name.
(name, addr) = email.utils.parseaddr(a)
addr_list.append(addr)
<shrug> but maybe we should try that on a few branches first before
committing to this string parsing mess ... ? Not that this is any less
stupid than the previous version I shared out. :(
--D
> +
> + return sorted(set(addr_list))
> +
> +def main():
> + parser = argparse.ArgumentParser(description = "List email addresses of contributors to a series of git commits.")
> + parser.add_argument("revspec", nargs = '?', default = None, \
> + help = "git revisions to process.")
> + parser.add_argument("--delimiter", type = str, default = '\n', \
> + help = "Separate each email address with this string.")
> + args = parser.parse_args()
> +
> + fd = find_developers()
> + if args.revspec:
> + # read git commits from repo
> + contributors = fd.run(backtick(['git', 'log', '--pretty=medium',
> + args.revspec]))
> + else:
> + # read patch from stdin
> + contributors = fd.run(sys.stdin.readlines())
> +
> + print(args.delimiter.join(sorted(contributors)))
> + return 0
> +
> +if __name__ == '__main__':
> + sys.exit(main())
> +
>
> --
> 2.47.2
>
>
^ permalink raw reply [flat|nested] 17+ messages in thread
* Re: [PATCH v3 6/8] git-contributors: make revspec required and shebang fix
2025-02-11 17:26 ` [PATCH v3 6/8] git-contributors: make revspec required and shebang fix Andrey Albershteyn
@ 2025-02-11 18:59 ` Darrick J. Wong
0 siblings, 0 replies; 17+ messages in thread
From: Darrick J. Wong @ 2025-02-11 18:59 UTC (permalink / raw)
To: Andrey Albershteyn; +Cc: linux-xfs, Andrey Albershteyn
On Tue, Feb 11, 2025 at 06:26:58PM +0100, Andrey Albershteyn wrote:
> Without default value script will show help instead of just hanging
> waiting for input on stdin.
>
> Shebang fix for system with different python location than the
> /usr/bin one.
>
> Signed-off-by: Andrey Albershteyn <aalbersh@kernel.org>
> ---
> tools/git-contributors.py | 5 ++---
> 1 file changed, 2 insertions(+), 3 deletions(-)
>
> diff --git a/tools/git-contributors.py b/tools/git-contributors.py
> index 83bbe8ce0ee1dcbd591c6d3016d553fac2a7d286..628d6d0b4d8795e10b1317fa6fc91c6b98b21f3e 100755
> --- a/tools/git-contributors.py
> +++ b/tools/git-contributors.py
> @@ -1,4 +1,4 @@
> -#!/usr/bin/python3
> +#!/usr/bin/env python3
>
> # List all contributors to a series of git commits.
> # Copyright(C) 2025 Oracle, All Rights Reserved.
> @@ -71,8 +71,7 @@ class find_developers(object):
>
> def main():
> parser = argparse.ArgumentParser(description = "List email addresses of contributors to a series of git commits.")
> - parser.add_argument("revspec", nargs = '?', default = None, \
> - help = "git revisions to process.")
> + parser.add_argument("revspec", help = "git revisions to process.")
Might want to get rid of the "read patch from stdin" code if you're not
going to accept that anymore.
--D
> parser.add_argument("--delimiter", type = str, default = '\n', \
> help = "Separate each email address with this string.")
> args = parser.parse_args()
>
> --
> 2.47.2
>
>
^ permalink raw reply [flat|nested] 17+ messages in thread
* Re: [PATCH v3 8/8] release.sh: add -f to generate for-next update email
2025-02-11 17:27 ` [PATCH v3 8/8] release.sh: add -f to generate for-next update email Andrey Albershteyn
@ 2025-02-11 19:09 ` Darrick J. Wong
0 siblings, 0 replies; 17+ messages in thread
From: Darrick J. Wong @ 2025-02-11 19:09 UTC (permalink / raw)
To: Andrey Albershteyn; +Cc: linux-xfs, Andrey Albershteyn
On Tue, Feb 11, 2025 at 06:27:00PM +0100, Andrey Albershteyn wrote:
> Add --for-next/-f to generate ANNOUNCE email for for-next branch
> update. This doesn't require new commit/tarball/tags, so skip it.
>
> Signed-off-by: Andrey Albershteyn <aalbersh@kernel.org>
> ---
> release.sh | 92 ++++++++++++++++++++++++++++++++++++++++----------------------
> 1 file changed, 59 insertions(+), 33 deletions(-)
>
> diff --git a/release.sh b/release.sh
> index 3d272aebdb1fe7a3b47689b9dc129a26d6a9eb20..863c2c5e061b73e232c0bb01e879a115b6dd55bb 100755
> --- a/release.sh
> +++ b/release.sh
> @@ -14,12 +14,14 @@ set -e
> KUP=0
> COMMIT=1
> LAST_HEAD=""
> +FOR_NEXT=0
>
> help() {
> echo "$(basename) - create xfsprogs release"
> printf "\t[--kup|-k] upload final tarball with KUP\n"
> printf "\t[--no-commit|-n] don't create release commit\n"
> printf "\t[--last-head|-l] commit of the last release\n"
> + printf "\t[--for-next|-f] generate announce email for for-next update\n"
> }
>
> update_version() {
> @@ -42,6 +44,48 @@ update_version() {
> sed -i "1s/^/xfsprogs (${version}-1) unstable; urgency=low\n/" ./debian/changelog
> }
>
> +prepare_mail() {
Hoisting this to a function probably ought to be done in the patch that
adds the email message body creation in the first place, so that the
changes for --for-next are more obvious.
The end result looks ok at least, so
Reviewed-by: "Darrick J. Wong" <djwong@kernel.org>
--D
> + branch="$1"
> + mail_file=$(mktemp)
> + if [ -n "$LAST_HEAD" ]; then
> + if [ $branch == "master" ]; then
> + reason="$(git describe --abbrev=0 $branch) released"
> + else
> + reason="for-next updated to $(git log --oneline --format="%h" -1 $branch)"
> + fi;
> + cat << EOF > $mail_file
> +To: linux-xfs@vger.kernel.org
> +Cc: $(./tools/git-contributors.py $LAST_HEAD..$branch --delimiter ' ')
> +Subject: [ANNOUNCE] xfsprogs: $reason
> +
> +Hi folks,
> +
> +The xfsprogs $branch branch in repository at:
> +
> + git://git.kernel.org/pub/scm/fs/xfs/xfsprogs-dev.git
> +
> +has just been updated.
> +
> +Patches often get missed, so if your outstanding patches are properly reviewed
> +on the list and not included in this update, please let me know.
> +
> +The for-next branch has also been updated to match the state of master.
> +
> +The new head of the $branch branch is commit:
> +
> +$(git log --oneline --format="%H" -1 $branch)
> +
> +New commits:
> +
> +$(git shortlog --format="[%h] %s" $LAST_HEAD..$branch)
> +
> +Code Diffstat:
> +
> +$(git diff --stat --summary -C -M $LAST_HEAD..$branch)
> +EOF
> + fi
> +}
> +
> while [ $# -gt 0 ]; do
> case "$1" in
> --kup|-k)
> @@ -54,6 +98,9 @@ while [ $# -gt 0 ]; do
> LAST_HEAD=$2
> shift
> ;;
> + --for-next|-f)
> + FOR_NEXT=1
> + ;;
> --help|-h)
> help
> exit 0
> @@ -66,6 +113,17 @@ while [ $# -gt 0 ]; do
> shift
> done
>
> +if [ $FOR_NEXT -eq 1 ]; then
> + echo "Push your for-next branch:"
> + printf "\tgit push origin for-next:for-next\n"
> + prepare_mail "for-next"
> + if [ -n "$LAST_HEAD" ]; then
> + echo "Command to send ANNOUNCE email"
> + printf "\tneomutt -H $mail_file\n"
> + fi
> + exit 0
> +fi
> +
> if [ -z "$EDITOR" ]; then
> EDITOR=$(command -v vi)
> fi
> @@ -128,39 +186,7 @@ if [ $KUP -eq 1 ]; then
> pub/linux/utils/fs/xfs/xfsprogs/
> fi;
>
> -mail_file=$(mktemp)
> -if [ -n "$LAST_HEAD" ]; then
> - cat << EOF > $mail_file
> -To: linux-xfs@vger.kernel.org
> -Cc: $(./tools/git-contributors.py $LAST_HEAD.. --delimiter ' ')
> -Subject: [ANNOUNCE] xfsprogs $(git describe --abbrev=0) released
> -
> -Hi folks,
> -
> -The xfsprogs repository at:
> -
> - git://git.kernel.org/pub/scm/fs/xfs/xfsprogs-dev.git
> -
> -has just been updated.
> -
> -Patches often get missed, so if your outstanding patches are properly reviewed
> -on the list and not included in this update, please let me know.
> -
> -The for-next branch has also been updated to match the state of master.
> -
> -The new head of the master branch is commit:
> -
> -$(git log --oneline --format="%H" -1)
> -
> -New commits:
> -
> -$(git shortlog --format="[%h] %s" $LAST_HEAD..HEAD)
> -
> -Code Diffstat:
> -
> -$(git diff --stat --summary -C -M $LAST_HEAD..HEAD)
> -EOF
> -fi
> +prepare_mail "master"
>
> echo ""
> echo "Done. Please remember to push out tags and the branch."
>
> --
> 2.47.2
>
>
^ permalink raw reply [flat|nested] 17+ messages in thread
* Re: [PATCH v3 5/8] Add git-contributors script to notify about merges
2025-02-11 18:58 ` Darrick J. Wong
@ 2025-02-12 11:16 ` Andrey Albershteyn
2025-02-12 11:37 ` Andrey Albershteyn
2025-02-12 21:29 ` Darrick J. Wong
0 siblings, 2 replies; 17+ messages in thread
From: Andrey Albershteyn @ 2025-02-12 11:16 UTC (permalink / raw)
To: Darrick J. Wong; +Cc: linux-xfs, Andrey Albershteyn
On 2025-02-11 10:58:04, Darrick J. Wong wrote:
> On Tue, Feb 11, 2025 at 06:26:57PM +0100, Andrey Albershteyn wrote:
> > Add python script used to collect emails over all changes merged in
> > the next release.
> >
> > CC: Darrick J. Wong <djwong@kernel.org>
> > Signed-off-by: Darrick J. Wong <djwong@kernel.org>
> > Reviewed-by: Darrick J. Wong <djwong@kernel.org>
> > Signed-off-by: Andrey Albershteyn <aalbersh@kernel.org>
> > ---
> > tools/git-contributors.py | 94 +++++++++++++++++++++++++++++++++++++++++++++++
> > 1 file changed, 94 insertions(+)
> >
> > diff --git a/tools/git-contributors.py b/tools/git-contributors.py
> > new file mode 100755
> > index 0000000000000000000000000000000000000000..83bbe8ce0ee1dcbd591c6d3016d553fac2a7d286
> > --- /dev/null
> > +++ b/tools/git-contributors.py
> > @@ -0,0 +1,94 @@
> > +#!/usr/bin/python3
> > +
> > +# List all contributors to a series of git commits.
> > +# Copyright(C) 2025 Oracle, All Rights Reserved.
> > +# Licensed under GPL 2.0 or later
> > +
> > +import re
> > +import subprocess
> > +import io
> > +import sys
> > +import argparse
> > +import email.utils
> > +
> > +DEBUG = False
> > +
> > +def backtick(args):
> > + '''Generator function that yields lines of a program's stdout.'''
> > + if DEBUG:
> > + print(' '.join(args))
> > + p = subprocess.Popen(args, stdout = subprocess.PIPE)
> > + for line in io.TextIOWrapper(p.stdout, encoding="utf-8"):
> > + yield line
> > +
> > +class find_developers(object):
> > + def __init__(self):
> > + tags = '%s|%s|%s|%s|%s|%s|%s|%s' % (
> > + 'signed-off-by',
> > + 'acked-by',
> > + 'cc',
> > + 'reviewed-by',
> > + 'reported-by',
> > + 'tested-by',
> > + 'suggested-by',
> > + 'reported-and-tested-by')
> > + # some tag, a colon, a space, and everything after that
> > + regex1 = r'^(%s):\s+(.+)$' % tags
> > +
> > + self.r1 = re.compile(regex1, re.I)
> > +
> > + def run(self, lines):
> > + addr_list = []
> > +
> > + for line in lines:
> > + l = line.strip()
> > +
> > + # emailutils can handle abominations like:
> > + #
> > + # Reviewed-by: Bogus J. Simpson <bogus@simpson.com>
> > + # Reviewed-by: "Bogus J. Simpson" <bogus@simpson.com>
> > + # Reviewed-by: bogus@simpson.com
> > + # Cc: <stable@vger.kernel.org> # v6.9
> > + # Tested-by: Moo Cow <foo@bar.com> # powerpc
> > + m = self.r1.match(l)
> > + if not m:
> > + continue
> > + (name, addr) = email.utils.parseaddr(m.expand(r'\g<2>'))
> > +
> > + # This last split removes anything after a hash mark,
> > + # because someone could have provided an improperly
> > + # formatted email address:
> > + #
> > + # Cc: stable@vger.kernel.org # v6.19+
> > + #
> > + # emailutils doesn't seem to catch this, and I can't
> > + # fully tell from RFC2822 that this isn't allowed. I
> > + # think it is because dtext doesn't forbid spaces or
> > + # hash marks.
> > + addr_list.append(addr.split('#')[0])
>
> I think it's the case that the canonical stable cc tag format for kernel
> patches as provided by the stable kernel process rules document:
>
> Cc: <stable@vger.kernel.org> # vX.Y
>
> is not actually actually rfc5322 compliant, so strings like that break
> Python's emailutils parsers. parseaddr() completely chokes on this, and
> retuns name=='' and addr=='', because the only thing that can come after
> the address portion are whitespace, EOL, or a comma followed by more
> email addresses. There's definitely not supposed to be an octothorpe
> followed by even more text.
>
> In the end I let myself be nerdsniped with even more string parsing bs,
> and this loop body is the result:
>
> l = line.strip()
>
> # First, does this line match any of the headers we
> # know about?
> m = self.r1.match(l)
> if not m:
> continue
>
> # The split removes everything after an octothorpe
> # (hash mark), because someone could have provided an
> # improperly formatted email address:
> #
> # Cc: stable@vger.kernel.org # v6.19+
> #
> # This, according to my reading of RFC5322, is allowed
> # because octothorpes can be part of atom text.
> # However, it is interepreted as if there weren't any
> # whitespace ("stable@vger.kernel.org#v6.19+"). The
> # grammar allows for this form, even though this is not
> # a correct Internet domain name.
> #
> # Worse, if you follow the format specified in the
> # kernel's SubmittingPatches file:
> #
> # Cc: <stable@vger.kernel.org> # v6.9
> #
> # emailutils will not know how to parse this, and
> # returns empty strings. I think this is because the
> # angle-addr specification allows only whitespace
> # between the closing angle bracket and the CRLF.
> #
> # Hack around both problems by ignoring everything
> # after an octothorpe, no matter where it occurs in the
> # string. If someone has one in their name or the
> # email address, too bad.
> a = m.expand(r'\g<2>').split('#')[0]
>
> # emailutils can extract email addresses from headers
> # that roughly follow the destination address field
> # format:
> #
> # Reviewed-by: Bogus J. Simpson <bogus@simpson.com>
> # Reviewed-by: "Bogus J. Simpson" <bogus@simpson.com>
> # Reviewed-by: bogus@simpson.com
> # Tested-by: Moo Cow <foo@bar.com>
> #
> # Use it to extract the email address, because we don't
> # care about the display name.
> (name, addr) = email.utils.parseaddr(a)
> addr_list.append(addr)
>
> <shrug> but maybe we should try that on a few branches first before
> committing to this string parsing mess ... ? Not that this is any less
> stupid than the previous version I shared out. :(
Can we just drop anything with 'stable@'? These are patches from
libxfs syncs, do they have any value for stable@ list?
But the change is still make sense if anyone uses hash mark for
something else, I will apply your change.
--
- Andrey
^ permalink raw reply [flat|nested] 17+ messages in thread
* Re: [PATCH v3 5/8] Add git-contributors script to notify about merges
2025-02-12 11:16 ` Andrey Albershteyn
@ 2025-02-12 11:37 ` Andrey Albershteyn
2025-02-12 22:24 ` Darrick J. Wong
2025-02-12 21:29 ` Darrick J. Wong
1 sibling, 1 reply; 17+ messages in thread
From: Andrey Albershteyn @ 2025-02-12 11:37 UTC (permalink / raw)
To: Darrick J. Wong; +Cc: linux-xfs, Andrey Albershteyn
On 2025-02-12 12:16:46, Andrey Albershteyn wrote:
> On 2025-02-11 10:58:04, Darrick J. Wong wrote:
> > On Tue, Feb 11, 2025 at 06:26:57PM +0100, Andrey Albershteyn wrote:
> > > Add python script used to collect emails over all changes merged in
> > > the next release.
> > >
> > > CC: Darrick J. Wong <djwong@kernel.org>
> > > Signed-off-by: Darrick J. Wong <djwong@kernel.org>
> > > Reviewed-by: Darrick J. Wong <djwong@kernel.org>
> > > Signed-off-by: Andrey Albershteyn <aalbersh@kernel.org>
> > > ---
> > > tools/git-contributors.py | 94 +++++++++++++++++++++++++++++++++++++++++++++++
> > > 1 file changed, 94 insertions(+)
> > >
> > > diff --git a/tools/git-contributors.py b/tools/git-contributors.py
> > > new file mode 100755
> > > index 0000000000000000000000000000000000000000..83bbe8ce0ee1dcbd591c6d3016d553fac2a7d286
> > > --- /dev/null
> > > +++ b/tools/git-contributors.py
> > > @@ -0,0 +1,94 @@
> > > +#!/usr/bin/python3
> > > +
> > > +# List all contributors to a series of git commits.
> > > +# Copyright(C) 2025 Oracle, All Rights Reserved.
> > > +# Licensed under GPL 2.0 or later
> > > +
> > > +import re
> > > +import subprocess
> > > +import io
> > > +import sys
> > > +import argparse
> > > +import email.utils
> > > +
> > > +DEBUG = False
> > > +
> > > +def backtick(args):
> > > + '''Generator function that yields lines of a program's stdout.'''
> > > + if DEBUG:
> > > + print(' '.join(args))
> > > + p = subprocess.Popen(args, stdout = subprocess.PIPE)
> > > + for line in io.TextIOWrapper(p.stdout, encoding="utf-8"):
> > > + yield line
> > > +
> > > +class find_developers(object):
> > > + def __init__(self):
> > > + tags = '%s|%s|%s|%s|%s|%s|%s|%s' % (
> > > + 'signed-off-by',
> > > + 'acked-by',
> > > + 'cc',
> > > + 'reviewed-by',
> > > + 'reported-by',
> > > + 'tested-by',
> > > + 'suggested-by',
> > > + 'reported-and-tested-by')
> > > + # some tag, a colon, a space, and everything after that
> > > + regex1 = r'^(%s):\s+(.+)$' % tags
> > > +
> > > + self.r1 = re.compile(regex1, re.I)
> > > +
> > > + def run(self, lines):
> > > + addr_list = []
> > > +
> > > + for line in lines:
> > > + l = line.strip()
> > > +
> > > + # emailutils can handle abominations like:
> > > + #
> > > + # Reviewed-by: Bogus J. Simpson <bogus@simpson.com>
> > > + # Reviewed-by: "Bogus J. Simpson" <bogus@simpson.com>
> > > + # Reviewed-by: bogus@simpson.com
> > > + # Cc: <stable@vger.kernel.org> # v6.9
> > > + # Tested-by: Moo Cow <foo@bar.com> # powerpc
> > > + m = self.r1.match(l)
> > > + if not m:
> > > + continue
> > > + (name, addr) = email.utils.parseaddr(m.expand(r'\g<2>'))
> > > +
> > > + # This last split removes anything after a hash mark,
> > > + # because someone could have provided an improperly
> > > + # formatted email address:
> > > + #
> > > + # Cc: stable@vger.kernel.org # v6.19+
> > > + #
> > > + # emailutils doesn't seem to catch this, and I can't
> > > + # fully tell from RFC2822 that this isn't allowed. I
> > > + # think it is because dtext doesn't forbid spaces or
> > > + # hash marks.
> > > + addr_list.append(addr.split('#')[0])
> >
> > I think it's the case that the canonical stable cc tag format for kernel
> > patches as provided by the stable kernel process rules document:
> >
> > Cc: <stable@vger.kernel.org> # vX.Y
> >
> > is not actually actually rfc5322 compliant, so strings like that break
> > Python's emailutils parsers. parseaddr() completely chokes on this, and
> > retuns name=='' and addr=='', because the only thing that can come after
> > the address portion are whitespace, EOL, or a comma followed by more
> > email addresses. There's definitely not supposed to be an octothorpe
> > followed by even more text.
> >
> > In the end I let myself be nerdsniped with even more string parsing bs,
> > and this loop body is the result:
> >
> > l = line.strip()
> >
> > # First, does this line match any of the headers we
> > # know about?
> > m = self.r1.match(l)
> > if not m:
> > continue
> >
> > # The split removes everything after an octothorpe
> > # (hash mark), because someone could have provided an
> > # improperly formatted email address:
> > #
> > # Cc: stable@vger.kernel.org # v6.19+
> > #
> > # This, according to my reading of RFC5322, is allowed
> > # because octothorpes can be part of atom text.
> > # However, it is interepreted as if there weren't any
> > # whitespace ("stable@vger.kernel.org#v6.19+"). The
> > # grammar allows for this form, even though this is not
> > # a correct Internet domain name.
> > #
> > # Worse, if you follow the format specified in the
> > # kernel's SubmittingPatches file:
> > #
> > # Cc: <stable@vger.kernel.org> # v6.9
> > #
> > # emailutils will not know how to parse this, and
> > # returns empty strings. I think this is because the
> > # angle-addr specification allows only whitespace
> > # between the closing angle bracket and the CRLF.
> > #
> > # Hack around both problems by ignoring everything
> > # after an octothorpe, no matter where it occurs in the
> > # string. If someone has one in their name or the
> > # email address, too bad.
> > a = m.expand(r'\g<2>').split('#')[0]
> >
> > # emailutils can extract email addresses from headers
> > # that roughly follow the destination address field
> > # format:
> > #
> > # Reviewed-by: Bogus J. Simpson <bogus@simpson.com>
> > # Reviewed-by: "Bogus J. Simpson" <bogus@simpson.com>
> > # Reviewed-by: bogus@simpson.com
> > # Tested-by: Moo Cow <foo@bar.com>
> > #
> > # Use it to extract the email address, because we don't
> > # care about the display name.
> > (name, addr) = email.utils.parseaddr(a)
> > addr_list.append(addr)
> >
> > <shrug> but maybe we should try that on a few branches first before
> > committing to this string parsing mess ... ? Not that this is any less
> > stupid than the previous version I shared out. :(
>
> Can we just drop anything with 'stable@'? These are patches from
> libxfs syncs, do they have any value for stable@ list?
>
> But the change is still make sense if anyone uses hash mark for
> something else, I will apply your change.
>
Hmm, there's seems to be more cases to handle:
Cc: 1000974@bugs.debian.org, gustavoars@kernel.org, keescook@chromium.org
Reported-by: Xu, Wen <wen.xu@gatech.edu>
Both fail to parse, the first one as it need to be split and second
one due to comma
--
- Andrey
^ permalink raw reply [flat|nested] 17+ messages in thread
* Re: [PATCH v3 5/8] Add git-contributors script to notify about merges
2025-02-12 11:16 ` Andrey Albershteyn
2025-02-12 11:37 ` Andrey Albershteyn
@ 2025-02-12 21:29 ` Darrick J. Wong
1 sibling, 0 replies; 17+ messages in thread
From: Darrick J. Wong @ 2025-02-12 21:29 UTC (permalink / raw)
To: Andrey Albershteyn; +Cc: linux-xfs, Andrey Albershteyn
On Wed, Feb 12, 2025 at 12:16:46PM +0100, Andrey Albershteyn wrote:
> On 2025-02-11 10:58:04, Darrick J. Wong wrote:
> > On Tue, Feb 11, 2025 at 06:26:57PM +0100, Andrey Albershteyn wrote:
> > > Add python script used to collect emails over all changes merged in
> > > the next release.
> > >
> > > CC: Darrick J. Wong <djwong@kernel.org>
> > > Signed-off-by: Darrick J. Wong <djwong@kernel.org>
> > > Reviewed-by: Darrick J. Wong <djwong@kernel.org>
> > > Signed-off-by: Andrey Albershteyn <aalbersh@kernel.org>
> > > ---
> > > tools/git-contributors.py | 94 +++++++++++++++++++++++++++++++++++++++++++++++
> > > 1 file changed, 94 insertions(+)
> > >
> > > diff --git a/tools/git-contributors.py b/tools/git-contributors.py
> > > new file mode 100755
> > > index 0000000000000000000000000000000000000000..83bbe8ce0ee1dcbd591c6d3016d553fac2a7d286
> > > --- /dev/null
> > > +++ b/tools/git-contributors.py
> > > @@ -0,0 +1,94 @@
> > > +#!/usr/bin/python3
> > > +
> > > +# List all contributors to a series of git commits.
> > > +# Copyright(C) 2025 Oracle, All Rights Reserved.
> > > +# Licensed under GPL 2.0 or later
> > > +
> > > +import re
> > > +import subprocess
> > > +import io
> > > +import sys
> > > +import argparse
> > > +import email.utils
> > > +
> > > +DEBUG = False
> > > +
> > > +def backtick(args):
> > > + '''Generator function that yields lines of a program's stdout.'''
> > > + if DEBUG:
> > > + print(' '.join(args))
> > > + p = subprocess.Popen(args, stdout = subprocess.PIPE)
> > > + for line in io.TextIOWrapper(p.stdout, encoding="utf-8"):
> > > + yield line
> > > +
> > > +class find_developers(object):
> > > + def __init__(self):
> > > + tags = '%s|%s|%s|%s|%s|%s|%s|%s' % (
> > > + 'signed-off-by',
> > > + 'acked-by',
> > > + 'cc',
> > > + 'reviewed-by',
> > > + 'reported-by',
> > > + 'tested-by',
> > > + 'suggested-by',
> > > + 'reported-and-tested-by')
> > > + # some tag, a colon, a space, and everything after that
> > > + regex1 = r'^(%s):\s+(.+)$' % tags
> > > +
> > > + self.r1 = re.compile(regex1, re.I)
> > > +
> > > + def run(self, lines):
> > > + addr_list = []
> > > +
> > > + for line in lines:
> > > + l = line.strip()
> > > +
> > > + # emailutils can handle abominations like:
> > > + #
> > > + # Reviewed-by: Bogus J. Simpson <bogus@simpson.com>
> > > + # Reviewed-by: "Bogus J. Simpson" <bogus@simpson.com>
> > > + # Reviewed-by: bogus@simpson.com
> > > + # Cc: <stable@vger.kernel.org> # v6.9
> > > + # Tested-by: Moo Cow <foo@bar.com> # powerpc
> > > + m = self.r1.match(l)
> > > + if not m:
> > > + continue
> > > + (name, addr) = email.utils.parseaddr(m.expand(r'\g<2>'))
> > > +
> > > + # This last split removes anything after a hash mark,
> > > + # because someone could have provided an improperly
> > > + # formatted email address:
> > > + #
> > > + # Cc: stable@vger.kernel.org # v6.19+
> > > + #
> > > + # emailutils doesn't seem to catch this, and I can't
> > > + # fully tell from RFC2822 that this isn't allowed. I
> > > + # think it is because dtext doesn't forbid spaces or
> > > + # hash marks.
> > > + addr_list.append(addr.split('#')[0])
> >
> > I think it's the case that the canonical stable cc tag format for kernel
> > patches as provided by the stable kernel process rules document:
> >
> > Cc: <stable@vger.kernel.org> # vX.Y
> >
> > is not actually actually rfc5322 compliant, so strings like that break
> > Python's emailutils parsers. parseaddr() completely chokes on this, and
> > retuns name=='' and addr=='', because the only thing that can come after
> > the address portion are whitespace, EOL, or a comma followed by more
> > email addresses. There's definitely not supposed to be an octothorpe
> > followed by even more text.
> >
> > In the end I let myself be nerdsniped with even more string parsing bs,
> > and this loop body is the result:
> >
> > l = line.strip()
> >
> > # First, does this line match any of the headers we
> > # know about?
> > m = self.r1.match(l)
> > if not m:
> > continue
> >
> > # The split removes everything after an octothorpe
> > # (hash mark), because someone could have provided an
> > # improperly formatted email address:
> > #
> > # Cc: stable@vger.kernel.org # v6.19+
> > #
> > # This, according to my reading of RFC5322, is allowed
> > # because octothorpes can be part of atom text.
> > # However, it is interepreted as if there weren't any
> > # whitespace ("stable@vger.kernel.org#v6.19+"). The
> > # grammar allows for this form, even though this is not
> > # a correct Internet domain name.
> > #
> > # Worse, if you follow the format specified in the
> > # kernel's SubmittingPatches file:
> > #
> > # Cc: <stable@vger.kernel.org> # v6.9
> > #
> > # emailutils will not know how to parse this, and
> > # returns empty strings. I think this is because the
> > # angle-addr specification allows only whitespace
> > # between the closing angle bracket and the CRLF.
> > #
> > # Hack around both problems by ignoring everything
> > # after an octothorpe, no matter where it occurs in the
> > # string. If someone has one in their name or the
> > # email address, too bad.
> > a = m.expand(r'\g<2>').split('#')[0]
> >
> > # emailutils can extract email addresses from headers
> > # that roughly follow the destination address field
> > # format:
> > #
> > # Reviewed-by: Bogus J. Simpson <bogus@simpson.com>
> > # Reviewed-by: "Bogus J. Simpson" <bogus@simpson.com>
> > # Reviewed-by: bogus@simpson.com
> > # Tested-by: Moo Cow <foo@bar.com>
> > #
> > # Use it to extract the email address, because we don't
> > # care about the display name.
> > (name, addr) = email.utils.parseaddr(a)
> > addr_list.append(addr)
> >
> > <shrug> but maybe we should try that on a few branches first before
> > committing to this string parsing mess ... ? Not that this is any less
> > stupid than the previous version I shared out. :(
>
> Can we just drop anything with 'stable@'? These are patches from
> libxfs syncs, do they have any value for stable@ list?
None at all; we should probably make libxfs-apply filter those out.
> But the change is still make sense if anyone uses hash mark for
> something else, I will apply your change.
<shrug> I've occasionally seen people leave trailers such as:
Acked-by: "Cowmoo Userguy" <cow@user.com> # xfs
On treewide changes, so I think we should handle hashmarks correctly
even if we rip out the stable@vger cc's.
--D
>
> --
> - Andrey
>
>
^ permalink raw reply [flat|nested] 17+ messages in thread
* Re: [PATCH v3 5/8] Add git-contributors script to notify about merges
2025-02-12 11:37 ` Andrey Albershteyn
@ 2025-02-12 22:24 ` Darrick J. Wong
0 siblings, 0 replies; 17+ messages in thread
From: Darrick J. Wong @ 2025-02-12 22:24 UTC (permalink / raw)
To: Andrey Albershteyn; +Cc: linux-xfs, Andrey Albershteyn
On Wed, Feb 12, 2025 at 12:37:45PM +0100, Andrey Albershteyn wrote:
> On 2025-02-12 12:16:46, Andrey Albershteyn wrote:
> > On 2025-02-11 10:58:04, Darrick J. Wong wrote:
> > > On Tue, Feb 11, 2025 at 06:26:57PM +0100, Andrey Albershteyn wrote:
> > > > Add python script used to collect emails over all changes merged in
> > > > the next release.
> > > >
> > > > CC: Darrick J. Wong <djwong@kernel.org>
> > > > Signed-off-by: Darrick J. Wong <djwong@kernel.org>
> > > > Reviewed-by: Darrick J. Wong <djwong@kernel.org>
> > > > Signed-off-by: Andrey Albershteyn <aalbersh@kernel.org>
> > > > ---
> > > > tools/git-contributors.py | 94 +++++++++++++++++++++++++++++++++++++++++++++++
> > > > 1 file changed, 94 insertions(+)
> > > >
> > > > diff --git a/tools/git-contributors.py b/tools/git-contributors.py
> > > > new file mode 100755
> > > > index 0000000000000000000000000000000000000000..83bbe8ce0ee1dcbd591c6d3016d553fac2a7d286
> > > > --- /dev/null
> > > > +++ b/tools/git-contributors.py
> > > > @@ -0,0 +1,94 @@
> > > > +#!/usr/bin/python3
> > > > +
> > > > +# List all contributors to a series of git commits.
> > > > +# Copyright(C) 2025 Oracle, All Rights Reserved.
> > > > +# Licensed under GPL 2.0 or later
> > > > +
> > > > +import re
> > > > +import subprocess
> > > > +import io
> > > > +import sys
> > > > +import argparse
> > > > +import email.utils
> > > > +
> > > > +DEBUG = False
> > > > +
> > > > +def backtick(args):
> > > > + '''Generator function that yields lines of a program's stdout.'''
> > > > + if DEBUG:
> > > > + print(' '.join(args))
> > > > + p = subprocess.Popen(args, stdout = subprocess.PIPE)
> > > > + for line in io.TextIOWrapper(p.stdout, encoding="utf-8"):
> > > > + yield line
> > > > +
> > > > +class find_developers(object):
> > > > + def __init__(self):
> > > > + tags = '%s|%s|%s|%s|%s|%s|%s|%s' % (
> > > > + 'signed-off-by',
> > > > + 'acked-by',
> > > > + 'cc',
> > > > + 'reviewed-by',
> > > > + 'reported-by',
> > > > + 'tested-by',
> > > > + 'suggested-by',
> > > > + 'reported-and-tested-by')
> > > > + # some tag, a colon, a space, and everything after that
> > > > + regex1 = r'^(%s):\s+(.+)$' % tags
> > > > +
> > > > + self.r1 = re.compile(regex1, re.I)
> > > > +
> > > > + def run(self, lines):
> > > > + addr_list = []
> > > > +
> > > > + for line in lines:
> > > > + l = line.strip()
> > > > +
> > > > + # emailutils can handle abominations like:
> > > > + #
> > > > + # Reviewed-by: Bogus J. Simpson <bogus@simpson.com>
> > > > + # Reviewed-by: "Bogus J. Simpson" <bogus@simpson.com>
> > > > + # Reviewed-by: bogus@simpson.com
> > > > + # Cc: <stable@vger.kernel.org> # v6.9
> > > > + # Tested-by: Moo Cow <foo@bar.com> # powerpc
> > > > + m = self.r1.match(l)
> > > > + if not m:
> > > > + continue
> > > > + (name, addr) = email.utils.parseaddr(m.expand(r'\g<2>'))
> > > > +
> > > > + # This last split removes anything after a hash mark,
> > > > + # because someone could have provided an improperly
> > > > + # formatted email address:
> > > > + #
> > > > + # Cc: stable@vger.kernel.org # v6.19+
> > > > + #
> > > > + # emailutils doesn't seem to catch this, and I can't
> > > > + # fully tell from RFC2822 that this isn't allowed. I
> > > > + # think it is because dtext doesn't forbid spaces or
> > > > + # hash marks.
> > > > + addr_list.append(addr.split('#')[0])
> > >
> > > I think it's the case that the canonical stable cc tag format for kernel
> > > patches as provided by the stable kernel process rules document:
> > >
> > > Cc: <stable@vger.kernel.org> # vX.Y
> > >
> > > is not actually actually rfc5322 compliant, so strings like that break
> > > Python's emailutils parsers. parseaddr() completely chokes on this, and
> > > retuns name=='' and addr=='', because the only thing that can come after
> > > the address portion are whitespace, EOL, or a comma followed by more
> > > email addresses. There's definitely not supposed to be an octothorpe
> > > followed by even more text.
> > >
> > > In the end I let myself be nerdsniped with even more string parsing bs,
> > > and this loop body is the result:
> > >
> > > l = line.strip()
> > >
> > > # First, does this line match any of the headers we
> > > # know about?
> > > m = self.r1.match(l)
> > > if not m:
> > > continue
> > >
> > > # The split removes everything after an octothorpe
> > > # (hash mark), because someone could have provided an
> > > # improperly formatted email address:
> > > #
> > > # Cc: stable@vger.kernel.org # v6.19+
> > > #
> > > # This, according to my reading of RFC5322, is allowed
> > > # because octothorpes can be part of atom text.
> > > # However, it is interepreted as if there weren't any
> > > # whitespace ("stable@vger.kernel.org#v6.19+"). The
> > > # grammar allows for this form, even though this is not
> > > # a correct Internet domain name.
> > > #
> > > # Worse, if you follow the format specified in the
> > > # kernel's SubmittingPatches file:
> > > #
> > > # Cc: <stable@vger.kernel.org> # v6.9
> > > #
> > > # emailutils will not know how to parse this, and
> > > # returns empty strings. I think this is because the
> > > # angle-addr specification allows only whitespace
> > > # between the closing angle bracket and the CRLF.
> > > #
> > > # Hack around both problems by ignoring everything
> > > # after an octothorpe, no matter where it occurs in the
> > > # string. If someone has one in their name or the
> > > # email address, too bad.
> > > a = m.expand(r'\g<2>').split('#')[0]
> > >
> > > # emailutils can extract email addresses from headers
> > > # that roughly follow the destination address field
> > > # format:
> > > #
> > > # Reviewed-by: Bogus J. Simpson <bogus@simpson.com>
> > > # Reviewed-by: "Bogus J. Simpson" <bogus@simpson.com>
> > > # Reviewed-by: bogus@simpson.com
> > > # Tested-by: Moo Cow <foo@bar.com>
> > > #
> > > # Use it to extract the email address, because we don't
> > > # care about the display name.
> > > (name, addr) = email.utils.parseaddr(a)
> > > addr_list.append(addr)
> > >
> > > <shrug> but maybe we should try that on a few branches first before
> > > committing to this string parsing mess ... ? Not that this is any less
> > > stupid than the previous version I shared out. :(
> >
> > Can we just drop anything with 'stable@'? These are patches from
> > libxfs syncs, do they have any value for stable@ list?
> >
> > But the change is still make sense if anyone uses hash mark for
> > something else, I will apply your change.
> >
>
> Hmm, there's seems to be more cases to handle:
>
> Cc: 1000974@bugs.debian.org, gustavoars@kernel.org, keescook@chromium.org
Ugh, ok, will go handle that one.
> Reported-by: Xu, Wen <wen.xu@gatech.edu>
>
> Both fail to parse, the first one as it need to be split and second
> one due to comma
Technically speaking people are supposed to be quoting name punctuation
in the manner specified by the RFC ("Xu, Wen" <wen.xu@gatech.edu>) but
there's basically zero validation of any freeform git commit trailers
so everyone is stuck with inconsistent piles of regular expression
hacks.
(No, I'm not a fan of "be liberal in what you accept"; one ought to have
a strong motivation for taking on extra work)
Does this work? Note the change from --delimiter to --separator.
--D
#!/usr/bin/env python3
# List all contributors to a series of git commits.
# Copyright(C) 2025 Oracle, All Rights Reserved.
# Licensed under GPL 2.0 or later
import re
import subprocess
import io
import sys
import argparse
import email.utils
DEBUG = False
def backtick(args):
'''Generator function that yields lines of a program's stdout.'''
if DEBUG:
print(' '.join(args))
p = subprocess.Popen(args, stdout = subprocess.PIPE)
for line in io.TextIOWrapper(p.stdout, encoding="utf-8"):
yield line
class find_developers(object):
def __init__(self):
tags = '%s|%s|%s|%s|%s|%s|%s|%s' % (
'signed-off-by',
'acked-by',
'cc',
'reviewed-by',
'reported-by',
'tested-by',
'suggested-by',
'reported-and-tested-by')
# some tag, a colon, a space, and everything after that
regex1 = r'^(%s):\s+(.+)$' % tags
self.r1 = re.compile(regex1, re.I)
# regex to guess if this is a list of multiple addresses.
# Not sure why the initial "^.*" is needed here.
self.r2 = re.compile(r'^.*,[^,]*@[^@]*,[^,]*@', re.I)
# regex to match on anything inside a pair of angle brackets
self.r3 = re.compile(r'^.*<(.+)>', re.I)
def _handle_addr(self, addr):
# The next split removes everything after an octothorpe (hash
# mark), because someone could have provided an improperly
# formatted email address:
#
# Cc: stable@vger.kernel.org # v6.19+
#
# This, according to my reading of RFC5322, is allowed because
# octothorpes can be part of atom text. However, it is
# interepreted as if there weren't any whitespace
# ("stable@vger.kernel.org#v6.19+"). The grammar allows for
# this form, even though this is not a correct Internet domain
# name.
#
# Worse, if you follow the format specified in the kernel's
# SubmittingPatches file:
#
# Cc: <stable@vger.kernel.org> # v6.9
#
# emailutils will not know how to parse this, and returns empty
# strings. I think this is because the angle-addr
# specification allows only whitespace between the closing
# angle bracket and the CRLF.
#
# Hack around both problems by ignoring everything after an
# octothorpe, no matter where it occurs in the string. If
# someone has one in their name or the email address, too bad.
a = addr.split('#')[0]
# emailutils can extract email addresses from headers that
# roughly follow the destination address field format:
#
# Reviewed-by: Bogus J. Simpson <bogus@simpson.com>
# Reviewed-by: "Bogus J. Simpson" <bogus@simpson.com>
# Reviewed-by: bogus@simpson.com
#
# Use it to extract the email address, because we don't care
# about the display name.
(name, addr) = email.utils.parseaddr(a)
if DEBUG:
print(f'A:{a}:NAME:{name}:ADDR:{addr}:')
if len(addr) > 0:
return addr
# If emailutils fails to find anything, let's see if there's
# a sequence of characters within angle brackets and hope that
# is an email address. This works around things like:
#
# Reported-by: Xu, Wen <wen.xu@gatech.edu>
#
# Which should have had the name in quotations because there's
# a comma.
m = self.r3.match(a)
if m:
addr = m.expand(r'\g<1>')
if DEBUG:
print(f"M3:{addr}:M:{m}:")
return addr
# No idea, just spit the whole thing out and hope for the best.
return a
def run(self, lines):
addr_list = []
for line in lines:
l = line.strip()
# First, does this line match any of the headers we
# know about?
m = self.r1.match(l)
if not m:
continue
rightside = m.expand(r'\g<2>')
n = self.r2.match(rightside)
if n:
# Break the line into an array of addresses,
# delimited by commas, then handle each
# address.
addrs = rightside.split(',')
if DEBUG:
print(f"0LINE:{rightside}:ADDRS:{addrs}:M:{n}")
for addr in addrs:
a = self._handle_addr(addr)
addr_list.append(a)
else:
# Otherwise treat the line as a single email
# address.
if DEBUG:
print(f"1LINE:{rightside}:M:{n}")
a = self._handle_addr(rightside)
addr_list.append(a)
return sorted(set(addr_list))
def main():
global DEBUG
parser = argparse.ArgumentParser(description = "List email addresses of contributors to a series of git commits.")
parser.add_argument("revspec", nargs = '?', default = None, \
help = "git revisions to process.")
parser.add_argument("--separator", type = str, default = '\n', \
help = "Separate each email address with this string.")
parser.add_argument('--debug', action = 'store_true', default = False, \
help = argparse.SUPPRESS)
args = parser.parse_args()
if args.debug:
DEBUG = True
fd = find_developers()
if args.revspec:
# read git commits from repo
contributors = fd.run(backtick(['git', 'log', '--pretty=medium',
args.revspec]))
else:
# read patch from stdin
contributors = fd.run(sys.stdin.readlines())
print(args.delimiter.join(sorted(contributors)))
return 0
if __name__ == '__main__':
sys.exit(main())
^ permalink raw reply [flat|nested] 17+ messages in thread
end of thread, other threads:[~2025-02-12 22:24 UTC | newest]
Thread overview: 17+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2025-02-11 17:26 [PATCH v3 0/8] Update release.sh Andrey Albershteyn
2025-02-11 17:26 ` [PATCH v3 1/8] release.sh: add signing and fix outdated commands Andrey Albershteyn
2025-02-11 17:26 ` [PATCH v3 2/8] release.sh: add --kup to upload release tarball to kernel.org Andrey Albershteyn
2025-02-11 17:26 ` [PATCH v3 3/8] release.sh: update version files make commit optional Andrey Albershteyn
2025-02-11 17:26 ` [PATCH v3 4/8] release.sh: generate ANNOUNCE email Andrey Albershteyn
2025-02-11 17:26 ` [PATCH v3 5/8] Add git-contributors script to notify about merges Andrey Albershteyn
2025-02-11 18:58 ` Darrick J. Wong
2025-02-12 11:16 ` Andrey Albershteyn
2025-02-12 11:37 ` Andrey Albershteyn
2025-02-12 22:24 ` Darrick J. Wong
2025-02-12 21:29 ` Darrick J. Wong
2025-02-11 17:26 ` [PATCH v3 6/8] git-contributors: make revspec required and shebang fix Andrey Albershteyn
2025-02-11 18:59 ` Darrick J. Wong
2025-02-11 17:26 ` [PATCH v3 7/8] release.sh: use git-contributors to --cc contributors Andrey Albershteyn
2025-02-11 17:58 ` Darrick J. Wong
2025-02-11 17:27 ` [PATCH v3 8/8] release.sh: add -f to generate for-next update email Andrey Albershteyn
2025-02-11 19:09 ` Darrick J. Wong
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox