[PATCH v3 0/2] Record maintainer/reviewer Git Lab handles

[PATCH v3 0/2] Record maintainer/reviewer Git Lab handles

[Daniel P. Berrangé]

When triaging GitLab issues, whether regular bugs or security
disclosures, we need to be able to assign suitable accounts.

We can only assign Git Lab handles for accounts that are members
of the QEMU group, or QEMU repository with the "Reporter" role
or higher.

Currently one needs to run 'get_maintainer.pl -f <file>' to get
the maintainer / reviewer names & emails, and then look in the
GitLab members page to see if there is a match.

This simplifies that by including the GitLab handle alongside
the email addresses:

The GitLab account handle -> real name mapping is stored in
separate files .gitlab-map-auto & .gitlab-map-manual to avoid
redundancy in the MAINTAINERS file. The former file is
auto-generated and catches 99% of the mappings, the latter is
for manual fixups.

With this series applied as is, we get the following stats

$ ./scripts/gitlab-map-check
$ ./scripts/gitlab-map-check
Missing GitLab handle for maintainer 'Akihiko Odaki'
Missing GitLab handle for maintainer 'Alano Song'
Missing GitLab handle for maintainer 'Albert Esteve'
Missing GitLab handle for maintainer 'Alberto Garcia'
Missing GitLab handle for maintainer 'Alejandro Jimenez'
Missing GitLab handle for maintainer 'Aleksandar Rikalo'
Missing GitLab handle for maintainer 'Alessandro Di Federico'
Missing GitLab handle for maintainer 'Alexandre Iooss'
Missing GitLab handle for maintainer 'Alexandre Ratchov'
Missing GitLab handle for maintainer 'Alexey Kardashevskiy'
Missing GitLab handle for maintainer 'Amit Shah'
Missing GitLab handle for maintainer 'Andrew Jeffery'
Missing GitLab handle for maintainer 'Anirudh Srinivasan'
Missing GitLab handle for maintainer 'Anton Johansson'
Missing GitLab handle for maintainer 'Antony Pavlov'
Missing GitLab handle for maintainer 'Ari Sundholm'
Missing GitLab handle for maintainer 'Artyom Tarasenko'
Missing GitLab handle for maintainer 'Aurelien Jarno'
Missing GitLab handle for maintainer 'BALATON Zoltan'
Missing GitLab handle for maintainer 'Ben Chaney'
Missing GitLab handle for maintainer 'Bernhard Beschow'
Missing GitLab handle for maintainer 'Bibo Mao'
Missing GitLab handle for maintainer 'Brad Smith'
Missing GitLab handle for maintainer 'Canokeys.org'
Missing GitLab handle for maintainer 'Chris Rauer'
Missing GitLab handle for maintainer 'Christoph Muellner'
Missing GitLab handle for maintainer 'Clément Chigot'
Missing GitLab handle for maintainer 'Clément Mathieu--Drif'
Missing GitLab handle for maintainer 'Collin Walling'
Missing GitLab handle for maintainer 'Corey Minyard'
Missing GitLab handle for maintainer 'Darren Kenny'
Missing GitLab handle for maintainer 'Denis V. Lunev'
Missing GitLab handle for maintainer 'Dmitry Fleytman'
Missing GitLab handle for maintainer 'Dmitry Osipenko'
Missing GitLab handle for maintainer 'Dorjoy Chowdhury'
Missing GitLab handle for maintainer 'Doru Blânzeanu'
Missing GitLab handle for maintainer 'Elena Ufimtseva'
Missing GitLab handle for maintainer 'Eric Farman'
Missing GitLab handle for maintainer 'Eugenio Pérez'
Missing GitLab handle for maintainer 'FangSheng Huang'
Missing GitLab handle for maintainer 'Farhan Ali'
Missing GitLab handle for maintainer 'Felipe Balbi'
Missing GitLab handle for maintainer 'Francisco Iglesias'
Missing GitLab handle for maintainer 'Frederic Konrad'
Missing GitLab handle for maintainer 'Gaurav Sharma'
Missing GitLab handle for maintainer 'Gautam Gala'
Missing GitLab handle for maintainer 'Giuseppe Lettieri'
Missing GitLab handle for maintainer 'Glenn Miles'
Missing GitLab handle for maintainer 'Gonglei'
Missing GitLab handle for maintainer 'Haixu Cui'
Missing GitLab handle for maintainer 'Halil Pasic'
Missing GitLab handle for maintainer 'Hao Wu'
Missing GitLab handle for maintainer 'Hari Bathini'
Missing GitLab handle for maintainer 'Hendrik Brueckner'
Missing GitLab handle for maintainer 'Hervé Poussineau'
Missing GitLab handle for maintainer 'Hongren (Zenithal) Zheng'
Missing GitLab handle for maintainer 'Huacai Chen'
Missing GitLab handle for maintainer 'Hyman Huang'
Missing GitLab handle for maintainer 'Ilya Dryomov'
Missing GitLab handle for maintainer 'Ilya Leoshkevich'
Missing GitLab handle for maintainer 'Ilya Maximets'
Missing GitLab handle for maintainer 'Jagannathan Raman'
Missing GitLab handle for maintainer 'Jamin Lin'
Missing GitLab handle for maintainer 'Jan Kiszka'
Missing GitLab handle for maintainer 'Jared Rossi'
Missing GitLab handle for maintainer 'Jason Herne'
Missing GitLab handle for maintainer 'Jean-Christophe Dubois'
Missing GitLab handle for maintainer 'Jesper Devantier'
Missing GitLab handle for maintainer 'Jeuk Kim'
Missing GitLab handle for maintainer 'Jiaxun Yang'
Missing GitLab handle for maintainer 'Jiri Pirko'
Missing GitLab handle for maintainer 'Jiri Slaby'
Missing GitLab handle for maintainer 'Joe Komlodi'
Missing GitLab handle for maintainer 'Joel Stanley'
Missing GitLab handle for maintainer 'John Levon'
Missing GitLab handle for maintainer 'Jonathan Cameron'
Missing GitLab handle for maintainer 'Kane Chen'
Missing GitLab handle for maintainer 'Kashyap Chamarthy'
Missing GitLab handle for maintainer 'Keith Busch'
Missing GitLab handle for maintainer 'Kohei Tokunaga'
Missing GitLab handle for maintainer 'Kuan-Wei Chiu'
Missing GitLab handle for maintainer 'Kyle Evans'
Missing GitLab handle for maintainer 'Leif Lindholm'
Missing GitLab handle for maintainer 'Liu Zhiwei'
Missing GitLab handle for maintainer 'LIU Zhiwei'
Missing GitLab handle for maintainer 'Li-Wen Hsu'
Missing GitLab handle for maintainer 'Li Zhijian'
Missing GitLab handle for maintainer 'Luc Michel'
Missing GitLab handle for maintainer 'Lukas Straub'
Missing GitLab handle for maintainer 'Mads Ynddal'
Missing GitLab handle for maintainer 'Magnus Kulke'
Missing GitLab handle for maintainer 'Maksim Davydov'
Missing GitLab handle for maintainer 'Marcelo Tosatti'
Missing GitLab handle for maintainer 'Mark Kanda'
Missing GitLab handle for maintainer 'Markus Armbruster'
Missing GitLab handle for maintainer 'Mathieu Poirier'
Missing GitLab handle for maintainer 'Matthew Rosato'
Missing GitLab handle for maintainer 'Matyas Bobek'
Missing GitLab handle for maintainer 'Mauro Carvalho Chehab'
Missing GitLab handle for maintainer 'Max Filippov'
Missing GitLab handle for maintainer 'Michael Ellerman'
Missing GitLab handle for maintainer 'Michael Rolnik'
Missing GitLab handle for maintainer 'Milan Zamazal'
Missing GitLab handle for maintainer 'Nabih Estefan'
Missing GitLab handle for maintainer 'Niek Linnenbank'
Missing GitLab handle for maintainer 'Ninad Palsule'
Missing GitLab handle for maintainer 'Palmer Dabbelt'
Missing GitLab handle for maintainer 'Patrick Leis'
Missing GitLab handle for maintainer 'Paul Durrant'
Missing GitLab handle for maintainer 'Pavel Pisa'
Missing GitLab handle for maintainer 'Pedro Barbuda'
Missing GitLab handle for maintainer 'Peter Lieven'
Missing GitLab handle for maintainer 'Phil Dennis-Jordan'
Missing GitLab handle for maintainer 'Philipp Tomsich'
Missing GitLab handle for maintainer 'Portia Stephens'
Missing GitLab handle for maintainer 'Ran Wang'
Missing GitLab handle for maintainer 'Raphael Norwitz'
Missing GitLab handle for maintainer 'Reinoud Zandijk'
Missing GitLab handle for maintainer 'Richard W.M. Jones'
Missing GitLab handle for maintainer 'Roman Bolshakov'
Missing GitLab handle for maintainer 'Sai Pavan Boddu'
Missing GitLab handle for maintainer 'Sairaj Kodilkar'
Missing GitLab handle for maintainer 'Samuel Tardieu'
Missing GitLab handle for maintainer 'Samuel Thibault'
Missing GitLab handle for maintainer 'Sergio Lopez'
Missing GitLab handle for maintainer 'Song Gao'
Missing GitLab handle for maintainer 'Sourabh'
Missing GitLab handle for maintainer 'Sourabh Jain'
Missing GitLab handle for maintainer 'Sriram Yagnaraman'
Missing GitLab handle for maintainer 'Stafford Horne'
Missing GitLab handle for maintainer 'Steven Lee'
Missing GitLab handle for maintainer 'Strahinja Jankovic'
Missing GitLab handle for maintainer 'Subbaraya Sundeep'
Missing GitLab handle for maintainer 'Sunil V L'
Missing GitLab handle for maintainer 'Sven Schnelle'
Missing GitLab handle for maintainer 'Thanos Makatos'
Missing GitLab handle for maintainer 'Titus Rwantare'
Missing GitLab handle for maintainer 'Tomita Moeko'
Missing GitLab handle for maintainer 'Tony Krowiak'
Missing GitLab handle for maintainer 'Troy Lee'
Missing GitLab handle for maintainer 'Tyrone Ting'
Missing GitLab handle for maintainer 'Vikram Garhwal'
Missing GitLab handle for maintainer 'Viktor Prutyanov'
Missing GitLab handle for maintainer 'Vincenzo Maffione'
Missing GitLab handle for maintainer 'Viresh Kumar'
Missing GitLab handle for maintainer 'WANG Xuerui'
Missing GitLab handle for maintainer 'Wei Liu'
Missing GitLab handle for maintainer 'Weiwei Li'
Missing GitLab handle for maintainer 'Xianglai Li'
Missing GitLab handle for maintainer 'Xie Yongji'
Missing GitLab handle for maintainer 'Yi Liu'
Missing GitLab handle for maintainer 'Yoshinori Sato'
Missing GitLab handle for maintainer 'Yuri Benditovich'
Missing GitLab handle for maintainer 'Zhang Chen'
Missing GitLab handle for maintainer 'Zhao Liu'
Missing GitLab handle for maintainer 'zhenwei pi'
Missing GitLab handle for maintainer 'Zhenzhong Duan'
Missing GitLab handle for maintainer 'Zhuoying Cai'
GitLab handles missing: 158 / present: 68

Plenty of those will have GitLab accounts, but are simply not members
of QEMU. Plenty more probably aren't registered with GitLab at all.

Short of emailing all of those people directly, I don't see a good
way to identify their GitLab accounts (if any), in order to invite
them to the QEMU project. I don't want to blindly invite accounts
which merely have the same name on GitLab as there's too much risk
of getting a mis-match.

Changed in v3:

 - Validate handle in .gitlab-map-manual maps to one in
   .gitlab-map-auto
 - Validate real name in .gitlab-map-manual maps to one in
   MAINTANIERS
 - Fix stripping of quotes in names
 - Fix trailing whitespace

Changed in v2:

 - Don't store data in MAINTAINERS file, use a separate mapping
 - Don't require a --gitlab flag to get_maintainer.pl, always
   show the handle
 - Add scripts for updating & analysing the mappings

Daniel P. Berrangé (2):
  gitlab: introduce files mapping GitLab accounts to real names
  get_maintainer: add ability to report Git Lab handle

 .gitlab-map-auto          | 93 +++++++++++++++++++++++++++++++++++++++
 .gitlab-map-manual        | 18 ++++++++
 scripts/get_maintainer.pl | 51 +++++++++++++++++++--
 scripts/gitlab-map-check  | 51 +++++++++++++++++++++
 scripts/gitlab-map-update | 45 +++++++++++++++++++
 5 files changed, 254 insertions(+), 4 deletions(-)
 create mode 100644 .gitlab-map-auto
 create mode 100644 .gitlab-map-manual
 create mode 100755 scripts/gitlab-map-check
 create mode 100755 scripts/gitlab-map-update

-- 
2.55.0

[PATCH v3 1/2] gitlab: introduce files mapping GitLab accounts to real names

[Daniel P. Berrangé]

It is desirable to be able to discover the GitLab account handle
assocaited with a real name in the MAINTAINERS file.

Rather that duplicating the same account handle multiple times,
inline with the MAINTAINERS file entries, this introduces mapping
files:

  * .gitlab-map-auto - data automatically queried from GitLab
    using the 'glab' tool and REST API
  * .gitlab-map-manual - manual overrides/augmentation for
    cases where the MAINTAINERS real name does not match the
    GitLab account real name

The former would need refreshing when we add new MAINTAINERS
entries, if the person had to be added as a GitLab account
member. For this purpose scripts/gitlab-map-update can be
used, assuming the user has the 'glab' client tool present
and configured with an access token.

To audit how many maintainers have GitLab handles present/missing
scripts/gitlab-map-check can run a report.

  $ ./scripts/gitlab-map-check
  Missing GitLab handle for maintainer 'Akihiko Odaki'
  Missing GitLab handle for maintainer 'Albert Esteve'
  ....
  Missing GitLab handle for maintainer 'Zhenzhong Duan'
  Missing GitLab handle for maintainer 'Zhuoying Cai'
  GitLab handles missing: 158 / present: 68

Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
---
 .gitlab-map-auto          | 93 +++++++++++++++++++++++++++++++++++++++
 .gitlab-map-manual        | 18 ++++++++
 scripts/gitlab-map-check  | 51 +++++++++++++++++++++
 scripts/gitlab-map-update | 45 +++++++++++++++++++
 4 files changed, 207 insertions(+)
 create mode 100644 .gitlab-map-auto
 create mode 100644 .gitlab-map-manual
 create mode 100755 scripts/gitlab-map-check
 create mode 100755 scripts/gitlab-map-update

diff --git a/.gitlab-map-auto b/.gitlab-map-auto
new file mode 100644
index 0000000000..c15c301435
--- /dev/null
+++ b/.gitlab-map-auto
@@ -0,0 +1,93 @@
+# This file is auto-generated by scripts/autolab-map-update
+#
+# This GitLab map associates GitLab account handles
+# with real names, in order to allow mapping from
+# MAINTAINERS entries. The format of entries is
+#
+#    {gitlab-handle}<tab>{real name}
+#
+# Manual overrides must be placed in .gitlab-map-manual
+TaoTang	Tao Tang
+a1xndr	Alexander Bulekov
+adi-g15-ibm	Aditya Gupta
+agraf	Alexander Graf
+alex.williamson	Alex Williamson
+aliang1	Aihua Liang
+alistair23	Alistair Francis
+anisinha	Ani Sinha
+anthony-linaro	Anthony Roberts
+anthonyper	Anthony PERARD
+berrange	Daniel P. Berrangé
+birkelund	Klaus Jensen
+bonzini	Paolo Bonzini
+brian-cain	Brian Cain
+bsdimp	Warner Losh
+cborntra	Christian Borntraeger
+chao23.liu	Chao Liu (Zevorn)
+cleber.gnu	Cleber Rosa
+clegoate	Cédric Le Goater
+cohuck	Cornelia Huck
+cota_	Emilio Cota
+dagrh	Dr. David Alan Gilbert
+danielhb	Daniel Henrique Barboza
+davidhildenbrand	David Hildenbrand
+dgibson	dgibson
+dwmw2	David Woodhouse
+eauger1	Eric Auger
+ebblake	Eric Blake
+edgar.iglesias	Edgar E. Iglesias
+ehabkost	Eduardo Habkost
+eldondev	Eldon
+epilys	Manos Pitsidianakis
+famzheng	Fam Zheng
+farosas	Fabiano Rosas
+gautammenghani	Gautam Menghani
+gkurz	Greg Kurz
+gusbromero	Gustavo Romero
+harshpb	Harsh Prateek Bora
+hdeller	Helge Deller
+hreitz	Hanna Czenczek
+imammedo	Igor Mammedov
+jasowang	Jason Wang
+jmacarthur	Jim MacArthur
+jsnow	John Snow
+juan.quintela	Juan Quintela
+kbastian-qemu	Bastian Koppelmann
+kmwolf	Kevin Wolf
+kostyanf14	Kostiantyn Kostiuk
+kraxel	Gerd Hoffmann
+lbmeng	Bin Meng
+legoater	Cédric Le Goater
+lvivier	Laurent Vivier
+lygstate	Yonggang Luo
+maciejsszmigiero	Maciej S. Szmigiero
+marcandre.lureau	Marc-André Lureau
+marcandre.lureau-rh	Marc-André Lureau
+mauromatteo.cascella	Mauro Matteo Cascella
+mcayland	Mark Cave-Ayland
+mdroth	Michael Roth
+mediouni-m	M. Mediouni
+mjt0k	Michael Tokarev
+mstredhat	MST
+npiggin	npiggin
+p-b-o	Pierrick Bouvier
+pauldzim	Paul Zimmerman
+peterx	Peter Xu
+philmd	Philippe Mathieu-Daudé
+pierrick.bouvier	Pierrick Bouvier
+pipo.sk	Peter Krempa
+pkrempa	Peter Krempa (work)
+pm215	Peter Maydell
+qemu-janitor	Qemu Janitor
+rathc	Chinmay Rath
+rth7680	Richard Henderson
+schoenebeck	Christian Schoenebeck
+sgarzarella	Stefano Garzarella
+sstabellini	Stefano Stabellini
+stefanberger	Stefan Berger
+stefanha	Stefan Hajnoczi
+stsquad	Alex Bennée
+stweil	Stefan Weil
+thuth	Thomas Huth
+vsementsov	Vladimir Sementsov-Ogievskiy
+xcancerberox	Joaquin de Andres
diff --git a/.gitlab-map-manual b/.gitlab-map-manual
new file mode 100644
index 0000000000..1551e2f55d
--- /dev/null
+++ b/.gitlab-map-manual
@@ -0,0 +1,18 @@
+# This GitLab map associates GitLab account handles
+# with real names, in order to allow mapping from
+# MAINTAINERS entries. The format of entries is
+#
+#    {gitlab-handle}<tab>{real name}
+#
+# This file is manually written, to augment the
+# auto-generated data in .gitlab-map-auto. This
+# needed where a GitLab account real name does
+# not exactly match the MAINTAINERS file real
+# name.
+#
+berrange	Daniel P. Berrange
+dgibson	David Gibson
+hreitz	Hanna Reitz
+mstredhat	Michael S. Tsirkin
+npiggin	Nicholas Piggin
+mediouni-m	Mohamed Mediouni
diff --git a/scripts/gitlab-map-check b/scripts/gitlab-map-check
new file mode 100755
index 0000000000..ff12e36593
--- /dev/null
+++ b/scripts/gitlab-map-check
@@ -0,0 +1,51 @@
+#!/bin/sh
+# SPDX-License-Identifier: GPL-2.0-or-later
+
+missing=0
+present=0
+
+grep -E '^(R|M):' MAINTAINERS | \
+sed -E -e 's/^(M|R): //' -e 's/ <.*//' | \
+sort | uniq > names.txt
+while IFS= read -r NAME
+do
+  grep "$NAME" .gitlab-map-{auto,manual} > /dev/null
+  if test $? != 0
+  then
+    echo "Missing GitLab handle for maintainer '$NAME'"
+    missing=$(($missing + 1))
+  else
+    present=$(($present + 1))
+  fi
+done < names.txt
+rm -f names.txt
+
+echo "GitLab handles missing: $missing / present: $present"
+
+# .gitlab-map-manual is to provide alternate real names
+# for cases where name in MANITAINERS does not match
+# the GitLab account real name. As such...
+
+# ...all gitlab handles in .gitlab-map-manual must also exist
+# in .gitlab-map-auto and ....
+for HANDLE in $(grep -v '^#' .gitlab-map-manual | awk '{print $1}')
+do
+    grep -E "^$HANDLE\s" .gitlab-map-auto >/dev/null
+    if test $? != 0
+    then
+	echo "Unexpected GitLab handle '$HANDLE' is not a member"
+    fi
+done
+
+# ...and all realnames in .gitlab-map-manual must also
+# exist in MAINTAINERS
+grep -v '^#' .gitlab-map-manual | \
+awk '{ saved = $1; $1 = ""; print substr($0, 2) }' | \
+while IFS= read -r NAME
+do
+    grep -E "\s$NAME\s" MAINTAINERS >/dev/null
+    if test $? != 0
+    then
+	echo "Unexpected real name '$NAME' is not a maintainer"
+    fi
+done
diff --git a/scripts/gitlab-map-update b/scripts/gitlab-map-update
new file mode 100755
index 0000000000..c134c226aa
--- /dev/null
+++ b/scripts/gitlab-map-update
@@ -0,0 +1,45 @@
+#!/bin/sh
+# SPDX-License-Identifier: GPL-2.0-or-later
+
+echo "Checking authentication token status"
+glab auth status
+
+if test $? != 0
+then
+  echo "ERROR: not authenticated with gitlab.com"
+  exit 1
+fi
+
+if ! test -f .gitlab-map-auto
+then
+  echo "ERROR: cannot find existing '.gitlab-map-auto file'"
+  echo "ERROR: this must be executed from the root of the"
+  echo "ERROR: git repository checkout"
+  exit 1
+fi
+
+fail=0
+echo "Updating member list from qemu-project"
+glab api --paginate /groups/qemu-project/members | \
+    jq -r -c '.[] | [.username, .name] | @tsv' | \
+    grep -v '^group_3038080_bot' > \
+    .gitlab-map-auto.tmp
+test $? != 0 && fail=1
+
+echo "Updating member list from qemu-project/qemu"
+glab api --paginate /projects/qemu-project%2fqemu/members | \
+    jq -r -c '.[] | [.username, .name] | @tsv' | \
+    grep -v 'Duo Developer' >> \
+    .gitlab-map-auto.tmp
+test $? != 0 && fail=1
+
+if test $fail == 0
+then
+  grep '^#' .gitlab-map-auto > .gitlab-map-auto.new
+  LC_ALL=C sort .gitlab-map-auto.tmp | uniq >> .gitlab-map-auto.new
+  mv .gitlab-map-auto.new .gitlab-map-auto
+  echo "OK: GitLab member list updated"
+else
+  echo "ERROR: Updating member list failed"
+fi
+rm -f .gitlab-map-auto.tmp
-- 
2.55.0

[PATCH v3 2/2] get_maintainer: add ability to report Git Lab handle

[Daniel P. Berrangé]

With the GitLab mapping files from the previous commit, the
get_manitainer.pl script is now able to report the gitlab
handle for each maintainer/reviewer when displaying output.

For example:

  $ ./scripts/get_maintainer.pl -f hw/scsi/lsi53c895a.c
  Paolo Bonzini <pbonzini@redhat.com> (supporter:SCSI, gitlab:@bonzini)
  Fam Zheng <fam@euphon.net> (reviewer:SCSI, gitlab:@famzheng)
  qemu-devel@nongnu.org (open list:All patches CC here)

Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
---
 scripts/get_maintainer.pl | 51 ++++++++++++++++++++++++++++++++++++---
 1 file changed, 47 insertions(+), 4 deletions(-)

diff --git a/scripts/get_maintainer.pl b/scripts/get_maintainer.pl
index 76be402e11..9679f27413 100755
--- a/scripts/get_maintainer.pl
+++ b/scripts/get_maintainer.pl
@@ -365,6 +365,31 @@ sub read_mailmap {
     close($mailmap_file);
 }
 
+my %gitlabmap;
+
+read_gitlabmap(".gitlab-map-auto");
+read_gitlabmap(".gitlab-map-manual");
+
+sub read_gitlabmap {
+    my $mapfile = shift;
+    open MAP, "<$mapfile"
+	or die "cannot open $mapfile: $!";
+
+    while (<MAP>) {
+	next if /^#/;
+	next if /^\s*$/;
+
+	if (/^(\S+)\t(.*)$/) {
+	    my $handle = $1;
+	    my $realname = $2;
+
+	    $gitlabmap{$realname} = $handle;
+	}
+    }
+
+    close MAP;
+}
+
 ## use the filenames on the command line or find the filenames in the patchfiles
 
 my @files = ();
@@ -1068,6 +1093,18 @@ sub email_inuse {
     return 0;
 }
 
+sub gitlab_handle {
+    my $name = shift;
+
+    $name =~ s/"//g;
+
+    my $gitlab_handle;
+    if (exists $gitlabmap{$name}) {
+	return $gitlabmap{$name};
+    }
+    return undef;
+}
+
 sub push_email_address {
     my ($line, $role) = @_;
 
@@ -1077,10 +1114,12 @@ sub push_email_address {
 	return 0;
     }
 
+    my $gitlab_handle = gitlab_handle($name);
+
     if (!$email_remove_duplicates) {
-	push(@email_to, [format_email($name, $address, $email_usename), $role]);
+	push(@email_to, [format_email($name, $address, $email_usename), $role, $gitlab_handle]);
     } elsif (!email_inuse($name, $address)) {
-	push(@email_to, [format_email($name, $address, $email_usename), $role]);
+	push(@email_to, [format_email($name, $address, $email_usename), $role, $gitlab_handle]);
 	$email_hash_name{lc($name)}++ if ($name ne "");
 	$email_hash_address{lc($address)}++;
     }
@@ -2026,10 +2065,14 @@ sub merge_email {
     my %saw;
 
     for (@_) {
-	my ($address, $role) = @$_;
+	my ($address, $role, $gitlab_handle) = @$_;
 	if (!$saw{$address}) {
 	    if ($output_roles) {
-		push(@lines, "$address ($role)");
+		if (defined $gitlab_handle) {
+		    push(@lines, "$address ($role, gitlab:\@$gitlab_handle)");
+		} else {
+		    push(@lines, "$address ($role)");
+		}
 	    } else {
 		push(@lines, $address);
 	    }
-- 
2.55.0

Re: [PATCH v3 1/2] gitlab: introduce files mapping GitLab accounts to real names

[Fabiano Rosas]

Daniel P. Berrangé <berrange@redhat.com> writes:

> It is desirable to be able to discover the GitLab account handle
> assocaited with a real name in the MAINTAINERS file.
>
> Rather that duplicating the same account handle multiple times,
> inline with the MAINTAINERS file entries, this introduces mapping
> files:
>
>   * .gitlab-map-auto - data automatically queried from GitLab
>     using the 'glab' tool and REST API
>   * .gitlab-map-manual - manual overrides/augmentation for
>     cases where the MAINTAINERS real name does not match the
>     GitLab account real name
>
> The former would need refreshing when we add new MAINTAINERS
> entries, if the person had to be added as a GitLab account
> member. For this purpose scripts/gitlab-map-update can be
> used, assuming the user has the 'glab' client tool present
> and configured with an access token.
>
> To audit how many maintainers have GitLab handles present/missing
> scripts/gitlab-map-check can run a report.
>
>   $ ./scripts/gitlab-map-check
>   Missing GitLab handle for maintainer 'Akihiko Odaki'
>   Missing GitLab handle for maintainer 'Albert Esteve'
>   ....
>   Missing GitLab handle for maintainer 'Zhenzhong Duan'
>   Missing GitLab handle for maintainer 'Zhuoying Cai'
>   GitLab handles missing: 158 / present: 68
>
> Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
> ---
>  .gitlab-map-auto          | 93 +++++++++++++++++++++++++++++++++++++++
>  .gitlab-map-manual        | 18 ++++++++
>  scripts/gitlab-map-check  | 51 +++++++++++++++++++++
>  scripts/gitlab-map-update | 45 +++++++++++++++++++
>  4 files changed, 207 insertions(+)
>  create mode 100644 .gitlab-map-auto
>  create mode 100644 .gitlab-map-manual
>  create mode 100755 scripts/gitlab-map-check
>  create mode 100755 scripts/gitlab-map-update
>
> diff --git a/.gitlab-map-auto b/.gitlab-map-auto
> new file mode 100644
> index 0000000000..c15c301435
> --- /dev/null
> +++ b/.gitlab-map-auto
> @@ -0,0 +1,93 @@
> +# This file is auto-generated by scripts/autolab-map-update

gitlab

> +#
> +# This GitLab map associates GitLab account handles
> +# with real names, in order to allow mapping from
> +# MAINTAINERS entries. The format of entries is
> +#
> +#    {gitlab-handle}<tab>{real name}
> +#
> +# Manual overrides must be placed in .gitlab-map-manual
> +TaoTang	Tao Tang
> +a1xndr	Alexander Bulekov
> +adi-g15-ibm	Aditya Gupta
> +agraf	Alexander Graf
> +alex.williamson	Alex Williamson
> +aliang1	Aihua Liang
> +alistair23	Alistair Francis
> +anisinha	Ani Sinha
> +anthony-linaro	Anthony Roberts
> +anthonyper	Anthony PERARD
> +berrange	Daniel P. Berrangé
> +birkelund	Klaus Jensen
> +bonzini	Paolo Bonzini
> +brian-cain	Brian Cain
> +bsdimp	Warner Losh
> +cborntra	Christian Borntraeger
> +chao23.liu	Chao Liu (Zevorn)
> +cleber.gnu	Cleber Rosa
> +clegoate	Cédric Le Goater
> +cohuck	Cornelia Huck
> +cota_	Emilio Cota
> +dagrh	Dr. David Alan Gilbert
> +danielhb	Daniel Henrique Barboza
> +davidhildenbrand	David Hildenbrand
> +dgibson	dgibson
> +dwmw2	David Woodhouse
> +eauger1	Eric Auger
> +ebblake	Eric Blake
> +edgar.iglesias	Edgar E. Iglesias
> +ehabkost	Eduardo Habkost
> +eldondev	Eldon
> +epilys	Manos Pitsidianakis
> +famzheng	Fam Zheng
> +farosas	Fabiano Rosas
> +gautammenghani	Gautam Menghani
> +gkurz	Greg Kurz
> +gusbromero	Gustavo Romero
> +harshpb	Harsh Prateek Bora
> +hdeller	Helge Deller
> +hreitz	Hanna Czenczek
> +imammedo	Igor Mammedov
> +jasowang	Jason Wang
> +jmacarthur	Jim MacArthur
> +jsnow	John Snow
> +juan.quintela	Juan Quintela
> +kbastian-qemu	Bastian Koppelmann
> +kmwolf	Kevin Wolf
> +kostyanf14	Kostiantyn Kostiuk
> +kraxel	Gerd Hoffmann
> +lbmeng	Bin Meng
> +legoater	Cédric Le Goater
> +lvivier	Laurent Vivier
> +lygstate	Yonggang Luo
> +maciejsszmigiero	Maciej S. Szmigiero
> +marcandre.lureau	Marc-André Lureau
> +marcandre.lureau-rh	Marc-André Lureau
> +mauromatteo.cascella	Mauro Matteo Cascella
> +mcayland	Mark Cave-Ayland
> +mdroth	Michael Roth
> +mediouni-m	M. Mediouni
> +mjt0k	Michael Tokarev
> +mstredhat	MST
> +npiggin	npiggin
> +p-b-o	Pierrick Bouvier
> +pauldzim	Paul Zimmerman
> +peterx	Peter Xu
> +philmd	Philippe Mathieu-Daudé
> +pierrick.bouvier	Pierrick Bouvier
> +pipo.sk	Peter Krempa
> +pkrempa	Peter Krempa (work)
> +pm215	Peter Maydell
> +qemu-janitor	Qemu Janitor
> +rathc	Chinmay Rath
> +rth7680	Richard Henderson
> +schoenebeck	Christian Schoenebeck
> +sgarzarella	Stefano Garzarella
> +sstabellini	Stefano Stabellini
> +stefanberger	Stefan Berger
> +stefanha	Stefan Hajnoczi
> +stsquad	Alex Bennée
> +stweil	Stefan Weil
> +thuth	Thomas Huth
> +vsementsov	Vladimir Sementsov-Ogievskiy
> +xcancerberox	Joaquin de Andres
> diff --git a/.gitlab-map-manual b/.gitlab-map-manual
> new file mode 100644
> index 0000000000..1551e2f55d
> --- /dev/null
> +++ b/.gitlab-map-manual
> @@ -0,0 +1,18 @@
> +# This GitLab map associates GitLab account handles
> +# with real names, in order to allow mapping from
> +# MAINTAINERS entries. The format of entries is
> +#
> +#    {gitlab-handle}<tab>{real name}
> +#
> +# This file is manually written, to augment the
> +# auto-generated data in .gitlab-map-auto. This

This is

> +# needed where a GitLab account real name does
> +# not exactly match the MAINTAINERS file real
> +# name.
> +#
> +berrange	Daniel P. Berrange
> +dgibson	David Gibson
> +hreitz	Hanna Reitz
> +mstredhat	Michael S. Tsirkin
> +npiggin	Nicholas Piggin
> +mediouni-m	Mohamed Mediouni
> diff --git a/scripts/gitlab-map-check b/scripts/gitlab-map-check
> new file mode 100755
> index 0000000000..ff12e36593
> --- /dev/null
> +++ b/scripts/gitlab-map-check
> @@ -0,0 +1,51 @@
> +#!/bin/sh
> +# SPDX-License-Identifier: GPL-2.0-or-later
> +
> +missing=0
> +present=0
> +
> +grep -E '^(R|M):' MAINTAINERS | \
> +sed -E -e 's/^(M|R): //' -e 's/ <.*//' | \
> +sort | uniq > names.txt
> +while IFS= read -r NAME
> +do
> +  grep "$NAME" .gitlab-map-{auto,manual} > /dev/null
> +  if test $? != 0
> +  then
> +    echo "Missing GitLab handle for maintainer '$NAME'"
> +    missing=$(($missing + 1))
> +  else
> +    present=$(($present + 1))
> +  fi
> +done < names.txt
> +rm -f names.txt
> +
> +echo "GitLab handles missing: $missing / present: $present"
> +
> +# .gitlab-map-manual is to provide alternate real names
> +# for cases where name in MANITAINERS does not match

MAINTAINERS

> +# the GitLab account real name. As such...
> +
> +# ...all gitlab handles in .gitlab-map-manual must also exist
> +# in .gitlab-map-auto and ....
> +for HANDLE in $(grep -v '^#' .gitlab-map-manual | awk '{print $1}')
> +do
> +    grep -E "^$HANDLE\s" .gitlab-map-auto >/dev/null
> +    if test $? != 0
> +    then
> +	echo "Unexpected GitLab handle '$HANDLE' is not a member"
> +    fi
> +done
> +
> +# ...and all realnames in .gitlab-map-manual must also
> +# exist in MAINTAINERS
> +grep -v '^#' .gitlab-map-manual | \
> +awk '{ saved = $1; $1 = ""; print substr($0, 2) }' | \

Is 'saved' doing anything here?

> +while IFS= read -r NAME
> +do
> +    grep -E "\s$NAME\s" MAINTAINERS >/dev/null
> +    if test $? != 0
> +    then
> +	echo "Unexpected real name '$NAME' is not a maintainer"
> +    fi
> +done
> diff --git a/scripts/gitlab-map-update b/scripts/gitlab-map-update
> new file mode 100755
> index 0000000000..c134c226aa
> --- /dev/null
> +++ b/scripts/gitlab-map-update
> @@ -0,0 +1,45 @@
> +#!/bin/sh
> +# SPDX-License-Identifier: GPL-2.0-or-later
> +
> +echo "Checking authentication token status"
> +glab auth status
> +
> +if test $? != 0
> +then
> +  echo "ERROR: not authenticated with gitlab.com"
> +  exit 1
> +fi
> +
> +if ! test -f .gitlab-map-auto
> +then
> +  echo "ERROR: cannot find existing '.gitlab-map-auto file'"
> +  echo "ERROR: this must be executed from the root of the"
> +  echo "ERROR: git repository checkout"
> +  exit 1
> +fi
> +
> +fail=0
> +echo "Updating member list from qemu-project"
> +glab api --paginate /groups/qemu-project/members | \
> +    jq -r -c '.[] | [.username, .name] | @tsv' | \
> +    grep -v '^group_3038080_bot' > \
> +    .gitlab-map-auto.tmp
> +test $? != 0 && fail=1
> +
> +echo "Updating member list from qemu-project/qemu"
> +glab api --paginate /projects/qemu-project%2fqemu/members | \
> +    jq -r -c '.[] | [.username, .name] | @tsv' | \
> +    grep -v 'Duo Developer' >> \
> +    .gitlab-map-auto.tmp
> +test $? != 0 && fail=1
> +
> +if test $fail == 0
> +then
> +  grep '^#' .gitlab-map-auto > .gitlab-map-auto.new
> +  LC_ALL=C sort .gitlab-map-auto.tmp | uniq >> .gitlab-map-auto.new
> +  mv .gitlab-map-auto.new .gitlab-map-auto
> +  echo "OK: GitLab member list updated"
> +else
> +  echo "ERROR: Updating member list failed"
> +fi
> +rm -f .gitlab-map-auto.tmp

Re: [PATCH v3 2/2] get_maintainer: add ability to report Git Lab handle

[Fabiano Rosas]

Daniel P. Berrangé <berrange@redhat.com> writes:

> With the GitLab mapping files from the previous commit, the
> get_manitainer.pl script is now able to report the gitlab
> handle for each maintainer/reviewer when displaying output.
>
> For example:
>
>   $ ./scripts/get_maintainer.pl -f hw/scsi/lsi53c895a.c
>   Paolo Bonzini <pbonzini@redhat.com> (supporter:SCSI, gitlab:@bonzini)
>   Fam Zheng <fam@euphon.net> (reviewer:SCSI, gitlab:@famzheng)
>   qemu-devel@nongnu.org (open list:All patches CC here)
>
> Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
> ---
>  scripts/get_maintainer.pl | 51 ++++++++++++++++++++++++++++++++++++---
>  1 file changed, 47 insertions(+), 4 deletions(-)
>
> diff --git a/scripts/get_maintainer.pl b/scripts/get_maintainer.pl
> index 76be402e11..9679f27413 100755
> --- a/scripts/get_maintainer.pl
> +++ b/scripts/get_maintainer.pl
> @@ -365,6 +365,31 @@ sub read_mailmap {
>      close($mailmap_file);
>  }
>  
> +my %gitlabmap;
> +
> +read_gitlabmap(".gitlab-map-auto");
> +read_gitlabmap(".gitlab-map-manual");
> +
> +sub read_gitlabmap {
> +    my $mapfile = shift;
> +    open MAP, "<$mapfile"
> +	or die "cannot open $mapfile: $!";
> +
> +    while (<MAP>) {
> +	next if /^#/;
> +	next if /^\s*$/;
> +
> +	if (/^(\S+)\t(.*)$/) {
> +	    my $handle = $1;
> +	    my $realname = $2;
> +
> +	    $gitlabmap{$realname} = $handle;
> +	}
> +    }
> +
> +    close MAP;
> +}
> +
>  ## use the filenames on the command line or find the filenames in the patchfiles
>  
>  my @files = ();
> @@ -1068,6 +1093,18 @@ sub email_inuse {
>      return 0;
>  }
>  
> +sub gitlab_handle {
> +    my $name = shift;
> +
> +    $name =~ s/"//g;
> +
> +    my $gitlab_handle;
> +    if (exists $gitlabmap{$name}) {
> +	return $gitlabmap{$name};
> +    }
> +    return undef;
> +}
> +
>  sub push_email_address {
>      my ($line, $role) = @_;
>  
> @@ -1077,10 +1114,12 @@ sub push_email_address {
>  	return 0;
>      }
>  
> +    my $gitlab_handle = gitlab_handle($name);
> +
>      if (!$email_remove_duplicates) {
> -	push(@email_to, [format_email($name, $address, $email_usename), $role]);
> +	push(@email_to, [format_email($name, $address, $email_usename), $role, $gitlab_handle]);
>      } elsif (!email_inuse($name, $address)) {
> -	push(@email_to, [format_email($name, $address, $email_usename), $role]);
> +	push(@email_to, [format_email($name, $address, $email_usename), $role, $gitlab_handle]);
>  	$email_hash_name{lc($name)}++ if ($name ne "");
>  	$email_hash_address{lc($address)}++;
>      }
> @@ -2026,10 +2065,14 @@ sub merge_email {
>      my %saw;
>  
>      for (@_) {
> -	my ($address, $role) = @$_;
> +	my ($address, $role, $gitlab_handle) = @$_;
>  	if (!$saw{$address}) {
>  	    if ($output_roles) {
> -		push(@lines, "$address ($role)");
> +		if (defined $gitlab_handle) {
> +		    push(@lines, "$address ($role, gitlab:\@$gitlab_handle)");
> +		} else {
> +		    push(@lines, "$address ($role)");
> +		}
>  	    } else {
>  		push(@lines, $address);
>  	    }

Reviewed-by: Fabiano Rosas <farosas@suse.de>