Re: [PATCH v2 1/2] gitlab: introduce files mapping GitLab accounts to real names

["Alex Bennée" <alex.bennee@linaro.org>]

Daniel P. Berrangé <berrange@redhat.com> writes:

> It is desirable to be able to discover the GitLab account handle
> assocaited with a real name in the MAINTAINERS file.
>
> Rather that duplicating the same account handle multiple times,
> inline with the MAINTAINERS file entries, this introduces mapping
> files:
>
>   * .gitlab-map-auto - data automatically queried from GitLab
>     using the 'glab' tool and REST API
>   * .gitlab-map-manual - manual overrides/augmentation for
>     cases where the MAINTAINERS real name does not match the
>     GitLab account real name
>
> The former would need refreshing when we add new MAINTAINERS
> entries, if the person had to be added as a GitLab account
> member. For this purpose scripts/gitlab-map-update can be
> used, assuming the user has the 'glab' client tool present
> and configured with an access token.
>
> To audit how many maintainers have GitLab handles present/missing
> scripts/gitlab-map-check can run a report.
>
>   $ ./scripts/gitlab-map-check
>   Missing GitLab handle: Akihiko Odaki
>   Missing GitLab handle: Albert Esteve
>   ....
>   Missing GitLab handle: Zhenzhong Duan
>   Missing GitLab handle: Zhuoying Cai
>   GitLab handles missing: 161 / present: 57
>
> Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
> ---
>  .gitlab-map-auto          | 83 +++++++++++++++++++++++++++++++++++++++
>  .gitlab-map-manual        | 18 +++++++++
>  scripts/gitlab-map-check  | 23 +++++++++++
>  scripts/gitlab-map-update | 44 +++++++++++++++++++++
>  4 files changed, 168 insertions(+)
>  create mode 100644 .gitlab-map-auto
>  create mode 100644 .gitlab-map-manual
>  create mode 100755 scripts/gitlab-map-check
>  create mode 100755 scripts/gitlab-map-update
>
> diff --git a/.gitlab-map-auto b/.gitlab-map-auto
> new file mode 100644
> index 0000000000..defecb2c9f
> --- /dev/null
> +++ b/.gitlab-map-auto
> @@ -0,0 +1,83 @@
> +# This file is auto-generated by scripts/autolab-map-update
> +#
> +# This GitLab map associates GitLab account handles
> +# with real names, in order to allow mapping from
> +# MAINTAINERS entries. The format of entries is
> +#
> +#    {gitlab-handle}<tab>{real name}
> +#
> +# Manual overrides must be placed in .gitlab-map-manual
> +a1xndr	Alexander Bulekov
> +adi-g15-ibm	Aditya Gupta
> +agraf	Alexander Graf
> +alex.williamson	Alex Williamson
> +aliang1	Aihua Liang
> +alistair23	Alistair Francis
> +anisinha	Ani Sinha
> +anthony-linaro	Anthony Roberts
> +berrange	Daniel P. Berrangé
> +birkelund	Klaus Jensen
> +bonzini	Paolo Bonzini
> +bonzini	Paolo Bonzini
> +brian-cain	Brian Cain
> +bsdimp	Warner Losh
> +cborntra	Christian Borntraeger
> +cleber.gnu	Cleber Rosa
> +clegoate	Cédric Le Goater
> +cohuck	Cornelia Huck
> +cota_	Emilio Cota
> +dagrh	Dr. David Alan Gilbert
> +danielhb	Daniel Henrique Barboza
> +davidhildenbrand	David Hildenbrand
> +dgibson	dgibson
> +dwmw2	David Woodhouse
> +ebblake	Eric Blake
> +ehabkost	Eduardo Habkost
> +eldondev	Eldon
> +epilys	Manos Pitsidianakis
> +famzheng	Fam Zheng
> +farosas	Fabiano Rosas
> +gkurz	Greg Kurz
> +gusbromero	Gustavo Romero
> +harshpb	Harsh Prateek Bora
> +hdeller	Helge Deller
> +hreitz	Hanna Czenczek
> +imammedo	Igor Mammedov
> +jasowang	Jason Wang
> +jmacarthur	Jim MacArthur
> +jsnow	John Snow
> +juan.quintela	Juan Quintela
> +kbastian-qemu	Bastian Koppelmann
> +kmwolf	Kevin Wolf
> +kostyanf14	Kostiantyn Kostiuk
> +kraxel	Gerd Hoffmann
> +lbmeng	Bin Meng
> +legoater	Cédric Le Goater
> +lvivier	Laurent Vivier
> +marcandre.lureau	Marc-André Lureau
> +marcandre.lureau-rh	Marc-André Lureau
> +mcayland	Mark Cave-Ayland
> +mdroth	Michael Roth
> +mediouni-m	M. Mediouni
> +mjt0k	Michael Tokarev
> +mstredhat	MST
> +npiggin	npiggin
> +pauldzim	Paul Zimmerman
> +philmd	Philippe Mathieu-Daudé
> +philmd	Philippe Mathieu-Daudé
> +pierrick.bouvier	Pierrick Bouvier
> +pipo.sk	Peter Krempa
> +pkrempa	Peter Krempa (work)
> +pm215	Peter Maydell
> +qemu-janitor	Qemu Janitor
> +rth7680	Richard Henderson
> +schoenebeck	Christian Schoenebeck
> +sgarzarella	Stefano Garzarella
> +stefanberger	Stefan Berger
> +stefanha	Stefan Hajnoczi
> +stefanha	Stefan Hajnoczi
> +stsquad	Alex Bennée
> +stweil	Stefan Weil
> +thuth	Thomas Huth
> +vsementsov	Vladimir Sementsov-Ogievskiy
> +xcancerberox	Joaquin de Andres
> diff --git a/.gitlab-map-manual b/.gitlab-map-manual
> new file mode 100644
> index 0000000000..7326971cdd
> --- /dev/null
> +++ b/.gitlab-map-manual
> @@ -0,0 +1,18 @@
> +# This GitLab map associates GitLab account handles
> +# with real names, in order to allow mapping from
> +# MAINTAINERS entries. The format of entries is
> +#
> +#    {gitlab-handle}<tab>{real name}
> +#
> +# This file is manually written, to augment the
> +# auto-generated data in .gitlab-map-auto. This
> +# needed where a GitLab account real name does
> +# not exactly match the MAINTAINERS file real
> +# name.
> +#
> +berrange	Daniel P. Berrange
> +dgibson	David Gibson
> +hreitz	Hanna Reitz 
> +mstredhat	Michael S. Tsirkin
> +npiggin	Nicholas Piggin
> +mediouni-m	Mohamed Mediouni
> diff --git a/scripts/gitlab-map-check b/scripts/gitlab-map-check
> new file mode 100755
> index 0000000000..69d5b8258a
> --- /dev/null
> +++ b/scripts/gitlab-map-check
> @@ -0,0 +1,23 @@
> +#!/bin/sh
> +# SPDX-License-Identifier: GPL-2.0-or-later
> +
> +missing=0
> +present=0
> +
> +grep -E '^(R|M):' MAINTAINERS | \
> +sed -E -e 's/^(M|R): //' -e 's/ <.*//' | \
> +sort | uniq > names.txt
> +while IFS= read -r NAME
> +do
> +  grep "$NAME" .gitlab-map-{auto,manual} > /dev/null
> +  if test $? != 0
> +  then
> +    echo "Missing GitLab handle: $NAME"
> +    missing=$(($missing + 1))
> +  else
> +    present=$(($present + 1))
> +  fi
> +done < names.txt
> +rm -f names.txt
> +
> +echo "GitLab handles missing: $missing / present: $present"
> diff --git a/scripts/gitlab-map-update b/scripts/gitlab-map-update
> new file mode 100755
> index 0000000000..9fe07a98ee
> --- /dev/null
> +++ b/scripts/gitlab-map-update
> @@ -0,0 +1,44 @@
> +#!/bin/sh
> +# SPDX-License-Identifier: GPL-2.0-or-later
> +
> +echo "Checking authentication token status"
> +glab auth status
> +
> +if test $? != 0
> +then
> +  echo "ERROR: not authenticated with gitlab.com"
> +  exit 1
> +fi
> +
> +if ! test -f .gitlab-map-auto
> +then
> +  echo "ERROR: cannot find existing '.gitlab-map-auto file'"
> +  echo "ERROR: this must be executed from the root of the"
> +  echo "ERROR: git repository checkout"
> +  exit 1
> +fi
> +
> +fail=0
> +echo "Updating member list from qemu-project"
> +glab api --paginate /groups/qemu-project/members | \
> +    jq -r -c '.[] | [.username, .name] | @tsv' | \
> +    grep -v '^group_3038080_bot' > \
> +    .gitlab-map-auto.tmp
> +test $? != 0 && fail=1
> +
> +echo "Updating member list from qemu-project/qemu"
> +glab api --paginate /projects/qemu-project%2fqemu/members | \
> +    jq -r -c '.[] | [.username, .name] | @tsv' >> \
> +    .gitlab-map-auto.tmp
> +test $? != 0 && fail=1
> +
> +if test $fail == 0

Testing with:

$ env GITLAB_TOKEN=(pass gitlab-qemu-api)  ./scripts/gitlab-map-update
Checking authentication token status
gitlab.com
  ✓ Logged in to gitlab.com as group_3038080_bot_40a04d3deae510d9bf50c5bebbb7bcd3 (/home/alex/.config/glab-cli/config.yml)
  ✓ Git operations for gitlab.com configured to use https protocol.
  ✓ API calls for gitlab.com are made over https protocol.
  ✓ REST API Endpoint: https://gitlab.com/api/v4/
  ✓ GraphQL Endpoint: https://gitlab.com/api/graphql/
  ! No token provided in configuration file.

! One of GITLAB_TOKEN, GITLAB_ACCESS_TOKEN, OAUTH_TOKEN environment variables is set. It will be used for all authentication.
Updating member list from qemu-project
Updating member list from qemu-project/qemu
./scripts/gitlab-map-update: 35: test: 0: unexpected operator
ERROR: Updating member list failed

> +then
> +  grep '^#' .gitlab-map-auto > .gitlab-map-auto.new
> +  LC_ALL=C sort .gitlab-map-auto.tmp >> .gitlab-map-auto.new
> +  mv .gitlab-map-auto.new .gitlab-map-auto
> +  echo "OK: GitLab member list updated"
> +else
> +  echo "ERROR: Updating member list failed"
> +fi
> +rm -f .gitlab-map-auto.tmp

-- 
Alex Bennée
Virtualisation Tech Lead @ Linaro