Re: [PATCH v2 1/1] scripts/release: make the script more robust, and release a source repository snapshot

[Petr Lautrbach <plautrba@redhat.com>]

Petr Lautrbach <plautrba@redhat.com> writes:

> Nicolas Iooss <nicolas.iooss@m4x.org> writes:
>
>> On Wed, Feb 3, 2021 at 10:34 AM Petr Lautrbach <plautrba@redhat.com> wrote:
>>>
>>> Nicolas Iooss <nicolas.iooss@m4x.org> writes:
>>>
>>> > Following Petr Lautrbach's suggestion, release a snapshot of the source
>>> > repository next to the individual archives which constitute a release.
>>> >
>>> > While at it, make scripts/release more robust:
>>> >
>>> > - Fix many warnings reported by shellcheck, by quoting strings.
>>> > - Use bash arrays for DIRS and DIRS_NEED_PREFIX
>>> > - Merge DIRS and DIRS_NEED_PREFIX into a single array, in order to
>>> >   produce SHA256 digests that are directly in alphabetical order, for
>>> >   https://github.com/SELinuxProject/selinux/wiki/Releases
>>> > - Use "set -e" in order to fail as soon as a command fails
>>> > - Change to the top-level directory at the start of the script, in order
>>> >   to be able to run it from anywhere.
>>> > - Use `cat $DIR/VERSION` and `git -C $DIR` instead of `cd $i ; cat VERSION`
>>> >   in order to prevent unexpected issues from directory change.
>>> >
>>> > Finally, if version tags already exists, re-use them. This enables using
>>> > this script to re-generate the release archive (and check that they
>>> > really match the git repository). Currently, running scripts/release
>>> > will produce the same archives as the ones published in the 3.2-rc1
>>> > release (with the same SHA256 digests as the ones on the release page,
>>> > https://github.com/SELinuxProject/selinux/wiki/Releases). This helps to
>>> > ensure that the behaviour of the script is still fine.
>>> >
>>> > Suggested-by: Petr Lautrbach <plautrba@redhat.com>
>>> > Signed-off-by: Nicolas Iooss <nicolas.iooss@m4x.org>
>>> > ---
>>> >  scripts/release | 95 +++++++++++++++++++++++++++++++------------------
>>> >  1 file changed, 60 insertions(+), 35 deletions(-)
>>> >
>>> > diff --git a/scripts/release b/scripts/release
>>> > index 895a0e1ca1a1..21e30ff54b80 100755
>>> > --- a/scripts/release
>>> > +++ b/scripts/release
>>> > @@ -1,43 +1,57 @@
>>> >  #!/bin/bash
>>> >
>>> > -PWD=`pwd`
>>> > -WIKIDIR=../selinux.wiki
>>> > -
>>> > -if [ \! -d $WIKIDIR ]; then
>>> > -    git clone git@github.com:SELinuxProject/selinux.wiki.git $WIKIDIR
>>> > -fi
>>> > +# Fail when a command fails
>>> > +set -e
>>> >
>>> > -RELEASE_TAG=`cat VERSION`
>>> > -DEST=releases/$RELEASE_TAG
>>> > -DIRS="libsepol libselinux libsemanage checkpolicy secilc policycoreutils mcstrans restorecond semodule-utils"
>>> > -DIRS_NEED_PREFIX="dbus gui python sandbox"
>>> > +# Ensure the script is running from the top level directory
>>> > +cd "$(dirname -- "$0")/.."
>>> >
>>> > -git tag -a $RELEASE_TAG -m "Release $RELEASE_TAG"
>>> > +WIKIDIR=../selinux.wiki
>>> >
>>> > -rm -rf $DEST
>>> > -mkdir -p $DEST
>>> > +if ! [ -d "$WIKIDIR" ]; then
>>> > +     git clone git@github.com:SELinuxProject/selinux.wiki.git "$WIKIDIR"
>>> > +fi
>>> >
>>> > -for i in $DIRS; do
>>> > -     cd $i
>>> > -     VERS=`cat VERSION`
>>> > -     ARCHIVE=$i-$VERS.tar.gz
>>> > -     git tag $i-$VERS > /dev/null 2>&1
>>> > -     git archive -o ../$DEST/$ARCHIVE --prefix=$i-$VERS/ $i-$VERS
>>> > -     cd ..
>>> > -done
>>> > +RELEASE_TAG="$(cat VERSION)"
>>> > +DEST="releases/$RELEASE_TAG"
>>> > +DIRS=(
>>> > +     checkpolicy
>>> > +     libselinux
>>> > +     libsemanage
>>> > +     libsepol
>>> > +     mcstrans
>>> > +     policycoreutils
>>> > +     restorecond
>>> > +     secilc
>>> > +     selinux-dbus
>>> > +     selinux-gui
>>> > +     selinux-python
>>> > +     selinux-sandbox
>>> > +     semodule-utils
>>> > +)
>>> > +
>>> > +if git rev-parse "$RELEASE_TAG" > /dev/null ; then
>>> > +     echo "Warning: tag $RELEASE_TAG already exists"
>>> > +else
>>> > +     git tag -a "$RELEASE_TAG" -m "Release $RELEASE_TAG"
>>> > +fi
>>>
>>> fatal: ambiguous argument '3.2-rc2': unknown revision or path not in the working tree.
>>> Use '--' to separate paths from revisions, like this:
>>> 'git <command> [<revision>...] -- [<file>...]'
>>>
>>>
>>> > -for i in $DIRS_NEED_PREFIX; do
>>> > -     cd $i
>>> > -     VERS=`cat VERSION`
>>> > -     ARCHIVE=selinux-$i-$VERS.tar.gz
>>> > -     git tag selinux-$i-$VERS > /dev/null 2>&1
>>> > -     git archive -o ../$DEST/$ARCHIVE --prefix=selinux-$i-$VERS/ selinux-$i-$VERS
>>> > -     cd ..
>>> > +rm -rf "$DEST"
>>> > +mkdir -p "$DEST"
>>> > +
>>> > +for COMPONENT in "${DIRS[@]}"; do
>>> > +     DIR="${COMPONENT#selinux-}"
>>> > +     VERS="$(cat "$DIR/VERSION")"
>>> > +     TAG="$COMPONENT-$VERS"
>>> > +     if git rev-parse "$TAG" > /dev/null ; then
>>> > +             echo "Warning: tag $TAG already exists"
>>> > +     else
>>> > +             git tag "$TAG" > /dev/null
>>> > +     fi
>>>
>>> fatal: ambiguous argument 'checkpolicy-3.2-rc2': unknown revision or path not in the working tree.
>>> Use '--' to separate paths from revisions, like this:
>>> 'git <command> [<revision>...] -- [<file>...]'
>>>
>>> The following change fixes both:
>>>
>>> --- a/scripts/release
>>> +++ b/scripts/release
>>> @@ -30,7 +30,7 @@ DIRS=(
>>>         semodule-utils
>>>  )
>>>
>>> -if git rev-parse "$RELEASE_TAG" > /dev/null ; then
>>> +if git rev-parse "$RELEASE_TAG" &> /dev/null ; then
>>>         echo "Warning: tag $RELEASE_TAG already exists"
>>>  else
>>>         git tag -a "$RELEASE_TAG" -m "Release $RELEASE_TAG"
>>> @@ -43,7 +43,7 @@ for COMPONENT in "${DIRS[@]}"; do
>>>         DIR="${COMPONENT#selinux-}"
>>>         VERS="$(cat "$DIR/VERSION")"
>>>         TAG="$COMPONENT-$VERS"
>>> -       if git rev-parse "$TAG" > /dev/null ; then
>>> +       if git rev-parse "$TAG" &> /dev/null ; then
>>>                 echo "Warning: tag $TAG already exists"
>>>         else
>>>                 git tag "$TAG" > /dev/null
>>
>> Oops, indeed. The errors do not seem to be fatal, but yes, it is
>> better to hide them, like you suggest. You can modify my patch with
>> this change, or write a patch.
>>
>> Anyway, your suggestion looks good to me.
>
>
> I'm just preparing 3.2-rc2 so I'm going to modify your patch, merge and
> use it for the new release. 
>
>

It's merged now. Thanks!