Re: [PATCH v2 1/1] scripts/release: make the script more robust, and release a source repository snapshot

[Petr Lautrbach <plautrba@redhat.com>]

Nicolas Iooss <nicolas.iooss@m4x.org> writes:

> On Wed, Feb 3, 2021 at 10:34 AM Petr Lautrbach <plautrba@redhat.com> wrote:
>>
>> Nicolas Iooss <nicolas.iooss@m4x.org> writes:
>>
>> > Following Petr Lautrbach's suggestion, release a snapshot of the source
>> > repository next to the individual archives which constitute a release.
>> >
>> > While at it, make scripts/release more robust:
>> >
>> > - Fix many warnings reported by shellcheck, by quoting strings.
>> > - Use bash arrays for DIRS and DIRS_NEED_PREFIX
>> > - Merge DIRS and DIRS_NEED_PREFIX into a single array, in order to
>> >   produce SHA256 digests that are directly in alphabetical order, for
>> >   https://github.com/SELinuxProject/selinux/wiki/Releases
>> > - Use "set -e" in order to fail as soon as a command fails
>> > - Change to the top-level directory at the start of the script, in order
>> >   to be able to run it from anywhere.
>> > - Use `cat $DIR/VERSION` and `git -C $DIR` instead of `cd $i ; cat VERSION`
>> >   in order to prevent unexpected issues from directory change.
>> >
>> > Finally, if version tags already exists, re-use them. This enables using
>> > this script to re-generate the release archive (and check that they
>> > really match the git repository). Currently, running scripts/release
>> > will produce the same archives as the ones published in the 3.2-rc1
>> > release (with the same SHA256 digests as the ones on the release page,
>> > https://github.com/SELinuxProject/selinux/wiki/Releases). This helps to
>> > ensure that the behaviour of the script is still fine.
>> >
>> > Suggested-by: Petr Lautrbach <plautrba@redhat.com>
>> > Signed-off-by: Nicolas Iooss <nicolas.iooss@m4x.org>
>> > ---
>> >  scripts/release | 95 +++++++++++++++++++++++++++++++------------------
>> >  1 file changed, 60 insertions(+), 35 deletions(-)
>> >
>> > diff --git a/scripts/release b/scripts/release
>> > index 895a0e1ca1a1..21e30ff54b80 100755
>> > --- a/scripts/release
>> > +++ b/scripts/release
>> > @@ -1,43 +1,57 @@
>> >  #!/bin/bash
>> >
>> > -PWD=`pwd`
>> > -WIKIDIR=../selinux.wiki
>> > -
>> > -if [ \! -d $WIKIDIR ]; then
>> > -    git clone git@github.com:SELinuxProject/selinux.wiki.git $WIKIDIR
>> > -fi
>> > +# Fail when a command fails
>> > +set -e
>> >
>> > -RELEASE_TAG=`cat VERSION`
>> > -DEST=releases/$RELEASE_TAG
>> > -DIRS="libsepol libselinux libsemanage checkpolicy secilc policycoreutils mcstrans restorecond semodule-utils"
>> > -DIRS_NEED_PREFIX="dbus gui python sandbox"
>> > +# Ensure the script is running from the top level directory
>> > +cd "$(dirname -- "$0")/.."
>> >
>> > -git tag -a $RELEASE_TAG -m "Release $RELEASE_TAG"
>> > +WIKIDIR=../selinux.wiki
>> >
>> > -rm -rf $DEST
>> > -mkdir -p $DEST
>> > +if ! [ -d "$WIKIDIR" ]; then
>> > +     git clone git@github.com:SELinuxProject/selinux.wiki.git "$WIKIDIR"
>> > +fi
>> >
>> > -for i in $DIRS; do
>> > -     cd $i
>> > -     VERS=`cat VERSION`
>> > -     ARCHIVE=$i-$VERS.tar.gz
>> > -     git tag $i-$VERS > /dev/null 2>&1
>> > -     git archive -o ../$DEST/$ARCHIVE --prefix=$i-$VERS/ $i-$VERS
>> > -     cd ..
>> > -done
>> > +RELEASE_TAG="$(cat VERSION)"
>> > +DEST="releases/$RELEASE_TAG"
>> > +DIRS=(
>> > +     checkpolicy
>> > +     libselinux
>> > +     libsemanage
>> > +     libsepol
>> > +     mcstrans
>> > +     policycoreutils
>> > +     restorecond
>> > +     secilc
>> > +     selinux-dbus
>> > +     selinux-gui
>> > +     selinux-python
>> > +     selinux-sandbox
>> > +     semodule-utils
>> > +)
>> > +
>> > +if git rev-parse "$RELEASE_TAG" > /dev/null ; then
>> > +     echo "Warning: tag $RELEASE_TAG already exists"
>> > +else
>> > +     git tag -a "$RELEASE_TAG" -m "Release $RELEASE_TAG"
>> > +fi
>>
>> fatal: ambiguous argument '3.2-rc2': unknown revision or path not in the working tree.
>> Use '--' to separate paths from revisions, like this:
>> 'git <command> [<revision>...] -- [<file>...]'
>>
>>
>> > -for i in $DIRS_NEED_PREFIX; do
>> > -     cd $i
>> > -     VERS=`cat VERSION`
>> > -     ARCHIVE=selinux-$i-$VERS.tar.gz
>> > -     git tag selinux-$i-$VERS > /dev/null 2>&1
>> > -     git archive -o ../$DEST/$ARCHIVE --prefix=selinux-$i-$VERS/ selinux-$i-$VERS
>> > -     cd ..
>> > +rm -rf "$DEST"
>> > +mkdir -p "$DEST"
>> > +
>> > +for COMPONENT in "${DIRS[@]}"; do
>> > +     DIR="${COMPONENT#selinux-}"
>> > +     VERS="$(cat "$DIR/VERSION")"
>> > +     TAG="$COMPONENT-$VERS"
>> > +     if git rev-parse "$TAG" > /dev/null ; then
>> > +             echo "Warning: tag $TAG already exists"
>> > +     else
>> > +             git tag "$TAG" > /dev/null
>> > +     fi
>>
>> fatal: ambiguous argument 'checkpolicy-3.2-rc2': unknown revision or path not in the working tree.
>> Use '--' to separate paths from revisions, like this:
>> 'git <command> [<revision>...] -- [<file>...]'
>>
>> The following change fixes both:
>>
>> --- a/scripts/release
>> +++ b/scripts/release
>> @@ -30,7 +30,7 @@ DIRS=(
>>         semodule-utils
>>  )
>>
>> -if git rev-parse "$RELEASE_TAG" > /dev/null ; then
>> +if git rev-parse "$RELEASE_TAG" &> /dev/null ; then
>>         echo "Warning: tag $RELEASE_TAG already exists"
>>  else
>>         git tag -a "$RELEASE_TAG" -m "Release $RELEASE_TAG"
>> @@ -43,7 +43,7 @@ for COMPONENT in "${DIRS[@]}"; do
>>         DIR="${COMPONENT#selinux-}"
>>         VERS="$(cat "$DIR/VERSION")"
>>         TAG="$COMPONENT-$VERS"
>> -       if git rev-parse "$TAG" > /dev/null ; then
>> +       if git rev-parse "$TAG" &> /dev/null ; then
>>                 echo "Warning: tag $TAG already exists"
>>         else
>>                 git tag "$TAG" > /dev/null
>
> Oops, indeed. The errors do not seem to be fatal, but yes, it is
> better to hide them, like you suggest. You can modify my patch with
> this change, or write a patch.
>
> Anyway, your suggestion looks good to me.


I'm just preparing 3.2-rc2 so I'm going to modify your patch, merge and
use it for the new release. 


>
> Thanks!
> Nicolas