[PATCH v4 0/4] Fix broken CI on newer github-actions runner image

[Jiang Xin <worldhello.net@gmail.com>]

From: Jiang Xin <zhiyou.jx@alibaba-inc.com>

GitHub CI runner image "ubuntu-latest" will upgrade to "ubuntu-22.04"
soon. Our CI will break on the new runner image because there is not
"gcc-8" package in ubuntu-22.04. See log of CI#10:

 * https://github.com/jiangxin/git-ci-test/actions/runs/3546701459/jobs/5956020103#step:3:106

Another issue in CI#10 is "pd4 -V” failed without an error output. See:

 * https://github.com/jiangxin/git-ci-test/actions/runs/3546701459/jobs/5956019973#step:3:315

This first issue is fixed in patch 1/4 by downgrading the version of
the runner image for jobs which need "gcc-8". The second issue is
fixed in patch 2/4 by removing the pipe from "p4d -V".  Then, we can
see there was a sigfault when running "p4d -V". See log of CI#11:

 * https://github.com/jiangxin/git-ci-test/actions/runs/3546723251/jobs/5956063536#step:3:311

This issue is fixed in patch 3/4 by upgrading version of p4 on Linux
to the same version as on macOS. But all p4 related test cases failed
becasue python was missing on ubuntu-22.04.  See the log of CI#12:

 * https://github.com/jiangxin/git-ci-test/actions/runs/3546734918/jobs/5956087614#step:4:22733

This issue is fixed in patch 4/4 by installing python2/python3 on
Linux.  All test cases passed. See log of CI#13:

 * https://github.com/jiangxin/git-ci-test/actions/runs/3546744257

# range-diff v3..v4

1:  1c0d639487 ! 1:  28588dad56 github-actions: run gcc-8 on ubuntu-20.04 image
    @@ Commit message
         "ubuntu-20.04" to version "ubuntu-22.04". It will fail to find and
         install "gcc-8" package on the new runner image.
     
    -    Change the runner images from "ubuntu-latest" to "ubuntu-20.04" in order
    -    to run with "gcc-8" as a dependency.
    +    Change some of the runner images from "ubuntu-latest" to "ubuntu-20.04"
    +    in order to install "gcc-8" as a dependency.
     
    -    Instead of use the environment "$runs_on_pool" as below:
    +    The first revision of this patch tried to replace "$runs_on_pool" in
    +    "ci/*.sh" with a new "$runs_on_os" environment variable based on the
    +    "os" filed in the matrix strategy. But these "os" fields in matrix
    +    strategies are obsolete legacies from commit [1] and commit [2], and
    +    are no longer useful. So remove these unused "os" fields.
     
    -        case "$runs_on_pool" in
    -        ubuntu-20.04 | ubuntu-latest)
    -            ;;
    -
    -    we can reuse the os field in the matrix, and use a new environment
    -    "$runs_on_os" as below:
    -
    -        case "$runs_on_os" in
    -        ubuntu)
    -            ;;
    -
    -    In this way, we can change the "ubuntu-latest" runner image to any
    -    version such as "ubuntu-22.04" to test CI behavior in advance.
    +    [1]: c08bb26010 (CI: rename the "Linux32" job to lower-case "linux32",
    +                     2021-11-23)
    +    [2]: 25715419bf (CI: don't run "make test" twice in one job, 2021-11-23)
     
         Reviewed-by: Johannes Schindelin <johannes.schindelin@gmx.de>
    +    Helped-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
         Signed-off-by: Jiang Xin <zhiyou.jx@alibaba-inc.com>
     
      ## .github/workflows/main.yml ##
     @@ .github/workflows/main.yml: jobs:
    -         vector:
    -           - jobname: linux-clang
    -             cc: clang
    -+            os: ubuntu
                  pool: ubuntu-latest
                - jobname: linux-sha256
                  cc: clang
    -@@ .github/workflows/main.yml: jobs:
    +-            os: ubuntu
    +             pool: ubuntu-latest
                - jobname: linux-gcc
                  cc: gcc
                  cc_package: gcc-8
     -            pool: ubuntu-latest
    -+            os: ubuntu
     +            pool: ubuntu-20.04
                - jobname: linux-TEST-vars
                  cc: gcc
     -            os: ubuntu
                  cc_package: gcc-8
     -            pool: ubuntu-latest
    -+            os: ubuntu
     +            pool: ubuntu-20.04
                - jobname: osx-clang
                  cc: clang
    -+            os: macos
                  pool: macos-latest
    -           - jobname: osx-gcc
    -             cc: gcc
    -             cc_package: gcc-9
    -+            os: macos
    -             pool: macos-latest
    -           - jobname: linux-gcc-default
    -             cc: gcc
    -+            os: ubuntu
    -             pool: ubuntu-latest
    -           - jobname: linux-leaks
    -             cc: gcc
    -+            os: ubuntu
    -             pool: ubuntu-latest
    -           - jobname: linux-asan
    -             cc: gcc
    -+            os: ubuntu
    -             pool: ubuntu-latest
    -           - jobname: linux-ubsan
    -             cc: gcc
    -+            os: ubuntu
    -             pool: ubuntu-latest
    -     env:
    -       CC: ${{matrix.vector.cc}}
    -       CC_PACKAGE: ${{matrix.vector.cc_package}}
    -       jobname: ${{matrix.vector.jobname}}
    --      runs_on_pool: ${{matrix.vector.pool}}
    -+      runs_on_os: ${{matrix.vector.os}}
    -     runs-on: ${{matrix.vector.pool}}
    -     steps:
    -     - uses: actions/checkout@v2
    +@@ .github/workflows/main.yml: jobs:
    +         - jobname: linux-musl
    +           image: alpine
    +         - jobname: linux32
    +-          os: ubuntu32
    +           image: daald/ubuntu32:xenial
    +         - jobname: pedantic
    +           image: fedora
     
      ## ci/install-dependencies.sh ##
     @@ ci/install-dependencies.sh: UBUNTU_COMMON_PKGS="make libssl-dev libcurl4-openssl-dev libexpat-dev
    -  tcl tk gettext zlib1g-dev perl-modules liberror-perl libauthen-sasl-perl
       libemail-valid-perl libio-socket-ssl-perl libnet-smtp-ssl-perl"
      
    --case "$runs_on_pool" in
    + case "$runs_on_pool" in
     -ubuntu-latest)
    -+case "$runs_on_os" in
    -+ubuntu)
    ++ubuntu-*)
      	sudo apt-get -q update
      	sudo apt-get -q -y install language-pack-is libsvn-perl apache2 \
      		$UBUNTU_COMMON_PKGS $CC_PACKAGE
    -@@ ci/install-dependencies.sh: ubuntu-latest)
    - 		cp git-lfs-$LINUX_GIT_LFS_VERSION/git-lfs .
    - 	popd
    - 	;;
    --macos-latest)
    -+macos)
    - 	export HOMEBREW_NO_AUTO_UPDATE=1 HOMEBREW_NO_INSTALL_CLEANUP=1
    - 	# Uncomment this if you want to run perf tests:
    - 	# brew install gnu-time
     
      ## ci/lib.sh ##
    -@@ ci/lib.sh: export DEFAULT_TEST_TARGET=prove
    - export GIT_TEST_CLONE_2GB=true
    +@@ ci/lib.sh: export GIT_TEST_CLONE_2GB=true
      export SKIP_DASHED_BUILT_INS=YesPlease
      
    --case "$runs_on_pool" in
    + case "$runs_on_pool" in
     -ubuntu-latest)
    -+case "$runs_on_os" in
    -+ubuntu)
    ++ubuntu-*)
      	if test "$jobname" = "linux-gcc-default"
      	then
      		break
    -@@ ci/lib.sh: ubuntu-latest)
    - 	GIT_LFS_PATH="$HOME/custom/git-lfs"
    - 	export PATH="$GIT_LFS_PATH:$P4_PATH:$PATH"
    - 	;;
    --macos-latest)
    -+macos)
    - 	if [ "$jobname" = osx-gcc ]
    - 	then
    - 		MAKEFLAGS="$MAKEFLAGS PYTHON_PATH=$(which python3)"
2:  fed880b8bd ! 2:  b914f1d2fc ci: show error message of "p4 -V"
    @@ Metadata
     Author: Jiang Xin <zhiyou.jx@alibaba-inc.com>
     
      ## Commit message ##
    -    ci: show error message of "p4 -V"
    +    ci: remove the pipe after "p4 -V" to cache errors
     
    -    When installing p4 as a dependency, we used to pipe output of "p4 -V" to
    -    validate the installation, but this would hide potential errors of p4.
    -    E.g.: A broken p4 installation fails to run.
    +    When installing p4 as a dependency, we used to pipe output of "p4 -V"
    +    and "p4d -V" to validate the installation and output a condensed version
    +    information. But this would hide potential errors of p4 and would stop
    +    with an empty output. E.g.: p4d version 16.2 running on ubuntu 22.04
    +    causes sigfaults.
     
    -    Add some instructions to show errors of command "p4 -V", so we can see
    -    why the command output doesn't match.
    +    By removing the pipe after "p4 -V" and "p4d -V", we may get a verbose
    +    output, and stop immediately on errors becuase we have "set -e" in
    +    "ci/lib.sh". Since we won't look at these trace logs unless something
    +    fails, so just including the raw output seems most sensible.
     
         Reviewed-by: Johannes Schindelin <johannes.schindelin@gmx.de>
    +    Helped-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
         Signed-off-by: Jiang Xin <zhiyou.jx@alibaba-inc.com>
     
      ## ci/install-dependencies.sh ##
    @@ ci/install-dependencies.sh: esac
      then
      	echo "$(tput setaf 6)Perforce Server Version$(tput sgr0)"
     -	p4d -V | grep Rev.
    -+	p4d -V | grep Rev. || { echo >&2 "p4d: bad version"; p4d -V; exit 1; }
    ++	p4d -V
      	echo "$(tput setaf 6)Perforce Client Version$(tput sgr0)"
     -	p4 -V | grep Rev.
    -+	p4 -V | grep Rev. || { echo >&2 "p4: bad version"; p4 -V; exit 1; }
    ++	p4 -V
      else
      	echo >&2 "WARNING: perforce wasn't installed, see above for clues why"
      fi
3:  da2f516fc9 = 3:  4aa2b30c36 ci: p4 on Linux has the same version as on macOS
4:  bd1850cc93 = 4:  35909fd459 ci: install python on ubuntu
      
--

Jiang Xin (4):
  github-actions: run gcc-8 on ubuntu-20.04 image
  ci: remove the pipe after "p4 -V" to cache errors
  ci: p4 on Linux has the same version as on macOS
  ci: install python on ubuntu

 .github/workflows/main.yml |  7 ++-----
 ci/install-dependencies.sh | 12 ++++++------
 ci/lib.sh                  | 11 +++++------
 3 files changed, 13 insertions(+), 17 deletions(-)

-- 
2.39.0.rc0