From: "Ævar Arnfjörð Bjarmason" <avarab@gmail.com>
To: git@vger.kernel.org
Cc: "Junio C Hamano" <gitster@pobox.com>,
"SZEDER Gábor" <szeder.dev@gmail.com>,
"Carlo Marcelo Arenas Belón" <carenas@gmail.com>,
"Johannes Schindelin" <Johannes.Schindelin@gmx.de>,
"Victoria Dye" <vdye@github.com>,
"Ævar Arnfjörð Bjarmason" <avarab@gmail.com>
Subject: [PATCH 02/10] CI: remove more dead Travis CI support
Date: Thu, 14 Jul 2022 21:39:39 +0200 [thread overview]
Message-ID: <patch-02.10-39ef9d2ab6e-20220714T193808Z-avarab@gmail.com> (raw)
In-Reply-To: <cover-00.10-00000000000-20220714T193808Z-avarab@gmail.com>
Remove code that I missed in 4a6e4b96026 (CI: remove Travis CI
support, 2021-11-23). This code was only called from or used by the
now-removed .travis.yml, or needed by the Travis CI environment.
For the symlinking in ci/run-build-and-tests.sh: Back when
3c93b829205 (travis-ci: build Git during the 'script' phase,
2018-01-08) and 4b060a4d973 (ci: use a junction on Windows instead of
a symlink, 2019-01-27) adjusted this "ln" command, the Windows build
would use ci/run-build-and-tests.sh.
As seen in 889cacb6897 (ci: configure GitHub Actions for CI/PR,
2020-04-11) the current windows build uses a different entry point
under the GitHub CI, which doesn't use this .prove caching. Namely
"ci/run-test-slice.sh".
We can be certain that it's never used in "ci/run-test-slice.sh"
because to have a ".prove" file we'd need to use "--state=save", which
we only do in the dead Azure codepath in ci/lib.sh. If it were used it
would do the wrong thing, because the different test slices would each
try to clobber the same "t/.prove" file.
If a subsequent run then used the "--state=failed,slow,save" it would
defeat the purpose of "ci/run-test-slice.sh", since all slices would
then run all tests. I.e. behavior of prove's "--state" options is to
select tests to run from the provided "--state" file, in addition to
those specified on the command-line.
For ci/run-docker{,-build}.sh: It was likewise last referenced in the
.travis.yml removed in my 4a6e4b96026. The current "dockerized" run in
".github/workflows/main.yml" calls the same entry points as the main
"regular" job.
For "ci/print-test-failures.sh" the creation of the tarball added in
aea8879a6ac (travis-ci: include the trash directories of failed tests
in the trace log, 2018-08-01) has been dead since my 4a6e4b96026,
which removed the fall-through case from the "case/esac" statement.
Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
---
ci/lib.sh | 10 ++----
ci/print-test-failures.sh | 39 ---------------------
ci/run-build-and-tests.sh | 5 ---
ci/run-docker-build.sh | 66 -----------------------------------
ci/run-docker.sh | 47 -------------------------
ci/run-test-slice.sh | 5 ---
| 50 --------------------------
7 files changed, 2 insertions(+), 220 deletions(-)
delete mode 100755 ci/run-docker-build.sh
delete mode 100755 ci/run-docker.sh
delete mode 100755 ci/util/extract-trash-dirs.sh
diff --git a/ci/lib.sh b/ci/lib.sh
index 4f12bcaabb9..f32c588f6ad 100755
--- a/ci/lib.sh
+++ b/ci/lib.sh
@@ -148,9 +148,6 @@ then
CI_COMMIT="$BUILD_SOURCEVERSION"
CI_JOB_ID="$BUILD_BUILDID"
CI_JOB_NUMBER="$BUILD_BUILDNUMBER"
- CI_OS_NAME="$(echo "$AGENT_OS" | tr A-Z a-z)"
- test darwin != "$CI_OS_NAME" || CI_OS_NAME=osx
- CI_REPO_SLUG="$(expr "$BUILD_REPOSITORY_URI" : '.*/\([^/]*/[^/]*\)$')"
CC="${CC:-gcc}"
# use a subdirectory of the cache dir (because the file share is shared
@@ -160,16 +157,13 @@ then
export GIT_PROVE_OPTS="--timer --jobs 10 --state=failed,slow,save"
export GIT_TEST_OPTS="--verbose-log -x --write-junit-xml"
MAKEFLAGS="$MAKEFLAGS --jobs=10"
- test windows_nt != "$CI_OS_NAME" ||
+ test Windows_NT != "$AGENT_OS" ||
GIT_TEST_OPTS="--no-chain-lint --no-bin-wrappers $GIT_TEST_OPTS"
elif test true = "$GITHUB_ACTIONS"
then
CI_TYPE=github-actions
CI_BRANCH="$GITHUB_REF"
CI_COMMIT="$GITHUB_SHA"
- CI_OS_NAME="$(echo "$RUNNER_OS" | tr A-Z a-z)"
- test macos != "$CI_OS_NAME" || CI_OS_NAME=osx
- CI_REPO_SLUG="$GITHUB_REPOSITORY"
CI_JOB_ID="$GITHUB_RUN_ID"
CC="${CC_PACKAGE:-${CC:-gcc}}"
DONT_SKIP_TAGS=t
@@ -200,7 +194,7 @@ then
export GIT_PROVE_OPTS="--timer --jobs 10"
export GIT_TEST_OPTS="--verbose-log -x --github-workflow-markup"
MAKEFLAGS="$MAKEFLAGS --jobs=10"
- test windows != "$CI_OS_NAME" ||
+ test Windows != "$RUNNER_OS" ||
GIT_TEST_OPTS="--no-chain-lint --no-bin-wrappers $GIT_TEST_OPTS"
else
echo "Could not identify CI type" >&2
diff --git a/ci/print-test-failures.sh b/ci/print-test-failures.sh
index 57277eefcd0..740ebb7c529 100755
--- a/ci/print-test-failures.sh
+++ b/ci/print-test-failures.sh
@@ -16,15 +16,6 @@ then
exit
fi
-case "$jobname" in
-osx-clang|osx-gcc)
- # base64 in OSX doesn't wrap its output at 76 columns by
- # default, but prints a single, very long line.
- base64_opts="-b 76"
- ;;
-esac
-
-combined_trash_size=0
for TEST_EXIT in test-results/*.exit
do
if [ "$(cat "$TEST_EXIT")" != "0" ]
@@ -42,47 +33,17 @@ do
azure-pipelines)
mkdir -p failed-test-artifacts
mv "$trash_dir" failed-test-artifacts
- continue
;;
github-actions)
mkdir -p failed-test-artifacts
echo "FAILED_TEST_ARTIFACTS=t/failed-test-artifacts" >>$GITHUB_ENV
cp "${TEST_EXIT%.exit}.out" failed-test-artifacts/
tar czf failed-test-artifacts/"$test_name".trash.tar.gz "$trash_dir"
- continue
;;
*)
echo "Unhandled CI type: $CI_TYPE" >&2
exit 1
;;
esac
- trash_tgz_b64="trash.$test_name.base64"
- if [ -d "$trash_dir" ]
- then
- tar czp "$trash_dir" |base64 $base64_opts >"$trash_tgz_b64"
-
- trash_size=$(wc -c <"$trash_tgz_b64")
- if [ $trash_size -gt 1048576 ]
- then
- # larger than 1MB
- echo "$(tput setaf 1)Didn't include the trash directory of '$test_name' in the trace log, it's too big$(tput sgr0)"
- continue
- fi
-
- new_combined_trash_size=$(($combined_trash_size + $trash_size))
- if [ $new_combined_trash_size -gt 1048576 ]
- then
- echo "$(tput setaf 1)Didn't include the trash directory of '$test_name' in the trace log, there is plenty of trash in there already.$(tput sgr0)"
- continue
- fi
- combined_trash_size=$new_combined_trash_size
-
- # DO NOT modify these two 'echo'-ed strings below
- # without updating 'ci/util/extract-trash-dirs.sh'
- # as well.
- echo "$(tput setaf 1)Start of trash directory of '$test_name':$(tput sgr0)"
- cat "$trash_tgz_b64"
- echo "$(tput setaf 1)End of trash directory of '$test_name'$(tput sgr0)"
- fi
fi
done
diff --git a/ci/run-build-and-tests.sh b/ci/run-build-and-tests.sh
index 8ebff425967..9a5032e6a77 100755
--- a/ci/run-build-and-tests.sh
+++ b/ci/run-build-and-tests.sh
@@ -5,11 +5,6 @@
. ${0%/*}/lib.sh
-case "$CI_OS_NAME" in
-windows*) cmd //c mklink //j t\\.prove "$(cygpath -aw "$cache_dir/.prove")";;
-*) ln -s "$cache_dir/.prove" t/.prove;;
-esac
-
run_tests=t
case "$jobname" in
diff --git a/ci/run-docker-build.sh b/ci/run-docker-build.sh
deleted file mode 100755
index 6cd832efb9c..00000000000
--- a/ci/run-docker-build.sh
+++ /dev/null
@@ -1,66 +0,0 @@
-#!/bin/sh
-#
-# Build and test Git inside container
-#
-# Usage:
-# run-docker-build.sh <host-user-id>
-#
-
-set -ex
-
-if test $# -ne 1 || test -z "$1"
-then
- echo >&2 "usage: run-docker-build.sh <host-user-id>"
- exit 1
-fi
-
-case "$jobname" in
-linux32)
- switch_cmd="linux32 --32bit i386"
- ;;
-linux-musl)
- switch_cmd=
- useradd () { adduser -D "$@"; }
- ;;
-*)
- exit 1
- ;;
-esac
-
-"${0%/*}/install-docker-dependencies.sh"
-
-# If this script runs inside a docker container, then all commands are
-# usually executed as root. Consequently, the host user might not be
-# able to access the test output files.
-# If a non 0 host user id is given, then create a user "ci" with that
-# user id to make everything accessible to the host user.
-HOST_UID=$1
-if test $HOST_UID -eq 0
-then
- # Just in case someone does want to run the test suite as root.
- CI_USER=root
-else
- CI_USER=ci
- if test "$(id -u $CI_USER 2>/dev/null)" = $HOST_UID
- then
- echo "user '$CI_USER' already exists with the requested ID $HOST_UID"
- else
- useradd -u $HOST_UID $CI_USER
- fi
-fi
-
-# Build and test
-command $switch_cmd su -m -l $CI_USER -c "
- set -ex
- export DEVELOPER='$DEVELOPER'
- export DEFAULT_TEST_TARGET='$DEFAULT_TEST_TARGET'
- export GIT_PROVE_OPTS='$GIT_PROVE_OPTS'
- export GIT_TEST_OPTS='$GIT_TEST_OPTS'
- export GIT_TEST_CLONE_2GB='$GIT_TEST_CLONE_2GB'
- export MAKEFLAGS='$MAKEFLAGS'
- export cache_dir='$cache_dir'
- cd /usr/src/git
- test -n '$cache_dir' && ln -s '$cache_dir/.prove' t/.prove
- make
- make test
-"
diff --git a/ci/run-docker.sh b/ci/run-docker.sh
deleted file mode 100755
index af89d1624a4..00000000000
--- a/ci/run-docker.sh
+++ /dev/null
@@ -1,47 +0,0 @@
-#!/bin/sh
-#
-# Download and run Docker image to build and test Git
-#
-
-. ${0%/*}/lib.sh
-
-case "$jobname" in
-linux32)
- CI_CONTAINER="daald/ubuntu32:xenial"
- ;;
-linux-musl)
- CI_CONTAINER=alpine
- ;;
-*)
- exit 1
- ;;
-esac
-
-docker pull "$CI_CONTAINER"
-
-# Use the following command to debug the docker build locally:
-# <host-user-id> must be 0 if podman is used as drop-in replacement for docker
-# $ docker run -itv "${PWD}:/usr/src/git" --entrypoint /bin/sh "$CI_CONTAINER"
-# root@container:/# export jobname=<jobname>
-# root@container:/# /usr/src/git/ci/run-docker-build.sh <host-user-id>
-
-container_cache_dir=/tmp/container-cache
-
-docker run \
- --interactive \
- --env DEVELOPER \
- --env DEFAULT_TEST_TARGET \
- --env GIT_PROVE_OPTS \
- --env GIT_TEST_OPTS \
- --env GIT_TEST_CLONE_2GB \
- --env MAKEFLAGS \
- --env jobname \
- --env cache_dir="$container_cache_dir" \
- --volume "${PWD}:/usr/src/git" \
- --volume "$cache_dir:$container_cache_dir" \
- "$CI_CONTAINER" \
- /usr/src/git/ci/run-docker-build.sh $(id -u $USER)
-
-check_unignored_build_artifacts
-
-save_good_tree
diff --git a/ci/run-test-slice.sh b/ci/run-test-slice.sh
index a3c67956a8d..312a3ef73fe 100755
--- a/ci/run-test-slice.sh
+++ b/ci/run-test-slice.sh
@@ -5,11 +5,6 @@
. ${0%/*}/lib.sh
-case "$CI_OS_NAME" in
-windows*) cmd //c mklink //j t\\.prove "$(cygpath -aw "$cache_dir/.prove")";;
-*) ln -s "$cache_dir/.prove" t/.prove;;
-esac
-
group "Run tests" make --quiet -C t T="$(cd t &&
./helper/test-tool path-utils slice-tests "$1" "$2" t[0-9]*.sh |
tr '\n' ' ')" ||
diff --git a/ci/util/extract-trash-dirs.sh b/ci/util/extract-trash-dirs.sh
deleted file mode 100755
index 8e67bec21a2..00000000000
--- a/ci/util/extract-trash-dirs.sh
+++ /dev/null
@@ -1,50 +0,0 @@
-#!/bin/sh
-
-error () {
- echo >&2 "error: $@"
- exit 1
-}
-
-find_embedded_trash () {
- while read -r line
- do
- case "$line" in
- *Start\ of\ trash\ directory\ of\ \'t[0-9][0-9][0-9][0-9]-*\':*)
- test_name="${line#*\'}"
- test_name="${test_name%\'*}"
-
- return 0
- esac
- done
-
- return 1
-}
-
-extract_embedded_trash () {
- while read -r line
- do
- case "$line" in
- *End\ of\ trash\ directory\ of\ \'$test_name\'*)
- return
- ;;
- *)
- printf '%s\n' "$line"
- ;;
- esac
- done
-
- error "unexpected end of input"
-}
-
-# Raw logs from Linux build jobs have CRLF line endings, while OSX
-# build jobs mostly have CRCRLF, except an odd line every now and
-# then that has CRCRCRLF. 'base64 -d' from 'coreutils' doesn't like
-# CRs and complains about "invalid input", so remove all CRs at the
-# end of lines.
-sed -e 's/\r*$//' | \
-while find_embedded_trash
-do
- echo "Extracting trash directory of '$test_name'"
-
- extract_embedded_trash |base64 -d |tar xzp
-done
--
2.37.1.996.g651fc6e809f
next prev parent reply other threads:[~2022-07-14 19:40 UTC|newest]
Thread overview: 11+ messages / expand[flat|nested] mbox.gz Atom feed top
2022-07-14 19:39 [PATCH 00/10] ci: make it easy to run locally, part 1 Ævar Arnfjörð Bjarmason
2022-07-14 19:39 ` [PATCH 01/10] CI: run "set -ex" early in ci/lib.sh Ævar Arnfjörð Bjarmason
2022-07-14 19:39 ` Ævar Arnfjörð Bjarmason [this message]
2022-07-14 19:39 ` [PATCH 03/10] CI: remove dead "tree skipping" code Ævar Arnfjörð Bjarmason
2022-07-14 19:39 ` [PATCH 04/10] CI: make "$jobname" explicit, remove fallback Ævar Arnfjörð Bjarmason
2022-07-14 19:39 ` [PATCH 05/10] CI/lib.sh: stop adding leading whitespace to $MAKEFLAGS Ævar Arnfjörð Bjarmason
2022-07-14 19:39 ` [PATCH 06/10] CI: consistently use "export" in ci/lib.sh Ævar Arnfjörð Bjarmason
2022-07-14 19:39 ` [PATCH 07/10] CI: export variables via a wrapper Ævar Arnfjörð Bjarmason
2022-07-14 19:39 ` [PATCH 08/10] CI: don't have "git grep" invoke a pager in tree content check Ævar Arnfjörð Bjarmason
2022-07-14 19:39 ` [PATCH 09/10] CI: have "static-analysis" run a "make ci-static-analysis" target Ævar Arnfjörð Bjarmason
2022-07-14 19:39 ` [PATCH 10/10] CI: have "static-analysis" run "check-builtins", not "documentation" Ævar Arnfjörð Bjarmason
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=patch-02.10-39ef9d2ab6e-20220714T193808Z-avarab@gmail.com \
--to=avarab@gmail.com \
--cc=Johannes.Schindelin@gmx.de \
--cc=carenas@gmail.com \
--cc=git@vger.kernel.org \
--cc=gitster@pobox.com \
--cc=szeder.dev@gmail.com \
--cc=vdye@github.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).