From: "Johannes Schindelin via GitGitGadget" <gitgitgadget@gmail.com>
To: git@vger.kernel.org
Cc: Jeff King <peff@peff.net>, Junio C Hamano <gitster@pobox.com>,
Johannes Schindelin <johannes.schindelin@gmx.de>
Subject: [PATCH v2 0/5] Enable GPG in the Windows part of the CI/PR builds
Date: Wed, 25 Mar 2020 05:41:16 +0000 [thread overview]
Message-ID: <pull.728.v2.git.git.1585114881.gitgitgadget@gmail.com> (raw)
In-Reply-To: <pull.728.git.git.1584968990.gitgitgadget@gmail.com>
While debugging the breakages introduced by hi/gpg-prefer-check-signature, I
noticed that the GPG prereq was not available on Windows, even if Git for
Windows' SDK comes with a fully functional GPG2.
The fix was easy, but finding out what was going on was not, so for good
measure, the fix is accompanied by a patch that will hopefully make future
investigations into GPG-related problems much, much easier.
Changes since v1:
* The prereqs are now lazy ones.
* A new patch was introduced to make tracing via -x work even with those
inter-dependent prereqs.
* The test-signing's stdout is redirected to /dev/null because it is
unreadable and unhelpful binary gibberish, anyway. (This imitates Peff's
patch.)
Johannes Schindelin (5):
tests(gpg): allow the gpg-agent to start on Windows
t/lib-gpg.sh: stop pretending to be a stand-alone script
tests: turn GPG, GPGSM and RFC1991 into lazy prereqs
tests: do not let lazy prereqs inside `test_expect_*` turn off tracing
tests: increase the verbosity of the GPG-related prereqs
t/lib-gpg.sh | 110 ++++++++++++++++++++++++++---------------------
t/t0000-basic.sh | 13 ++++++
t/test-lib.sh | 6 ++-
3 files changed, 77 insertions(+), 52 deletions(-)
base-commit: 30e9940356dc67959877f4b2417da33ebdefbb79
Published-As: https://github.com/gitgitgadget/git/releases/tag/pr-git-728%2Fdscho%2Fci-windows-gpg-v2
Fetch-It-Via: git fetch https://github.com/gitgitgadget/git pr-git-728/dscho/ci-windows-gpg-v2
Pull-Request: https://github.com/git/git/pull/728
Range-diff vs v1:
1: 287a21f1033 = 1: 287a21f1033 tests(gpg): allow the gpg-agent to start on Windows
-: ----------- > 2: c1811d54190 t/lib-gpg.sh: stop pretending to be a stand-alone script
2: dd26cb05a37 ! 3: 85457a7b618 tests(gpg): increase verbosity to allow debugging
@@ -1,21 +1,36 @@
Author: Johannes Schindelin <johannes.schindelin@gmx.de>
- tests(gpg): increase verbosity to allow debugging
+ tests: turn GPG, GPGSM and RFC1991 into lazy prereqs
- Especially when debugging a test failure that can only be reproduced in
- the CI build (e.g. when the developer has no access to a macOS machine
- other than running the tests on a macOS build agent), output should not
- be suppressed.
+ The code to set those prereqs is executed completely outside of any
+ `test_eval_` block. As a consequence, its output had to be suppressed so
+ that it does not clutter the output of a regular test script run.
- In the instance of `hi/gpg-prefer-check-signature`, where one
- GPG-related test failed for no apparent reason, the entire output of
- `gpg` and `gpgsm` was suppressed, even in verbose mode, leaving
- interested readers no clue what was going wrong.
+ Unfortunately, the output *stays* suppressed even when the `--verbose`
+ option is in effect.
- Let's fix this by redirecting the output not to `/dev/null`, but to the
- file descriptors that may, or may not, be redirected via
- `--verbose-log`. For good measure, also turn on tracing if the user
- asked for it, and prefix it with a helpful info message.
+ This hid important output when debugging why the GPG prereq was not
+ enabled in the Windows part of our CI builds.
+
+ In preparation for fixing that, let's move all of this code into lazy
+ prereqs.
+
+ The only slightly tricky part is the global environment variable
+ `GNUPGHOME`. Originally, it was configured only when we verified that
+ there is a `gpg` in the `PATH` that we can use. This is now no longer
+ possible, as lazy prereqs are evaluated in a subshell that changes the
+ working directory to a temporary one. Therefore, we simply _always_ set
+ that environment variable: it does not hurt anything because it does not
+ indicate the presence of a working GPG.
+
+ Side note: it was quite tempting to use a hack that is possible because
+ we do not validate what is passed to `test_lazy_prereq` (and it is
+ therefore possible to "break out" of the lazy_prereq subshell:
+
+ test_lazy_prereq GPG '...) && GNUPGHOME=... && (...'
+
+ However, this is rather tricksy hobbitses code, and the current patch is
+ _much_ easier to understand.
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
@@ -23,67 +38,128 @@
--- a/t/lib-gpg.sh
+++ b/t/lib-gpg.sh
@@
+-gpg_version=$(gpg --version 2>&1)
+-if test $? != 127
+-then
++# We always set GNUPGHOME, even if no usable GPG was found, as
++#
++# - It does not hurt, and
++#
++# - we cannot set global environment variables in lazy prereqs because they are
++# executed in an eval'ed subshell that changes the working directory to a
++# temporary one.
++
++GNUPGHOME="$PWD/gpghome"
++export GNUPGHOME
++
++test_lazy_prereq GPG '
++ gpg_version=$(gpg --version 2>&1)
++ test $? != 127 || exit 1
++
+ # As said here: http://www.gnupg.org/documentation/faqs.html#q6.19
+- # the gpg version 1.0.6 didn't parse trust packets correctly, so for
++ # the gpg version 1.0.6 did not parse trust packets correctly, so for
+ # that version, creation of signed tags using the generated key fails.
+ case "$gpg_version" in
+- 'gpg (GnuPG) 1.0.6'*)
++ "gpg (GnuPG) 1.0.6"*)
say "Your version of gpg (1.0.6) is too buggy for testing"
++ exit 1
;;
*)
-+ say_color info >&4 "Trying to set up GPG"
-+ want_trace && set -x
# Available key info:
- # * Type DSA and Elgamal, size 2048 bits, no expiration date,
- # name and email: C O Mitter <committer@example.com>
@@
- chmod 0700 ./gpghome &&
- GNUPGHOME="$PWD/gpghome" &&
- export GNUPGHOME &&
-- (gpgconf --kill gpg-agent >/dev/null 2>&1 || : ) &&
-- gpg --homedir "${GNUPGHOME}" 2>/dev/null --import \
-- "$TEST_DIRECTORY"/lib-gpg/keyring.gpg &&
-- gpg --homedir "${GNUPGHOME}" 2>/dev/null --import-ownertrust \
-- "$TEST_DIRECTORY"/lib-gpg/ownertrust &&
-- gpg --homedir "${GNUPGHOME}" </dev/null >/dev/null 2>&1 \
+ # To export ownertrust:
+ # gpg --homedir /tmp/gpghome --export-ownertrust \
+ # > lib-gpg/ownertrust
+- mkdir ./gpghome &&
+- chmod 0700 ./gpghome &&
+- GNUPGHOME="$PWD/gpghome" &&
+- export GNUPGHOME &&
++ mkdir "$GNUPGHOME" &&
++ chmod 0700 "$GNUPGHOME" &&
+ (gpgconf --kill gpg-agent >/dev/null 2>&1 || : ) &&
+ gpg --homedir "${GNUPGHOME}" 2>/dev/null --import \
+ "$TEST_DIRECTORY"/lib-gpg/keyring.gpg &&
+ gpg --homedir "${GNUPGHOME}" 2>/dev/null --import-ownertrust \
+ "$TEST_DIRECTORY"/lib-gpg/ownertrust &&
+ gpg --homedir "${GNUPGHOME}" </dev/null >/dev/null 2>&1 \
- --sign -u committer@example.com &&
-+ (gpgconf --kill gpg-agent >&3 2>&4 || : ) &&
-+ gpg --homedir "${GNUPGHOME}" --import \
-+ "$TEST_DIRECTORY"/lib-gpg/keyring.gpg >&3 2>&4 &&
-+ gpg --homedir "${GNUPGHOME}" --import-ownertrust \
-+ "$TEST_DIRECTORY"/lib-gpg/ownertrust >&3 2>&4 &&
-+ gpg --homedir "${GNUPGHOME}" </dev/null \
-+ --sign -u committer@example.com >&3 2>&4 &&
- test_set_prereq GPG &&
- # Available key info:
- # * see t/lib-gpg/gpgsm-gen-key.in
-@@
- # gpgsm --homedir /tmp/gpghome/ \
- # -o t/lib-gpg/gpgsm_cert.p12 \
- # --export-secret-key-p12 "committer@example.com"
+- test_set_prereq GPG &&
+- # Available key info:
+- # * see t/lib-gpg/gpgsm-gen-key.in
+- # To generate new certificate:
+- # * no passphrase
+- # gpgsm --homedir /tmp/gpghome/ \
+- # -o /tmp/gpgsm.crt.user \
+- # --generate-key \
+- # --batch t/lib-gpg/gpgsm-gen-key.in
+- # To import certificate:
+- # gpgsm --homedir /tmp/gpghome/ \
+- # --import /tmp/gpgsm.crt.user
+- # To export into a .p12 we can later import:
+- # gpgsm --homedir /tmp/gpghome/ \
+- # -o t/lib-gpg/gpgsm_cert.p12 \
+- # --export-secret-key-p12 "committer@example.com"
- echo | gpgsm --homedir "${GNUPGHOME}" 2>/dev/null \
-+ echo | gpgsm --homedir "${GNUPGHOME}" >&3 2>&4 \
- --passphrase-fd 0 --pinentry-mode loopback \
- --import "$TEST_DIRECTORY"/lib-gpg/gpgsm_cert.p12 &&
-
+- --passphrase-fd 0 --pinentry-mode loopback \
+- --import "$TEST_DIRECTORY"/lib-gpg/gpgsm_cert.p12 &&
+-
- gpgsm --homedir "${GNUPGHOME}" 2>/dev/null -K |
-+ gpgsm --homedir "${GNUPGHOME}" -K 2>&4 |
- grep fingerprint: |
- cut -d" " -f4 |
- tr -d '\n' >"${GNUPGHOME}/trustlist.txt" &&
-
- echo " S relax" >>"${GNUPGHOME}/trustlist.txt" &&
+- grep fingerprint: |
+- cut -d" " -f4 |
+- tr -d '\n' >"${GNUPGHOME}/trustlist.txt" &&
+-
+- echo " S relax" >>"${GNUPGHOME}/trustlist.txt" &&
- echo hello | gpgsm --homedir "${GNUPGHOME}" >/dev/null \
- -u committer@example.com -o /dev/null --sign - 2>&1 &&
-+ echo hello | gpgsm --homedir "${GNUPGHOME}" >&3 2>&4 \
-+ -u committer@example.com -o /dev/null --sign - &&
- test_set_prereq GPGSM
+- test_set_prereq GPGSM
++ --sign -u committer@example.com
;;
esac
- fi
+-fi
++'
++
++test_lazy_prereq GPGSM '
++ test_have_prereq GPG &&
++ # Available key info:
++ # * see t/lib-gpg/gpgsm-gen-key.in
++ # To generate new certificate:
++ # * no passphrase
++ # gpgsm --homedir /tmp/gpghome/ \
++ # -o /tmp/gpgsm.crt.user \
++ # --generate-key \
++ # --batch t/lib-gpg/gpgsm-gen-key.in
++ # To import certificate:
++ # gpgsm --homedir /tmp/gpghome/ \
++ # --import /tmp/gpgsm.crt.user
++ # To export into a .p12 we can later import:
++ # gpgsm --homedir /tmp/gpghome/ \
++ # -o t/lib-gpg/gpgsm_cert.p12 \
++ # --export-secret-key-p12 "committer@example.com"
++ echo | gpgsm --homedir "${GNUPGHOME}" 2>/dev/null \
++ --passphrase-fd 0 --pinentry-mode loopback \
++ --import "$TEST_DIRECTORY"/lib-gpg/gpgsm_cert.p12 &&
++
++ gpgsm --homedir "${GNUPGHOME}" 2>/dev/null -K |
++ grep fingerprint: |
++ cut -d" " -f4 |
++ tr -d "\\n" >"${GNUPGHOME}/trustlist.txt" &&
++
++ echo " S relax" >>"${GNUPGHOME}/trustlist.txt" &&
++ echo hello | gpgsm --homedir "${GNUPGHOME}" >/dev/null \
++ -u committer@example.com -o /dev/null --sign - 2>&1
++'
- if test_have_prereq GPG &&
+-if test_have_prereq GPG &&
- echo | gpg --homedir "${GNUPGHOME}" -b --rfc1991 >/dev/null 2>&1
-+ echo | gpg --homedir "${GNUPGHOME}" -b --rfc1991 >&3 2>&4
- then
- test_set_prereq RFC1991
- fi
-+want_trace && set +x
+-then
+- test_set_prereq RFC1991
+-fi
++test_lazy_prereq RFC1991 '
++ test_have_prereq GPG &&
++ echo | gpg --homedir "${GNUPGHOME}" -b --rfc1991 >/dev/null 2>&1
++'
sanitize_pgp() {
perl -ne '
-: ----------- > 4: 0767c8b77c8 tests: do not let lazy prereqs inside `test_expect_*` turn off tracing
-: ----------- > 5: 5e89b512513 tests: increase the verbosity of the GPG-related prereqs
--
gitgitgadget
next prev parent reply other threads:[~2020-03-25 5:41 UTC|newest]
Thread overview: 55+ messages / expand[flat|nested] mbox.gz Atom feed top
2020-03-23 13:09 [PATCH 0/2] Enable GPG in the Windows part of the CI/PR builds Johannes Schindelin via GitGitGadget
2020-03-23 13:09 ` [PATCH 1/2] tests(gpg): allow the gpg-agent to start on Windows Johannes Schindelin via GitGitGadget
2020-03-23 17:46 ` Junio C Hamano
2020-03-24 19:55 ` Johannes Schindelin
2020-03-24 20:59 ` Junio C Hamano
2020-03-24 22:26 ` Johannes Schindelin
2020-03-24 23:40 ` Junio C Hamano
2020-03-23 13:09 ` [PATCH 2/2] tests(gpg): increase verbosity to allow debugging Johannes Schindelin via GitGitGadget
2020-03-23 17:32 ` Jeff King
2020-03-23 18:04 ` Jeff King
2020-03-23 19:21 ` Junio C Hamano
2020-03-23 20:15 ` Jeff King
2020-03-23 21:28 ` Junio C Hamano
2020-03-23 21:31 ` Jeff King
2020-03-24 21:41 ` Johannes Schindelin
2020-03-24 22:05 ` Jeff King
2020-03-24 22:25 ` Johannes Schindelin
2020-03-24 22:33 ` Jeff King
2020-03-25 5:41 ` Johannes Schindelin via GitGitGadget [this message]
2020-03-25 5:41 ` [PATCH v2 1/5] tests(gpg): allow the gpg-agent to start on Windows Johannes Schindelin via GitGitGadget
2020-03-25 5:41 ` [PATCH v2 2/5] t/lib-gpg.sh: stop pretending to be a stand-alone script Johannes Schindelin via GitGitGadget
2020-03-26 8:21 ` Jeff King
2020-03-26 13:48 ` Johannes Schindelin
2020-03-26 19:31 ` Junio C Hamano
2020-03-25 5:41 ` [PATCH v2 3/5] tests: turn GPG, GPGSM and RFC1991 into lazy prereqs Johannes Schindelin via GitGitGadget
2020-03-25 17:25 ` Junio C Hamano
2020-03-26 8:35 ` Jeff King
2020-03-26 14:27 ` Johannes Schindelin
2020-03-27 9:10 ` Jeff King
2020-03-27 17:44 ` Junio C Hamano
2020-03-27 20:24 ` Eric Sunshine
2020-03-27 21:37 ` Junio C Hamano
2020-03-28 10:58 ` Jeff King
2020-03-28 10:54 ` Jeff King
2020-03-28 23:49 ` [PATCH v2] t/README: suggest how to leave test early with failure Junio C Hamano
2020-03-29 7:23 ` Eric Sunshine
2020-03-29 14:33 ` Jeff King
2020-03-30 18:39 ` [PATCH v2 3/5] tests: turn GPG, GPGSM and RFC1991 into lazy prereqs Johannes Schindelin
2020-03-31 9:34 ` Jeff King
2020-03-25 5:41 ` [PATCH v2 4/5] tests: do not let lazy prereqs inside `test_expect_*` turn off tracing Johannes Schindelin via GitGitGadget
2020-03-25 17:23 ` Junio C Hamano
2020-03-26 13:45 ` Johannes Schindelin
2020-03-26 8:49 ` Jeff King
2020-03-26 14:34 ` Johannes Schindelin
2020-03-25 5:41 ` [PATCH v2 5/5] tests: increase the verbosity of the GPG-related prereqs Johannes Schindelin via GitGitGadget
2020-03-26 8:50 ` Jeff King
2020-03-26 14:36 ` Johannes Schindelin
2020-03-26 15:35 ` [PATCH v3 0/5] Enable GPG in the Windows part of the CI/PR builds Johannes Schindelin via GitGitGadget
2020-03-26 15:35 ` [PATCH v3 1/5] tests(gpg): allow the gpg-agent to start on Windows Johannes Schindelin via GitGitGadget
2020-03-26 15:35 ` [PATCH v3 2/5] t/lib-gpg.sh: stop pretending to be a stand-alone script Johannes Schindelin via GitGitGadget
2020-03-26 15:35 ` [PATCH v3 3/5] tests: do not let lazy prereqs inside `test_expect_*` turn off tracing Johannes Schindelin via GitGitGadget
2020-03-26 15:35 ` [PATCH v3 4/5] tests: turn GPG, GPGSM and RFC1991 into lazy prereqs Johannes Schindelin via GitGitGadget
2020-03-26 15:35 ` [PATCH v3 5/5] tests: increase the verbosity of the GPG-related prereqs Johannes Schindelin via GitGitGadget
2020-03-27 9:12 ` [PATCH v3 0/5] Enable GPG in the Windows part of the CI/PR builds Jeff King
2020-03-27 17:45 ` Junio C Hamano
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=pull.728.v2.git.git.1585114881.gitgitgadget@gmail.com \
--to=gitgitgadget@gmail.com \
--cc=git@vger.kernel.org \
--cc=gitster@pobox.com \
--cc=johannes.schindelin@gmx.de \
--cc=peff@peff.net \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.