From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from list by lists.gnu.org with archive (Exim 4.90_1) id 1maSK8-0006Bi-9w for mharc-grub-devel@gnu.org; Tue, 12 Oct 2021 20:40:48 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:36562) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1maSJh-0005r9-Kx for grub-devel@gnu.org; Tue, 12 Oct 2021 20:40:23 -0400 Received: from mail-qt1-x82a.google.com ([2607:f8b0:4864:20::82a]:46700) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1maSJf-0000MD-F5 for grub-devel@gnu.org; Tue, 12 Oct 2021 20:40:21 -0400 Received: by mail-qt1-x82a.google.com with SMTP id y11so1050167qtn.13 for ; Tue, 12 Oct 2021 17:40:18 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=efficientek-com.20210112.gappssmtp.com; s=20210112; h=from:to:cc:subject:date:message-id:mime-version :content-transfer-encoding; bh=erktCoJhG4MErtR6Oud3J62EJQStV59qRGJ2DssbHJs=; b=Smy1zlzCIa51usv6v9GCow4yQuSG4HowtSYugf50MZeN04l0dPT8qv9+PnkkGhEiki 7N+S2I3IeJy3DFmcbeqHkQBR/+9+4CAztTiiGdxE+zqX7BCpbqGcQ8gM84NpuyT2SaQf VtmbR+WtVyiamloGHpFMAQlWJBxUKRGrMWTAkoMonMe+hUEO+2CFOYWa+qyr5r4MkStn LTFDNXB/vtQNgV1fFl2PMSdmyuMzyy3nGCNqZhZsfl9nwj+2/P9yVWuXtJVVfZkNR39v XxMxQ9OFcw2kr0hCsJrKBhkoQDfZ8LIGAKT+12mwO4Hr/1kZX8YbCvzYJR9fTNylvHAo Cu+g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:mime-version :content-transfer-encoding; bh=erktCoJhG4MErtR6Oud3J62EJQStV59qRGJ2DssbHJs=; b=WR35LkGQ8jaPieomBCEErjawQbrIAUbekSeq4aJxIOXte0ynIqNJlCfz1k2LVrEzkZ XSLwnu7pLi/oztyzVG4ilD+D2vGxber2jodQNQAu/Y6waaYTBIHB1PCDFz8v2UULUohE VQlJX2F0NwHU3OQ7l/A87yPRD0hReUYbfgn4EQTEFDB65dxtr7unIY5C0p8z0AoQvTeW Y/dZHSNHrBH6k2v6wggZdnPBzKBlj7qFjfOL6vFFO0WRYgxbivWN6oNIpsmpQGtYqO2Q MUd/jwgt7nK+9mIS9Z57KFCrHDelaFYP+SzBdaaNlxJ/TwAKfUo8akCXRzBH8PFg/9j4 u3KQ== X-Gm-Message-State: AOAM5336N+outP/73ncg3GhzKqxLYMVetpXOo7c5inOmJ8VGPxl41jSz RBVQW58ZUA4HqgawzbhCsUEE2htUhzghZQ== X-Google-Smtp-Source: ABdhPJykbGck2eR2d3JnbYtzviGjAfr8JcUx09RpncO2jZjJ/W4WkPm4fHXobGlZjtg/NkFMWOmn5w== X-Received: by 2002:ac8:58d6:: with SMTP id u22mr22658670qta.7.1634085617448; Tue, 12 Oct 2021 17:40:17 -0700 (PDT) Received: from localhost.localdomain ([37.218.244.251]) by smtp.gmail.com with ESMTPSA id x18sm6682367qkx.94.2021.10.12.17.40.16 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 12 Oct 2021 17:40:17 -0700 (PDT) From: Glenn Washburn To: grub-devel@gnu.org, Daniel Kiper Cc: Glenn Washburn Subject: [PATCH v4 0/9] Various fixes/improvements for tests Date: Tue, 12 Oct 2021 19:39:54 -0500 Message-Id: X-Mailer: git-send-email 2.27.0 MIME-Version: 1.0 Content-Transfer-Encoding: 8bit Received-SPF: pass client-ip=2607:f8b0:4864:20::82a; envelope-from=development@efficientek.com; helo=mail-qt1-x82a.google.com X-Spam_score_int: -18 X-Spam_score: -1.9 X-Spam_bar: - X-Spam_report: (-1.9 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: grub-devel@gnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: The development of GNU GRUB List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 13 Oct 2021 00:40:30 -0000 Updates since v3: * Dropped iso9660_test LANG patch * Merged v3 in v2 * Rewrote commit message for #4 to hopefully be clearer * Replace `...` with $(...) These are mostly test improvement and fixes that stand on their own. Glenn Glenn Washburn (9): tests: Fix partmap_test for arm*-efi, disk numbering has changed tests: When checking squashfs fstime, use superblock last modified time tests: Add set -e to missing tests tests: Do not occlude subshell error codes when used as input to the test command tests: Do not occlude grub-shell return code tests: Make setup errors in grub-fs-tester hard errors tests: A failure of mktemp should cause the test script to exit with code 99 tests: Exit with skipped exit code when test not performed tests: Use @BUILD_SHEBANG@ autoconf var instead of literal shell tests/ahci_test.in | 19 ++++++++++++------- tests/cdboot_test.in | 11 ++++++----- tests/core_compress_test.in | 8 +++++--- tests/ehci_test.in | 19 ++++++++++++------- tests/f2fs_test.in | 2 +- tests/fddboot_test.in | 19 ++++++++++--------- tests/gettext_strings_test.in | 2 +- tests/grub_cmd_date.in | 5 +++-- tests/grub_cmd_set_date.in | 6 +++--- tests/grub_cmd_sleep.in | 2 +- tests/grub_cmd_test.in | 7 ++++--- tests/grub_script_blockarg.in | 3 ++- tests/grub_script_expansion.in | 3 ++- tests/gzcompress_test.in | 3 ++- tests/hddboot_test.in | 9 +++++---- tests/lzocompress_test.in | 3 ++- tests/netboot_test.in | 15 ++++++++------- tests/ohci_test.in | 19 ++++++++++++------- tests/partmap_test.in | 18 +++++++++--------- tests/pata_test.in | 13 +++++++------ tests/pseries_test.in | 2 +- tests/syslinux_test.in | 2 +- tests/test_sha512sum.in | 7 ++++--- tests/uhci_test.in | 19 ++++++++++++------- tests/util/grub-fs-tester.in | 17 ++++++++++++----- tests/xzcompress_test.in | 3 ++- 26 files changed, 139 insertions(+), 97 deletions(-) Range-diff against v3: 1: 3a2cc2738 = 1: 3a2cc2738 tests: Fix partmap_test for arm*-efi, disk numbering has changed 2: 3da044f56 = 2: 3da044f56 tests: When checking squashfs fstime, use superblock last modified time 3: a061a35f5 = 3: a061a35f5 tests: Add set -e to missing tests 4: ab85eb4c8 ! 4: 5828e4c79 tests: Do not occlude subshell error codes @@ Metadata Author: Glenn Washburn ## Commit message ## - tests: Do not occlude subshell error codes + tests: Do not occlude subshell error codes when used as input to the test command - When a subshell's output is used as input to a "simple command", its return - code is not checked. These subshells contain an execution of the grub-shell - script which does the work of the actual test. If grub-shell returns an - error code, the test should fail. So refactor to not have the subshell which - contains grub-shell be direct input into a simple command (usually the test - command). Mostly this is accomplished by having the output first go to a - shell variable, and then using the shell variable in the simple command. + When using the output of a subshell as input, its error code is ignored in + the context of "set -e". Many test scripts use grub-shell in a subshell with + output used as an argument to the test command to test for expected output. + Refactor these tests so that the subshell output goes to a shell variable, + so that if the subshell errors the script will immediately exit with an + error code. ## tests/ahci_test.in ## @@ tests/ahci_test.in: echo "hello" > "$outfile" @@ tests/cdboot_test.in: case "${grub_modinfo_target_cpu}-${grub_modinfo_platform}" esac -if [ "$(echo hello | "${grubshell}" --boot=cd)" != "Hello World" ]; then -+v=`echo hello | "${grubshell}" --boot=cd` ++v=$(echo hello | "${grubshell}" --boot=cd) +if [ "$v" != "Hello World" ]; then exit 1 fi @@ tests/core_compress_test.in: case "${grub_modinfo_target_cpu}-${grub_modinfo_pla -if [ "$(echo hello | "${grubshell}" --grub-mkimage-extra=--compress=xz)" != "Hello World" ]; then -+v=`echo hello | "${grubshell}" --grub-mkimage-extra=--compress=xz` ++v=$(echo hello | "${grubshell}" --grub-mkimage-extra=--compress=xz) +if [ "$v" != "Hello World" ]; then exit 1 fi -if [ "$(echo hello | "${grubshell}" --grub-mkimage-extra=--compress=none)" != "Hello World" ]; then -+v=`echo hello | "${grubshell}" --grub-mkimage-extra=--compress=none` ++v=$(echo hello | "${grubshell}" --grub-mkimage-extra=--compress=none) +if [ "$v" != "Hello World" ]; then exit 1 fi @@ tests/fddboot_test.in: case "${grub_modinfo_target_cpu}-${grub_modinfo_platform} esac -if [ "$(echo hello | "${grubshell}" --boot=fd --mkrescue-arg="--compress=xz --fonts= --locales= --themes= -no-pad")" != "Hello World" ]; then -+v=`echo hello | "${grubshell}" --boot=fd --mkrescue-arg="--compress=xz --fonts= --locales= --themes= -no-pad"` ++v=$(echo hello | "${grubshell}" --boot=fd --mkrescue-arg="--compress=xz --fonts= --locales= --themes= -no-pad") +if [ "$v" != "Hello World" ]; then exit 1 fi @@ tests/gzcompress_test.in: if ! which gzip >/dev/null 2>&1; then fi -if [ "$(echo hello | "${grubshell}" --mkrescue-arg=--compress=gz)" != "Hello World" ]; then -+v=`echo hello | "${grubshell}" --mkrescue-arg=--compress=gz` ++v=$(echo hello | "${grubshell}" --mkrescue-arg=--compress=gz) +if [ "$v" != "Hello World" ]; then exit 1 fi @@ tests/hddboot_test.in: case "${grub_modinfo_target_cpu}-${grub_modinfo_platform} esac -if [ "$(echo hello | "${grubshell}" --boot=hd)" != "Hello World" ]; then -+v=`echo hello | "${grubshell}" --boot=hd` ++v=$(echo hello | "${grubshell}" --boot=hd) +if [ "$v" != "Hello World" ]; then exit 1 fi @@ tests/lzocompress_test.in: if ! which lzop >/dev/null 2>&1; then fi -if [ "$(echo hello | "${grubshell}" --mkrescue-arg=--compress=lzo)" != "Hello World" ]; then -+v=`echo hello | "${grubshell}" --mkrescue-arg=--compress=lzo` ++v=$(echo hello | "${grubshell}" --mkrescue-arg=--compress=lzo) +if [ "$v" != "Hello World" ]; then exit 1 fi @@ tests/netboot_test.in: case "${grub_modinfo_target_cpu}-${grub_modinfo_platform} esac -if [ "$(echo hello | "${grubshell}" --boot=net)" != "Hello World" ]; then -+v=`echo hello | "${grubshell}" --boot=net` ++v=$(echo hello | "${grubshell}" --boot=net) +if [ "$v" != "Hello World" ]; then exit 1 fi @@ tests/pata_test.in: echo "hello" > "$outfile" tar cf "$imgfile" "$outfile" -if [ "$(echo "nativedisk; source '($indisk)/$outfile';" | "${grubshell}" --qemu-opts="-$disk $imgfile")" != "Hello World" ]; then -+v=`echo "nativedisk; source '($indisk)/$outfile';" | "${grubshell}" --qemu-opts="-$disk $imgfile"` ++v=$(echo "nativedisk; source '($indisk)/$outfile';" | "${grubshell}" --qemu-opts="-$disk $imgfile") +if [ "$v" != "Hello World" ]; then rm "$imgfile" rm "$outfile" @@ tests/xzcompress_test.in: if ! which xz >/dev/null 2>&1; then fi -if [ "$(echo hello | "${grubshell}" --mkrescue-arg=--compress=xz)" != "Hello World" ]; then -+v=`echo hello | "${grubshell}" --mkrescue-arg=--compress=xz` ++v=$(echo hello | "${grubshell}" --mkrescue-arg=--compress=xz) +if [ "$v" != "Hello World" ]; then exit 1 fi 5: fac9d9cae = 5: 37097a173 tests: Do not occlude grub-shell return code 6: dae3591c4 = 6: c703a1b2f tests: Make setup errors in grub-fs-tester hard errors 7: 5f2fb19e9 = 7: 23a0d5358 tests: A failure of mktemp should cause the test script to exit with code 99 8: 2a2e491bd ! 8: a4d79a908 tests: Exit with skipped exit code when test not performed @@ tests/cdboot_test.in: grubshell=@builddir@/grub-shell + exit 77;; esac - v=`echo hello | "${grubshell}" --boot=cd` + v=$(echo hello | "${grubshell}" --boot=cd) ## tests/core_compress_test.in ## @@ tests/core_compress_test.in: grubshell=@builddir@/grub-shell @@ tests/fddboot_test.in: grubshell=@builddir@/grub-shell + exit 77;; esac - v=`echo hello | "${grubshell}" --boot=fd --mkrescue-arg="--compress=xz --fonts= --locales= --themes= -no-pad"` + v=$(echo hello | "${grubshell}" --boot=fd --mkrescue-arg="--compress=xz --fonts= --locales= --themes= -no-pad") ## tests/grub_cmd_date.in ## @@ tests/grub_cmd_date.in: set -e @@ tests/hddboot_test.in: grubshell=@builddir@/grub-shell + exit 77;; esac - v=`echo hello | "${grubshell}" --boot=hd` + v=$(echo hello | "${grubshell}" --boot=hd) ## tests/netboot_test.in ## @@ tests/netboot_test.in: grubshell=@builddir@/grub-shell @@ tests/netboot_test.in: grubshell=@builddir@/grub-shell + exit 77;; esac - v=`echo hello | "${grubshell}" --boot=net` + v=$(echo hello | "${grubshell}" --boot=net) ## tests/ohci_test.in ## @@ tests/ohci_test.in: grubshell=@builddir@/grub-shell 9: 606b9b818 = 9: a7119dfc4 tests: Use @BUILD_SHEBANG@ autoconf var instead of literal shell -- 2.27.0