[PATCH] tests/decode: Suppress "error: " string for expected-failure tests

[PATCH] tests/decode: Suppress "error: " string for expected-failure tests

[Peter Maydell]

The "expected failure" tests for decodetree result in the
error messages from decodetree ending up in logs and in
V=1 output:

>>> MALLOC_PERTURB_=226 /mnt/nvmedisk/linaro/qemu-from-laptop/qemu/build/x86/pyvenv/bin/python3 /mnt/nvmedisk/linaro/qemu-from-laptop/qemu/scripts/decodetree.py --output-null --test-for-error /mnt/nvmedisk/linaro/qemu-from-laptop/qemu/build/x86/../../tests/decode/err_argset1.decode
――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――― ✀  ――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――
/mnt/nvmedisk/linaro/qemu-from-laptop/qemu/build/x86/../../tests/decode/err_argset1.decode:5: error: duplicate argument "a"
―――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――
 1/44 qemu:decodetree / err_argset1                OK              0.05s

This then produces false positives when scanning the
logfiles for strings like "error: ".

For the expected-failure tests, make decodetree print
"detected:" instead of "error:".

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
---
My initial thought was "just get meson to suppress stderr for these
tests", but AFAICT meson has no way to do that, and we already have a
command line option to decodetree to request specific behaviour for
the tests that expect failure, so this seemed simplest.

 scripts/decodetree.py | 6 +++++-
 1 file changed, 5 insertions(+), 1 deletion(-)

diff --git a/scripts/decodetree.py b/scripts/decodetree.py
index a8a6cb69cda..e8b72da3a97 100644
--- a/scripts/decodetree.py
+++ b/scripts/decodetree.py
@@ -134,6 +134,10 @@ def error_with_file(file, lineno, *args):
     global output_file
     global output_fd
 
+    # For the test suite expected-errors case, don't print the
+    # string "error: ", so they don't turn up as false positives
+    # if you grep the meson logs for strings like that.
+    end = 'error: ' if not testforerror else 'detected: '
     prefix = ''
     if file:
         prefix += f'{file}:'
@@ -141,7 +145,7 @@ def error_with_file(file, lineno, *args):
         prefix += f'{lineno}:'
     if prefix:
         prefix += ' '
-    print(prefix, end='error: ', file=sys.stderr)
+    print(prefix, end=end, file=sys.stderr)
     print(*args, file=sys.stderr)
 
     if output_file and output_fd:
-- 
2.34.1

Re: [PATCH] tests/decode: Suppress "error: " string for expected-failure tests

[Philippe Mathieu-Daudé]

On 20/7/23 15:15, Peter Maydell wrote:
> The "expected failure" tests for decodetree result in the
> error messages from decodetree ending up in logs and in
> V=1 output:
> 
>>>> MALLOC_PERTURB_=226 /mnt/nvmedisk/linaro/qemu-from-laptop/qemu/build/x86/pyvenv/bin/python3 /mnt/nvmedisk/linaro/qemu-from-laptop/qemu/scripts/decodetree.py --output-null --test-for-error /mnt/nvmedisk/linaro/qemu-from-laptop/qemu/build/x86/../../tests/decode/err_argset1.decode
> ――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――― ✀  ――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――
> /mnt/nvmedisk/linaro/qemu-from-laptop/qemu/build/x86/../../tests/decode/err_argset1.decode:5: error: duplicate argument "a"
> ―――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――
>   1/44 qemu:decodetree / err_argset1                OK              0.05s
> 
> This then produces false positives when scanning the
> logfiles for strings like "error: ".
> 
> For the expected-failure tests, make decodetree print
> "detected:" instead of "error:".
> 
> Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
> ---
> My initial thought was "just get meson to suppress stderr for these
> tests", but AFAICT meson has no way to do that, and we already have a
> command line option to decodetree to request specific behaviour for
> the tests that expect failure, so this seemed simplest.
> 
>   scripts/decodetree.py | 6 +++++-
>   1 file changed, 5 insertions(+), 1 deletion(-)

Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org>

Re: [PATCH] tests/decode: Suppress "error: " string for expected-failure tests

[Richard Henderson]

On 7/20/23 14:15, Peter Maydell wrote:
> The "expected failure" tests for decodetree result in the
> error messages from decodetree ending up in logs and in
> V=1 output:
> 
>>>> MALLOC_PERTURB_=226 /mnt/nvmedisk/linaro/qemu-from-laptop/qemu/build/x86/pyvenv/bin/python3 /mnt/nvmedisk/linaro/qemu-from-laptop/qemu/scripts/decodetree.py --output-null --test-for-error /mnt/nvmedisk/linaro/qemu-from-laptop/qemu/build/x86/../../tests/decode/err_argset1.decode
> ――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――― ✀  ――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――
> /mnt/nvmedisk/linaro/qemu-from-laptop/qemu/build/x86/../../tests/decode/err_argset1.decode:5: error: duplicate argument "a"
> ―――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――
>   1/44 qemu:decodetree / err_argset1                OK              0.05s
> 
> This then produces false positives when scanning the
> logfiles for strings like "error: ".
> 
> For the expected-failure tests, make decodetree print
> "detected:" instead of "error:".
> 
> Signed-off-by: Peter Maydell<peter.maydell@linaro.org>
> ---
> My initial thought was "just get meson to suppress stderr for these
> tests", but AFAICT meson has no way to do that, and we already have a
> command line option to decodetree to request specific behaviour for
> the tests that expect failure, so this seemed simplest.

Reviewed-by: Richard Henderson <richard.henderson@linaro.org>

r~

Re: [PATCH] tests/decode: Suppress "error: " string for expected-failure tests

[Peter Maydell]

On Thu, 20 Jul 2023 at 14:15, Peter Maydell <peter.maydell@linaro.org> wrote:
>
> The "expected failure" tests for decodetree result in the
> error messages from decodetree ending up in logs and in
> V=1 output:
>

I'll put this in my arm pullreq since it makes pullreq
handling easier and I see you've just sent a tcg pullreq
(unless you'd rather take it yourself).

thanks
-- PMM