[kvm-unit-tests PATCH v2 0/2] Use same test names in the default and the TAP13 output format

[kvm-unit-tests PATCH v2 0/2] Use same test names in the default and the TAP13 output format

[Marc Hartmayer]

For everybody's convenience there is a branch:
https://gitlab.com/mhartmay/kvm-unit-tests/-/tree/tap_v2

Changelog:
v1 -> v2:
 + added r-b's to patch 1
 + patch 2:
  - I've not added Andrew's r-b since I've worked in the comment from
    Janosch (don't drop the first prefix)

Marc Hartmayer (2):
  runtime.bash: remove outdated comment
  Use same test names in the default and the TAP13 output format

 run_tests.sh         | 15 +++++++++------
 scripts/runtime.bash |  9 +++------
 2 files changed, 12 insertions(+), 12 deletions(-)

-- 
2.25.4

[kvm-unit-tests PATCH v2 1/2] runtime.bash: remove outdated comment

[Marc Hartmayer]

Since commit 6e1d3752d7ca ("tap13: list testcases individually") the
comment is no longer valid. Therefore let's remove it.

Reviewed-by: Andrew Jones <drjones@redhat.com>
Reviewed-by: Cornelia Huck <cohuck@redhat.com>
Signed-off-by: Marc Hartmayer <mhartmay@linux.ibm.com>
---
 scripts/runtime.bash | 3 ---
 1 file changed, 3 deletions(-)

diff --git a/scripts/runtime.bash b/scripts/runtime.bash
index c88e246245a6..caa4c5ba18cc 100644
--- a/scripts/runtime.bash
+++ b/scripts/runtime.bash
@@ -53,9 +53,6 @@ skip_nodefault()
 
 function print_result()
 {
-    # output test results in a TAP format
-    # https://testanything.org/tap-version-13-specification.html
-
     local status="$1"
     local testname="$2"
     local summary="$3"
-- 
2.25.4

Re: [kvm-unit-tests PATCH v2 1/2] runtime.bash: remove outdated comment

[Thomas Huth]

On 25/08/2020 12.20, Marc Hartmayer wrote:
> Since commit 6e1d3752d7ca ("tap13: list testcases individually") the
> comment is no longer valid. Therefore let's remove it.
> 
> Reviewed-by: Andrew Jones <drjones@redhat.com>
> Reviewed-by: Cornelia Huck <cohuck@redhat.com>
> Signed-off-by: Marc Hartmayer <mhartmay@linux.ibm.com>
> ---
>  scripts/runtime.bash | 3 ---
>  1 file changed, 3 deletions(-)
> 
> diff --git a/scripts/runtime.bash b/scripts/runtime.bash
> index c88e246245a6..caa4c5ba18cc 100644
> --- a/scripts/runtime.bash
> +++ b/scripts/runtime.bash
> @@ -53,9 +53,6 @@ skip_nodefault()
>  
>  function print_result()
>  {
> -    # output test results in a TAP format
> -    # https://testanything.org/tap-version-13-specification.html
> -
>      local status="$1"
>      local testname="$2"
>      local summary="$3"
> 

Reviewed-by: Thomas Huth <thuth@redhat.com>

[kvm-unit-tests PATCH v2 2/2] Use same test names in the default and the TAP13 output format

[Marc Hartmayer]

Use the same test names in the TAP13 output as in the default output
format. This makes the output more consistent. To achieve this, we
need to pass the test name as an argument to the function
`process_test_output`.

Before this change:
$ ./run_tests.sh
PASS selftest-setup (14 tests)
...

vs.

$ ./run_tests.sh -t
TAP version 13
ok 1 - selftest: true
ok 2 - selftest: argc == 3
...

After this change:
$ ./run_tests.sh
PASS selftest-setup (14 tests)
...

vs.

$ ./run_tests.sh -t
TAP version 13
ok 1 - selftest-setup: selftest: true
ok 2 - selftest-setup: selftest: argc == 3
...

While at it, introduce a local variable `kernel` in
`RUNTIME_log_stdout` since this makes the function easier to read.

Signed-off-by: Marc Hartmayer <mhartmay@linux.ibm.com>
---
 run_tests.sh         | 15 +++++++++------
 scripts/runtime.bash |  6 +++---
 2 files changed, 12 insertions(+), 9 deletions(-)

diff --git a/run_tests.sh b/run_tests.sh
index 01e36dcfa06e..f49dd864524e 100755
--- a/run_tests.sh
+++ b/run_tests.sh
@@ -81,18 +81,19 @@ if [[ $tap_output == "no" ]]; then
     postprocess_suite_output() { cat; }
 else
     process_test_output() {
+        local testname="$1"
         CR=$'\r'
         while read -r line; do
             line="${line%$CR}"
             case "${line:0:4}" in
                 PASS)
-                    echo "ok TEST_NUMBER - ${line#??????}" >&3
+                    echo "ok TEST_NUMBER - ${testname}: ${line#??????}" >&3
                     ;;
                 FAIL)
-                    echo "not ok TEST_NUMBER - ${line#??????}" >&3
+                    echo "not ok TEST_NUMBER - ${testname}: ${line#??????}" >&3
                     ;;
                 SKIP)
-                    echo "ok TEST_NUMBER - ${line#??????} # skip" >&3
+                    echo "ok TEST_NUMBER - ${testname}: ${line#??????} # skip" >&3
                     ;;
                 *)
                     ;;
@@ -114,12 +115,14 @@ else
     }
 fi
 
-RUNTIME_log_stderr () { process_test_output; }
+RUNTIME_log_stderr () { process_test_output "$1"; }
 RUNTIME_log_stdout () {
+    local testname="$1"
     if [ "$PRETTY_PRINT_STACKS" = "yes" ]; then
-        ./scripts/pretty_print_stacks.py $1 | process_test_output
+        local kernel="$2"
+        ./scripts/pretty_print_stacks.py "$kernel" | process_test_output "$testname"
     else
-        process_test_output
+        process_test_output "$testname"
     fi
 }
 
diff --git a/scripts/runtime.bash b/scripts/runtime.bash
index caa4c5ba18cc..294e6b15a5e2 100644
--- a/scripts/runtime.bash
+++ b/scripts/runtime.bash
@@ -140,10 +140,10 @@ function run()
     # extra_params in the config file may contain backticks that need to be
     # expanded, so use eval to start qemu.  Use "> >(foo)" instead of a pipe to
     # preserve the exit status.
-    summary=$(eval $cmdline 2> >(RUNTIME_log_stderr) \
-                             > >(tee >(RUNTIME_log_stdout $kernel) | extract_summary))
+    summary=$(eval $cmdline 2> >(RUNTIME_log_stderr $testname) \
+                             > >(tee >(RUNTIME_log_stdout $testname $kernel) | extract_summary))
     ret=$?
-    [ "$STANDALONE" != "yes" ] && echo > >(RUNTIME_log_stdout $kernel)
+    [ "$STANDALONE" != "yes" ] && echo > >(RUNTIME_log_stdout $testname $kernel)
 
     if [ $ret -eq 0 ]; then
         print_result "PASS" $testname "$summary"
-- 
2.25.4

Re: [kvm-unit-tests PATCH v2 2/2] Use same test names in the default and the TAP13 output format

[Cornelia Huck]

On Tue, 25 Aug 2020 12:20:36 +0200
Marc Hartmayer <mhartmay@linux.ibm.com> wrote:

> Use the same test names in the TAP13 output as in the default output
> format. This makes the output more consistent. To achieve this, we
> need to pass the test name as an argument to the function
> `process_test_output`.
> 
> Before this change:
> $ ./run_tests.sh
> PASS selftest-setup (14 tests)
> ...
> 
> vs.
> 
> $ ./run_tests.sh -t
> TAP version 13
> ok 1 - selftest: true
> ok 2 - selftest: argc == 3
> ...
> 
> After this change:
> $ ./run_tests.sh
> PASS selftest-setup (14 tests)
> ...
> 
> vs.
> 
> $ ./run_tests.sh -t
> TAP version 13
> ok 1 - selftest-setup: selftest: true
> ok 2 - selftest-setup: selftest: argc == 3
> ...
> 
> While at it, introduce a local variable `kernel` in
> `RUNTIME_log_stdout` since this makes the function easier to read.
> 
> Signed-off-by: Marc Hartmayer <mhartmay@linux.ibm.com>
> ---
>  run_tests.sh         | 15 +++++++++------
>  scripts/runtime.bash |  6 +++---
>  2 files changed, 12 insertions(+), 9 deletions(-)

Reviewed-by: Cornelia Huck <cohuck@redhat.com>

Re: [kvm-unit-tests PATCH v2 2/2] Use same test names in the default and the TAP13 output format

[Marc Hartmayer]

On Tue, Aug 25, 2020 at 02:13 PM +0200, Cornelia Huck <cohuck@redhat.com> wrote:
> On Tue, 25 Aug 2020 12:20:36 +0200
> Marc Hartmayer <mhartmay@linux.ibm.com> wrote:
>
>> Use the same test names in the TAP13 output as in the default output
>> format. This makes the output more consistent. To achieve this, we
>> need to pass the test name as an argument to the function
>> `process_test_output`.
>> 
>> Before this change:
>> $ ./run_tests.sh
>> PASS selftest-setup (14 tests)
>> ...
>> 
>> vs.
>> 
>> $ ./run_tests.sh -t
>> TAP version 13
>> ok 1 - selftest: true
>> ok 2 - selftest: argc == 3
>> ...
>> 
>> After this change:
>> $ ./run_tests.sh
>> PASS selftest-setup (14 tests)
>> ...
>> 
>> vs.
>> 
>> $ ./run_tests.sh -t
>> TAP version 13
>> ok 1 - selftest-setup: selftest: true
>> ok 2 - selftest-setup: selftest: argc == 3
>> ...
>> 
>> While at it, introduce a local variable `kernel` in
>> `RUNTIME_log_stdout` since this makes the function easier to read.
>> 
>> Signed-off-by: Marc Hartmayer <mhartmay@linux.ibm.com>
>> ---
>>  run_tests.sh         | 15 +++++++++------
>>  scripts/runtime.bash |  6 +++---
>>  2 files changed, 12 insertions(+), 9 deletions(-)
>
> Reviewed-by: Cornelia Huck <cohuck@redhat.com>
>

Thanks.

-- 
Kind regards / Beste Grüße
   Marc Hartmayer

IBM Deutschland Research & Development GmbH
Vorsitzender des Aufsichtsrats: Gregor Pillen 
Geschäftsführung: Dirk Wittkopp
Sitz der Gesellschaft: Böblingen
Registergericht: Amtsgericht Stuttgart, HRB 243294

Re: [kvm-unit-tests PATCH v2 2/2] Use same test names in the default and the TAP13 output format

[Thomas Huth]

On 25/08/2020 12.20, Marc Hartmayer wrote:
> Use the same test names in the TAP13 output as in the default output
> format. This makes the output more consistent. To achieve this, we
> need to pass the test name as an argument to the function
> `process_test_output`.
> 
> Before this change:
> $ ./run_tests.sh
> PASS selftest-setup (14 tests)
> ...
> 
> vs.
> 
> $ ./run_tests.sh -t
> TAP version 13
> ok 1 - selftest: true
> ok 2 - selftest: argc == 3
> ...
> 
> After this change:
> $ ./run_tests.sh
> PASS selftest-setup (14 tests)
> ...
> 
> vs.
> 
> $ ./run_tests.sh -t
> TAP version 13
> ok 1 - selftest-setup: selftest: true
> ok 2 - selftest-setup: selftest: argc == 3
> ...
> 
> While at it, introduce a local variable `kernel` in
> `RUNTIME_log_stdout` since this makes the function easier to read.
> 
> Signed-off-by: Marc Hartmayer <mhartmay@linux.ibm.com>
> ---
>  run_tests.sh         | 15 +++++++++------
>  scripts/runtime.bash |  6 +++---
>  2 files changed, 12 insertions(+), 9 deletions(-)

Reviewed-by: Thomas Huth <thuth@redhat.com>

Re: [kvm-unit-tests PATCH v2 2/2] Use same test names in the default and the TAP13 output format

[Marc Hartmayer]

On Tue, Aug 25, 2020 at 03:32 PM +0200, Thomas Huth <thuth@redhat.com> wrote:
> On 25/08/2020 12.20, Marc Hartmayer wrote:
>> Use the same test names in the TAP13 output as in the default output
>> format. This makes the output more consistent. To achieve this, we
>> need to pass the test name as an argument to the function
>> `process_test_output`.
>> 
>> Before this change:
>> $ ./run_tests.sh
>> PASS selftest-setup (14 tests)
>> ...
>> 
>> vs.
>> 
>> $ ./run_tests.sh -t
>> TAP version 13
>> ok 1 - selftest: true
>> ok 2 - selftest: argc == 3
>> ...
>> 
>> After this change:
>> $ ./run_tests.sh
>> PASS selftest-setup (14 tests)
>> ...
>> 
>> vs.
>> 
>> $ ./run_tests.sh -t
>> TAP version 13
>> ok 1 - selftest-setup: selftest: true
>> ok 2 - selftest-setup: selftest: argc == 3
>> ...
>> 
>> While at it, introduce a local variable `kernel` in
>> `RUNTIME_log_stdout` since this makes the function easier to read.
>> 
>> Signed-off-by: Marc Hartmayer <mhartmay@linux.ibm.com>
>> ---
>>  run_tests.sh         | 15 +++++++++------
>>  scripts/runtime.bash |  6 +++---
>>  2 files changed, 12 insertions(+), 9 deletions(-)
>
> Reviewed-by: Thomas Huth <thuth@redhat.com>
>

Thanks!

-- 
Kind regards / Beste Grüße
   Marc Hartmayer

IBM Deutschland Research & Development GmbH
Vorsitzender des Aufsichtsrats: Gregor Pillen 
Geschäftsführung: Dirk Wittkopp
Sitz der Gesellschaft: Böblingen
Registergericht: Amtsgericht Stuttgart, HRB 243294

Re: [kvm-unit-tests PATCH v2 0/2] Use same test names in the default and the TAP13 output format

[Marc Hartmayer]

On Tue, Aug 25, 2020 at 12:20 PM +0200, Marc Hartmayer <mhartmay@linux.ibm.com> wrote:
> For everybody's convenience there is a branch:
> https://gitlab.com/mhartmay/kvm-unit-tests/-/tree/tap_v2
>
> Changelog:
> v1 -> v2:
>  + added r-b's to patch 1
>  + patch 2:
>   - I've not added Andrew's r-b since I've worked in the comment from
>     Janosch (don't drop the first prefix)
>
> Marc Hartmayer (2):
>   runtime.bash: remove outdated comment
>   Use same test names in the default and the TAP13 output format
>
>  run_tests.sh         | 15 +++++++++------
>  scripts/runtime.bash |  9 +++------
>  2 files changed, 12 insertions(+), 12 deletions(-)
>
> -- 
> 2.25.4
>

Polite ping :) How should we proceed further?

-- 
Kind regards / Beste Grüße
   Marc Hartmayer

IBM Deutschland Research & Development GmbH
Vorsitzender des Aufsichtsrats: Gregor Pillen 
Geschäftsführung: Dirk Wittkopp
Sitz der Gesellschaft: Böblingen
Registergericht: Amtsgericht Stuttgart, HRB 243294

Re: [kvm-unit-tests PATCH v2 0/2] Use same test names in the default and the TAP13 output format

[Thomas Huth]

On 15/09/2020 11.38, Marc Hartmayer wrote:
> On Tue, Aug 25, 2020 at 12:20 PM +0200, Marc Hartmayer <mhartmay@linux.ibm.com> wrote:
>> For everybody's convenience there is a branch:
>> https://gitlab.com/mhartmay/kvm-unit-tests/-/tree/tap_v2
>>
>> Changelog:
>> v1 -> v2:
>>  + added r-b's to patch 1
>>  + patch 2:
>>   - I've not added Andrew's r-b since I've worked in the comment from
>>     Janosch (don't drop the first prefix)
>>
>> Marc Hartmayer (2):
>>   runtime.bash: remove outdated comment
>>   Use same test names in the default and the TAP13 output format
>>
>>  run_tests.sh         | 15 +++++++++------
>>  scripts/runtime.bash |  9 +++------
>>  2 files changed, 12 insertions(+), 12 deletions(-)
>>
>> -- 
>> 2.25.4
>>
> 
> Polite ping :) How should we proceed further?

Sorry, it's been some quite busy weeks ... I'll try to collect some
pending kvm-unit-tests patches in the next days and then send a pull
request to Paolo...

 Thomas

Re: [kvm-unit-tests PATCH v2 0/2] Use same test names in the default and the TAP13 output format

[Marc Hartmayer]

On Tue, Sep 22, 2020 at 10:53 AM +0200, Thomas Huth <thuth@redhat.com> wrote:
> On 15/09/2020 11.38, Marc Hartmayer wrote:
>> On Tue, Aug 25, 2020 at 12:20 PM +0200, Marc Hartmayer <mhartmay@linux.ibm.com> wrote:
>>> For everybody's convenience there is a branch:
>>> https://gitlab.com/mhartmay/kvm-unit-tests/-/tree/tap_v2
>>>
>>> Changelog:
>>> v1 -> v2:
>>>  + added r-b's to patch 1
>>>  + patch 2:
>>>   - I've not added Andrew's r-b since I've worked in the comment from
>>>     Janosch (don't drop the first prefix)
>>>
>>> Marc Hartmayer (2):
>>>   runtime.bash: remove outdated comment
>>>   Use same test names in the default and the TAP13 output format
>>>
>>>  run_tests.sh         | 15 +++++++++------
>>>  scripts/runtime.bash |  9 +++------
>>>  2 files changed, 12 insertions(+), 12 deletions(-)
>>>
>>> -- 
>>> 2.25.4
>>>
>> 
>> Polite ping :) How should we proceed further?
>
> Sorry, it's been some quite busy weeks ... I'll try to collect some
> pending kvm-unit-tests patches in the next days and then send a pull
> request to Paolo...

No problem :) Thanks for taking care!

>
>  Thomas
>
-- 
Kind regards / Beste Grüße
   Marc Hartmayer

IBM Deutschland Research & Development GmbH
Vorsitzender des Aufsichtsrats: Gregor Pillen 
Geschäftsführung: Dirk Wittkopp
Sitz der Gesellschaft: Böblingen
Registergericht: Amtsgericht Stuttgart, HRB 243294

Re: [kvm-unit-tests PATCH v2 0/2] Use same test names in the default and the TAP13 output format

[Paolo Bonzini]

On 22/09/20 10:53, Thomas Huth wrote:
> On 15/09/2020 11.38, Marc Hartmayer wrote:
>> On Tue, Aug 25, 2020 at 12:20 PM +0200, Marc Hartmayer <mhartmay@linux.ibm.com> wrote:
>>> For everybody's convenience there is a branch:
>>> https://gitlab.com/mhartmay/kvm-unit-tests/-/tree/tap_v2
>>>
>>> Changelog:
>>> v1 -> v2:
>>>  + added r-b's to patch 1
>>>  + patch 2:
>>>   - I've not added Andrew's r-b since I've worked in the comment from
>>>     Janosch (don't drop the first prefix)
>>>
>>> Marc Hartmayer (2):
>>>   runtime.bash: remove outdated comment
>>>   Use same test names in the default and the TAP13 output format
>>>
>>>  run_tests.sh         | 15 +++++++++------
>>>  scripts/runtime.bash |  9 +++------
>>>  2 files changed, 12 insertions(+), 12 deletions(-)
>>>
>>> -- 
>>> 2.25.4
>>>
>>
>> Polite ping :) How should we proceed further?
> 
> Sorry, it's been some quite busy weeks ... I'll try to collect some
> pending kvm-unit-tests patches in the next days and then send a pull
> request to Paolo...
> 
>  Thomas
> 

Don't worry, I'm finally back and learning again how to do code review
with "lighter" series. :)

Paolo

Re: [kvm-unit-tests PATCH v2 0/2] Use same test names in the default and the TAP13 output format

[Paolo Bonzini]

On 25/08/20 12:20, Marc Hartmayer wrote:
> For everybody's convenience there is a branch:
> https://gitlab.com/mhartmay/kvm-unit-tests/-/tree/tap_v2
> 
> Changelog:
> v1 -> v2:
>  + added r-b's to patch 1
>  + patch 2:
>   - I've not added Andrew's r-b since I've worked in the comment from
>     Janosch (don't drop the first prefix)
> 
> Marc Hartmayer (2):
>   runtime.bash: remove outdated comment
>   Use same test names in the default and the TAP13 output format
> 
>  run_tests.sh         | 15 +++++++++------
>  scripts/runtime.bash |  9 +++------
>  2 files changed, 12 insertions(+), 12 deletions(-)
> 

Queued, thanks.

Paolo