Re: [PATCH v2 07/15] KVM: selftests: Add various print flags to KVM Selftest Runner

[Sean Christopherson <seanjc@google.com>]

On Fri, Jun 06, 2025, Vipin Sharma wrote:
> Add various print flags to selectively print outputs on terminal based
> on test execution status (passed, failed, timed out, skipped, no run).
> 
> Provide further options to print only particular execution status, like
> print only status of failed tests.
> 
> Example: To print status, stdout and stderr for failed tests and only
> print status of passed test:
> 
>    python3 runner --test-dirs tests  --print-failed \
>    --print-passed-status
> 
> Signed-off-by: Vipin Sharma <vipinsh@google.com>
> ---
>  .../testing/selftests/kvm/runner/__main__.py  | 114 ++++++++++++++++++
>  .../selftests/kvm/runner/test_runner.py       |  10 +-
>  2 files changed, 123 insertions(+), 1 deletion(-)
> 
> diff --git a/tools/testing/selftests/kvm/runner/__main__.py b/tools/testing/selftests/kvm/runner/__main__.py
> index 3f11a20e76a9..4406d8e4847a 100644
> --- a/tools/testing/selftests/kvm/runner/__main__.py
> +++ b/tools/testing/selftests/kvm/runner/__main__.py
> @@ -64,9 +64,115 @@ def cli():
>                          default=False,
>                          help="Print only test's status and avoid printing stdout and stderr of the tests")
>  
> +    parser.add_argument("--print-passed",
> +                        action="store_true",
> +                        default=False,
> +                        help="Print passed test's stdout, stderr and status."
> +                        )
> +
> +    parser.add_argument("--print-passed-status",
> +                        action="store_true",
> +                        default=False,
> +                        help="Print only passed test's status."
> +                        )

Waaay too many booleans :-)

And they don't provide the right granularity.  E.g. I don't want stdout for FAILED,
I just want stderr for the assert.

After some fiddling, I came up with this:

    parser.add_argument("--print-passed", default="full", const="full", nargs='?', choices=["off", "full", "stderr", "stdout", "status"],
                        help="blah"
                        )

    parser.add_argument("--print-failed", default="full", const="full", nargs='?', choices=["off", "full", "stderr", "stdout", "status"],
                        help="Full = print each test's stdout, stderr and status; status = only status."
                        )

    parser.add_argument("--print-skipped", default="full", const="full", nargs='?', choices=["off", "full", "stderr", "stdout", "status"],
                        help="Print skipped test's stdout, stderr and status."
                        )

    parser.add_argument("--print-timed-out", default="full", const="full", nargs='?', choices=["off", "full", "stderr", "stdout", "status"],
                        help="Print timed out test's stdout, stderr and status."
                        )

    parser.add_argument("--print-no-run", default="full", const="full", nargs='?', choices=["off", "full", "stderr", "stdout", "status"],
                        help="Print stdout, stderr and status for tests which didn't run."
                        )

The const+nargs magic makes a bare option equivalent to "full", e.g. --print-timed-out
and --print-timed-out=full are the same.  That gives the user complete control
over the output, with _less_ code in the runner.  E.g. to print stdout and/or stderr:

        print_level = self.print_stds.get(test_result.status);
        if (print_level == "full" or print_level == "stdout"):
            logger.info("*** stdout ***\n" + test_result.stdout)

        if (print_level == "full" or print_level == "stderr"):
            logger.info("*** stderr ***\n" + test_result.stderr)