All of lore.kernel.org
 help / color / mirror / Atom feed
From: Sean Christopherson <seanjc@google.com>
To: Vipin Sharma <vipinsh@google.com>
Cc: kvm@vger.kernel.org, kvmarm@lists.linux.dev,
	kvm-riscv@lists.infradead.org,
	 linux-arm-kernel@lists.infradead.org, pbonzini@redhat.com,
	 anup@brainfault.org, borntraeger@linux.ibm.com,
	frankja@linux.ibm.com,  imbrenda@linux.ibm.com, maz@kernel.org,
	oliver.upton@linux.dev,  dmatlack@google.com
Subject: Re: [PATCH v2 06/15] KVM: selftests: Add a flag to print only test status in KVM Selftests run
Date: Wed, 9 Jul 2025 14:55:25 -0700	[thread overview]
Message-ID: <aG7lTY2ItIkfX7dm@google.com> (raw)
In-Reply-To: <20250606235619.1841595-7-vipinsh@google.com>

On Fri, Jun 06, 2025, Vipin Sharma wrote:
> Add a command line argument, --print-status, to limit content printed on
> terminal by default. When this flag is passed only print final status of
> tests i.e. passed, failed, timed out, etc.
> 
> Example:
>   python3 runner --test-dirs tests  --print-status
> 
> Signed-off-by: Vipin Sharma <vipinsh@google.com>
> ---
>  tools/testing/selftests/kvm/runner/__main__.py    | 5 +++++
>  tools/testing/selftests/kvm/runner/test_runner.py | 3 ++-
>  2 files changed, 7 insertions(+), 1 deletion(-)
> 
> diff --git a/tools/testing/selftests/kvm/runner/__main__.py b/tools/testing/selftests/kvm/runner/__main__.py
> index 48d7ce00a097..3f11a20e76a9 100644
> --- a/tools/testing/selftests/kvm/runner/__main__.py
> +++ b/tools/testing/selftests/kvm/runner/__main__.py
> @@ -59,6 +59,11 @@ def cli():
>                          type=int,
>                          help="Maximum number of tests that can be run concurrently. (Default: 1)")
>  
> +    parser.add_argument("--print-status",
> +                        action="store_true",
> +                        default=False,
> +                        help="Print only test's status and avoid printing stdout and stderr of the tests")
> +
>      return parser.parse_args()
>  
>  
> diff --git a/tools/testing/selftests/kvm/runner/test_runner.py b/tools/testing/selftests/kvm/runner/test_runner.py
> index 0a6e5e0ca0f5..474408fcab51 100644
> --- a/tools/testing/selftests/kvm/runner/test_runner.py
> +++ b/tools/testing/selftests/kvm/runner/test_runner.py
> @@ -17,6 +17,7 @@ class TestRunner:
>          self.tests = []
>          self.output_dir = args.output
>          self.jobs = args.jobs
> +        self.print_status = args.print_status
>  
>          for test_file in test_files:
>              self.tests.append(Selftest(test_file, args.executable,
> @@ -29,7 +30,7 @@ class TestRunner:
>      def _log_result(self, test_result):
>          logger.log(test_result.status,
>                     f"[{test_result.status}] {test_result.test_path}")
> -        if (self.output_dir is None):
> +        if (self.output_dir is None and self.print_status is False):

This flag confused the hell out of me.  It's not at all obvious that it *reduces*
output, and that it overrides all the other --print-xxx options.  I see no reason
to provide this.  I suspect most users will want curated console information, or
none at all.

If we keep this, the runner should explicitly disallow --print-status with any
other --print-xxx option.

>              logger.info("************** STDOUT BEGIN **************")
>              logger.info(test_result.stdout)
>              logger.info("************** STDOUT END **************")
> -- 
> 2.50.0.rc0.604.gd4ff7b7c86-goog
> 

-- 
kvm-riscv mailing list
kvm-riscv@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/kvm-riscv

WARNING: multiple messages have this Message-ID (diff)
From: Sean Christopherson <seanjc@google.com>
To: Vipin Sharma <vipinsh@google.com>
Cc: kvm@vger.kernel.org, kvmarm@lists.linux.dev,
	kvm-riscv@lists.infradead.org,
	 linux-arm-kernel@lists.infradead.org, pbonzini@redhat.com,
	 anup@brainfault.org, borntraeger@linux.ibm.com,
	frankja@linux.ibm.com,  imbrenda@linux.ibm.com, maz@kernel.org,
	oliver.upton@linux.dev,  dmatlack@google.com
Subject: Re: [PATCH v2 06/15] KVM: selftests: Add a flag to print only test status in KVM Selftests run
Date: Wed, 9 Jul 2025 14:55:25 -0700	[thread overview]
Message-ID: <aG7lTY2ItIkfX7dm@google.com> (raw)
In-Reply-To: <20250606235619.1841595-7-vipinsh@google.com>

On Fri, Jun 06, 2025, Vipin Sharma wrote:
> Add a command line argument, --print-status, to limit content printed on
> terminal by default. When this flag is passed only print final status of
> tests i.e. passed, failed, timed out, etc.
> 
> Example:
>   python3 runner --test-dirs tests  --print-status
> 
> Signed-off-by: Vipin Sharma <vipinsh@google.com>
> ---
>  tools/testing/selftests/kvm/runner/__main__.py    | 5 +++++
>  tools/testing/selftests/kvm/runner/test_runner.py | 3 ++-
>  2 files changed, 7 insertions(+), 1 deletion(-)
> 
> diff --git a/tools/testing/selftests/kvm/runner/__main__.py b/tools/testing/selftests/kvm/runner/__main__.py
> index 48d7ce00a097..3f11a20e76a9 100644
> --- a/tools/testing/selftests/kvm/runner/__main__.py
> +++ b/tools/testing/selftests/kvm/runner/__main__.py
> @@ -59,6 +59,11 @@ def cli():
>                          type=int,
>                          help="Maximum number of tests that can be run concurrently. (Default: 1)")
>  
> +    parser.add_argument("--print-status",
> +                        action="store_true",
> +                        default=False,
> +                        help="Print only test's status and avoid printing stdout and stderr of the tests")
> +
>      return parser.parse_args()
>  
>  
> diff --git a/tools/testing/selftests/kvm/runner/test_runner.py b/tools/testing/selftests/kvm/runner/test_runner.py
> index 0a6e5e0ca0f5..474408fcab51 100644
> --- a/tools/testing/selftests/kvm/runner/test_runner.py
> +++ b/tools/testing/selftests/kvm/runner/test_runner.py
> @@ -17,6 +17,7 @@ class TestRunner:
>          self.tests = []
>          self.output_dir = args.output
>          self.jobs = args.jobs
> +        self.print_status = args.print_status
>  
>          for test_file in test_files:
>              self.tests.append(Selftest(test_file, args.executable,
> @@ -29,7 +30,7 @@ class TestRunner:
>      def _log_result(self, test_result):
>          logger.log(test_result.status,
>                     f"[{test_result.status}] {test_result.test_path}")
> -        if (self.output_dir is None):
> +        if (self.output_dir is None and self.print_status is False):

This flag confused the hell out of me.  It's not at all obvious that it *reduces*
output, and that it overrides all the other --print-xxx options.  I see no reason
to provide this.  I suspect most users will want curated console information, or
none at all.

If we keep this, the runner should explicitly disallow --print-status with any
other --print-xxx option.

>              logger.info("************** STDOUT BEGIN **************")
>              logger.info(test_result.stdout)
>              logger.info("************** STDOUT END **************")
> -- 
> 2.50.0.rc0.604.gd4ff7b7c86-goog
> 

  reply	other threads:[~2025-07-09 22:57 UTC|newest]

Thread overview: 54+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2025-06-06 23:56 [PATCH v2 00/15] Add KVM Selftests runner Vipin Sharma
2025-06-06 23:56 ` Vipin Sharma
2025-06-06 23:56 ` [PATCH v2 01/15] KVM: selftest: Create KVM selftest runner Vipin Sharma
2025-06-06 23:56   ` Vipin Sharma
2025-07-10  0:20   ` Sean Christopherson
2025-07-10  0:20     ` Sean Christopherson
2025-06-06 23:56 ` [PATCH v2 02/15] KVM: selftests: Enable selftests runner to find executables in different path Vipin Sharma
2025-06-06 23:56   ` Vipin Sharma
2025-07-09 21:39   ` Sean Christopherson
2025-07-09 21:39     ` Sean Christopherson
2025-06-06 23:56 ` [PATCH v2 03/15] KVM: selftests: Add timeout option in selftests runner Vipin Sharma
2025-06-06 23:56   ` Vipin Sharma
2025-07-09 21:46   ` Sean Christopherson
2025-07-09 21:46     ` Sean Christopherson
2025-07-09 22:25   ` Sean Christopherson
2025-07-09 22:25     ` Sean Christopherson
2025-06-06 23:56 ` [PATCH v2 04/15] KVM: selftests: Add option to save selftest runner output to a directory Vipin Sharma
2025-06-06 23:56   ` Vipin Sharma
2025-07-09 21:52   ` Sean Christopherson
2025-07-09 21:52     ` Sean Christopherson
2025-06-06 23:56 ` [PATCH v2 05/15] KVM: selftests: Run tests concurrently in KVM selftests runner Vipin Sharma
2025-06-06 23:56   ` Vipin Sharma
2025-06-06 23:56 ` [PATCH v2 06/15] KVM: selftests: Add a flag to print only test status in KVM Selftests run Vipin Sharma
2025-06-06 23:56   ` Vipin Sharma
2025-07-09 21:55   ` Sean Christopherson [this message]
2025-07-09 21:55     ` Sean Christopherson
2025-06-06 23:56 ` [PATCH v2 07/15] KVM: selftests: Add various print flags to KVM Selftest Runner Vipin Sharma
2025-06-06 23:56   ` Vipin Sharma
2025-07-09 22:01   ` Sean Christopherson
2025-07-09 22:01     ` Sean Christopherson
2025-06-06 23:56 ` [PATCH v2 08/15] KVM: selftests: Print sticky KVM Selftests Runner status at bottom Vipin Sharma
2025-06-06 23:56   ` Vipin Sharma
2025-06-06 23:56 ` [PATCH v2 09/15] KVM: selftests: Add a flag to print only sticky summary in the selftests runner Vipin Sharma
2025-06-06 23:56   ` Vipin Sharma
2025-06-06 23:56 ` [PATCH v2 10/15] KVM: selftests: Add flag to suppress all output from Selftest KVM Runner Vipin Sharma
2025-06-06 23:56   ` Vipin Sharma
2025-06-06 23:56 ` [PATCH v2 11/15] KVM: selftests: Auto generate default tests for KVM Selftests Runner Vipin Sharma
2025-06-06 23:56   ` Vipin Sharma
2025-07-09 23:06   ` Oliver Upton
2025-07-09 23:06     ` Oliver Upton
2025-07-10  0:18     ` Sean Christopherson
2025-07-10  0:18       ` Sean Christopherson
2025-06-06 23:56 ` [PATCH v2 12/15] KVM: selftests: Add x86 auto generated test files " Vipin Sharma
2025-06-06 23:56   ` Vipin Sharma
2025-06-06 23:56 ` [PATCH v2 13/15] KVM: selftests: Add arm64 " Vipin Sharma
2025-06-06 23:56   ` Vipin Sharma
2025-06-06 23:56 ` [PATCH v2 14/15] KVM: selftests: Add s390 " Vipin Sharma
2025-06-06 23:56   ` Vipin Sharma
2025-06-06 23:56 ` [PATCH v2 15/15] KVM: selftests: Add riscv " Vipin Sharma
2025-06-06 23:56   ` Vipin Sharma
2025-06-09 12:54   ` Andrew Jones
2025-06-09 12:54     ` Andrew Jones
2025-07-09 22:25 ` [PATCH v2 00/15] Add KVM Selftests runner Sean Christopherson
2025-07-09 22:25   ` Sean Christopherson

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=aG7lTY2ItIkfX7dm@google.com \
    --to=seanjc@google.com \
    --cc=anup@brainfault.org \
    --cc=borntraeger@linux.ibm.com \
    --cc=dmatlack@google.com \
    --cc=frankja@linux.ibm.com \
    --cc=imbrenda@linux.ibm.com \
    --cc=kvm-riscv@lists.infradead.org \
    --cc=kvm@vger.kernel.org \
    --cc=kvmarm@lists.linux.dev \
    --cc=linux-arm-kernel@lists.infradead.org \
    --cc=maz@kernel.org \
    --cc=oliver.upton@linux.dev \
    --cc=pbonzini@redhat.com \
    --cc=vipinsh@google.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.