All of lore.kernel.org
 help / color / mirror / Atom feed
From: Sean Christopherson <seanjc@google.com>
To: Vipin Sharma <vipinsh@google.com>
Cc: kvm@vger.kernel.org, kvmarm@lists.linux.dev,
	kvm-riscv@lists.infradead.org,
	 linux-arm-kernel@lists.infradead.org, pbonzini@redhat.com,
	 anup@brainfault.org, borntraeger@linux.ibm.com,
	frankja@linux.ibm.com,  imbrenda@linux.ibm.com, maz@kernel.org,
	oliver.upton@linux.dev,  dmatlack@google.com
Subject: Re: [PATCH v2 03/15] KVM: selftests: Add timeout option in selftests runner
Date: Wed, 9 Jul 2025 14:46:20 -0700	[thread overview]
Message-ID: <aG7jLBAqo1F3SjCl@google.com> (raw)
In-Reply-To: <20250606235619.1841595-4-vipinsh@google.com>

On Fri, Jun 06, 2025, Vipin Sharma wrote:
> diff --git a/tools/testing/selftests/kvm/runner/command.py b/tools/testing/selftests/kvm/runner/command.py
> index a63ff53a92b3..44c8e0875779 100644
> --- a/tools/testing/selftests/kvm/runner/command.py
> +++ b/tools/testing/selftests/kvm/runner/command.py
> @@ -12,14 +12,16 @@ class Command:
>      Returns the exit code, std output and std error of the command.
>      """
>  
> -    def __init__(self, command):
> +    def __init__(self, command, timeout):
>          self.command = command
> +        self.timeout = timeout
>  
>      def run(self):
>          run_args = {
>              "universal_newlines": True,
>              "shell": True,
>              "capture_output": True,
> +            "timeout": self.timeout,
>          }
> @@ -48,10 +50,13 @@ class Selftest:
>              self.stderr = "File doesn't exists."
>              return
>  
> -        ret, self.stdout, self.stderr = self.command.run()
> -        if ret == 0:
> -            self.status = SelftestStatus.PASSED
> -        elif ret == 4:
> -            self.status = SelftestStatus.SKIPPED
> -        else:
> -            self.status = SelftestStatus.FAILED
> +        try:
> +            ret, self.stdout, self.stderr = self.command.run()

I don't see any value in having both command.py and selftest.py.  TimeoutExpired
*really* should be handled by Command, especially with respect to stdout/stderr
(more on that later), but that complicates converting return codes to SelftestStatus.

And even if we do figure out a clean split, one of Command or Selftest would end
but being little more than a wrapper or trampoline, with more boilerplate code
than novel logic.

I don't anticipate turning this into a general execution framework.  The entire
purpose is to run a selftest, so I think it makes sense to scrap Command and just
have Selftest deal with running the executable.

> +            if ret == 0:
> +                self.status = SelftestStatus.PASSED
> +            elif ret == 4:
> +                self.status = SelftestStatus.SKIPPED
> +            else:
> +                self.status = SelftestStatus.FAILED
> +        except subprocess.TimeoutExpired as e:
> +            self.status = SelftestStatus.TIMED_OUT
> diff --git a/tools/testing/selftests/kvm/runner/test_runner.py b/tools/testing/selftests/kvm/runner/test_runner.py
> index 104f0b4c2e4e..1409e1cfe7d5 100644
> --- a/tools/testing/selftests/kvm/runner/test_runner.py
> +++ b/tools/testing/selftests/kvm/runner/test_runner.py
> @@ -15,7 +15,7 @@ class TestRunner:
>          self.tests = []
>  
>          for test_file in test_files:
> -            self.tests.append(Selftest(test_file, args.executable))
> +            self.tests.append(Selftest(test_file, args.executable, args.timeout))
>  
>      def _log_result(self, test_result):
>          logger.log(test_result.status,
> -- 
> 2.50.0.rc0.604.gd4ff7b7c86-goog
> 

-- 
kvm-riscv mailing list
kvm-riscv@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/kvm-riscv

WARNING: multiple messages have this Message-ID (diff)
From: Sean Christopherson <seanjc@google.com>
To: Vipin Sharma <vipinsh@google.com>
Cc: kvm@vger.kernel.org, kvmarm@lists.linux.dev,
	kvm-riscv@lists.infradead.org,
	 linux-arm-kernel@lists.infradead.org, pbonzini@redhat.com,
	 anup@brainfault.org, borntraeger@linux.ibm.com,
	frankja@linux.ibm.com,  imbrenda@linux.ibm.com, maz@kernel.org,
	oliver.upton@linux.dev,  dmatlack@google.com
Subject: Re: [PATCH v2 03/15] KVM: selftests: Add timeout option in selftests runner
Date: Wed, 9 Jul 2025 14:46:20 -0700	[thread overview]
Message-ID: <aG7jLBAqo1F3SjCl@google.com> (raw)
In-Reply-To: <20250606235619.1841595-4-vipinsh@google.com>

On Fri, Jun 06, 2025, Vipin Sharma wrote:
> diff --git a/tools/testing/selftests/kvm/runner/command.py b/tools/testing/selftests/kvm/runner/command.py
> index a63ff53a92b3..44c8e0875779 100644
> --- a/tools/testing/selftests/kvm/runner/command.py
> +++ b/tools/testing/selftests/kvm/runner/command.py
> @@ -12,14 +12,16 @@ class Command:
>      Returns the exit code, std output and std error of the command.
>      """
>  
> -    def __init__(self, command):
> +    def __init__(self, command, timeout):
>          self.command = command
> +        self.timeout = timeout
>  
>      def run(self):
>          run_args = {
>              "universal_newlines": True,
>              "shell": True,
>              "capture_output": True,
> +            "timeout": self.timeout,
>          }
> @@ -48,10 +50,13 @@ class Selftest:
>              self.stderr = "File doesn't exists."
>              return
>  
> -        ret, self.stdout, self.stderr = self.command.run()
> -        if ret == 0:
> -            self.status = SelftestStatus.PASSED
> -        elif ret == 4:
> -            self.status = SelftestStatus.SKIPPED
> -        else:
> -            self.status = SelftestStatus.FAILED
> +        try:
> +            ret, self.stdout, self.stderr = self.command.run()

I don't see any value in having both command.py and selftest.py.  TimeoutExpired
*really* should be handled by Command, especially with respect to stdout/stderr
(more on that later), but that complicates converting return codes to SelftestStatus.

And even if we do figure out a clean split, one of Command or Selftest would end
but being little more than a wrapper or trampoline, with more boilerplate code
than novel logic.

I don't anticipate turning this into a general execution framework.  The entire
purpose is to run a selftest, so I think it makes sense to scrap Command and just
have Selftest deal with running the executable.

> +            if ret == 0:
> +                self.status = SelftestStatus.PASSED
> +            elif ret == 4:
> +                self.status = SelftestStatus.SKIPPED
> +            else:
> +                self.status = SelftestStatus.FAILED
> +        except subprocess.TimeoutExpired as e:
> +            self.status = SelftestStatus.TIMED_OUT
> diff --git a/tools/testing/selftests/kvm/runner/test_runner.py b/tools/testing/selftests/kvm/runner/test_runner.py
> index 104f0b4c2e4e..1409e1cfe7d5 100644
> --- a/tools/testing/selftests/kvm/runner/test_runner.py
> +++ b/tools/testing/selftests/kvm/runner/test_runner.py
> @@ -15,7 +15,7 @@ class TestRunner:
>          self.tests = []
>  
>          for test_file in test_files:
> -            self.tests.append(Selftest(test_file, args.executable))
> +            self.tests.append(Selftest(test_file, args.executable, args.timeout))
>  
>      def _log_result(self, test_result):
>          logger.log(test_result.status,
> -- 
> 2.50.0.rc0.604.gd4ff7b7c86-goog
> 

  reply	other threads:[~2025-07-09 22:57 UTC|newest]

Thread overview: 54+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2025-06-06 23:56 [PATCH v2 00/15] Add KVM Selftests runner Vipin Sharma
2025-06-06 23:56 ` Vipin Sharma
2025-06-06 23:56 ` [PATCH v2 01/15] KVM: selftest: Create KVM selftest runner Vipin Sharma
2025-06-06 23:56   ` Vipin Sharma
2025-07-10  0:20   ` Sean Christopherson
2025-07-10  0:20     ` Sean Christopherson
2025-06-06 23:56 ` [PATCH v2 02/15] KVM: selftests: Enable selftests runner to find executables in different path Vipin Sharma
2025-06-06 23:56   ` Vipin Sharma
2025-07-09 21:39   ` Sean Christopherson
2025-07-09 21:39     ` Sean Christopherson
2025-06-06 23:56 ` [PATCH v2 03/15] KVM: selftests: Add timeout option in selftests runner Vipin Sharma
2025-06-06 23:56   ` Vipin Sharma
2025-07-09 21:46   ` Sean Christopherson [this message]
2025-07-09 21:46     ` Sean Christopherson
2025-07-09 22:25   ` Sean Christopherson
2025-07-09 22:25     ` Sean Christopherson
2025-06-06 23:56 ` [PATCH v2 04/15] KVM: selftests: Add option to save selftest runner output to a directory Vipin Sharma
2025-06-06 23:56   ` Vipin Sharma
2025-07-09 21:52   ` Sean Christopherson
2025-07-09 21:52     ` Sean Christopherson
2025-06-06 23:56 ` [PATCH v2 05/15] KVM: selftests: Run tests concurrently in KVM selftests runner Vipin Sharma
2025-06-06 23:56   ` Vipin Sharma
2025-06-06 23:56 ` [PATCH v2 06/15] KVM: selftests: Add a flag to print only test status in KVM Selftests run Vipin Sharma
2025-06-06 23:56   ` Vipin Sharma
2025-07-09 21:55   ` Sean Christopherson
2025-07-09 21:55     ` Sean Christopherson
2025-06-06 23:56 ` [PATCH v2 07/15] KVM: selftests: Add various print flags to KVM Selftest Runner Vipin Sharma
2025-06-06 23:56   ` Vipin Sharma
2025-07-09 22:01   ` Sean Christopherson
2025-07-09 22:01     ` Sean Christopherson
2025-06-06 23:56 ` [PATCH v2 08/15] KVM: selftests: Print sticky KVM Selftests Runner status at bottom Vipin Sharma
2025-06-06 23:56   ` Vipin Sharma
2025-06-06 23:56 ` [PATCH v2 09/15] KVM: selftests: Add a flag to print only sticky summary in the selftests runner Vipin Sharma
2025-06-06 23:56   ` Vipin Sharma
2025-06-06 23:56 ` [PATCH v2 10/15] KVM: selftests: Add flag to suppress all output from Selftest KVM Runner Vipin Sharma
2025-06-06 23:56   ` Vipin Sharma
2025-06-06 23:56 ` [PATCH v2 11/15] KVM: selftests: Auto generate default tests for KVM Selftests Runner Vipin Sharma
2025-06-06 23:56   ` Vipin Sharma
2025-07-09 23:06   ` Oliver Upton
2025-07-09 23:06     ` Oliver Upton
2025-07-10  0:18     ` Sean Christopherson
2025-07-10  0:18       ` Sean Christopherson
2025-06-06 23:56 ` [PATCH v2 12/15] KVM: selftests: Add x86 auto generated test files " Vipin Sharma
2025-06-06 23:56   ` Vipin Sharma
2025-06-06 23:56 ` [PATCH v2 13/15] KVM: selftests: Add arm64 " Vipin Sharma
2025-06-06 23:56   ` Vipin Sharma
2025-06-06 23:56 ` [PATCH v2 14/15] KVM: selftests: Add s390 " Vipin Sharma
2025-06-06 23:56   ` Vipin Sharma
2025-06-06 23:56 ` [PATCH v2 15/15] KVM: selftests: Add riscv " Vipin Sharma
2025-06-06 23:56   ` Vipin Sharma
2025-06-09 12:54   ` Andrew Jones
2025-06-09 12:54     ` Andrew Jones
2025-07-09 22:25 ` [PATCH v2 00/15] Add KVM Selftests runner Sean Christopherson
2025-07-09 22:25   ` Sean Christopherson

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=aG7jLBAqo1F3SjCl@google.com \
    --to=seanjc@google.com \
    --cc=anup@brainfault.org \
    --cc=borntraeger@linux.ibm.com \
    --cc=dmatlack@google.com \
    --cc=frankja@linux.ibm.com \
    --cc=imbrenda@linux.ibm.com \
    --cc=kvm-riscv@lists.infradead.org \
    --cc=kvm@vger.kernel.org \
    --cc=kvmarm@lists.linux.dev \
    --cc=linux-arm-kernel@lists.infradead.org \
    --cc=maz@kernel.org \
    --cc=oliver.upton@linux.dev \
    --cc=pbonzini@redhat.com \
    --cc=vipinsh@google.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.