[kvm-unit-tests PATCH] runtime: Skip tests if the target "kernel" file doesn't exist

[kvm-unit-tests PATCH] runtime: Skip tests if the target "kernel" file doesn't exist

[Sean Christopherson]

Skip the test if its target kernel/test file isn't available so that
skipping a test that isn't supported for a given config doesn't require
manually flagging the testcase in unittests.cfg.  This fixes "failures"
on x86 with CONFIG_EFI=y due to some tests not being built for EFI, but
not being annotated in x86/unittests.cfg.

Alternatively, testcases could be marked noefi (or efi-only), but that'd
require more manual effort, and there's no obvious advantage to doing so.

Signed-off-by: Sean Christopherson <seanjc@google.com>
---
 scripts/runtime.bash | 5 +++++
 1 file changed, 5 insertions(+)

diff --git a/scripts/runtime.bash b/scripts/runtime.bash
index ee229631..a94d940d 100644
--- a/scripts/runtime.bash
+++ b/scripts/runtime.bash
@@ -150,6 +150,11 @@ function run()
         done
     fi
 
+    if [ ! -f "$kernel" ]; then
+        print_result "SKIP" $testname "" "Test file '$kernel' not found";
+        return 2;
+    fi
+
     log=$(premature_failure) && {
         skip=true
         if [ "${CONFIG_EFI}" == "y" ]; then

base-commit: 72d110d8286baf1b355301cc8c8bdb42be2663fb
-- 
2.49.0.1204.g71687c7c1d-goog

Re: [kvm-unit-tests PATCH] runtime: Skip tests if the target "kernel" file doesn't exist

[Andrew Jones]

On Thu, May 29, 2025 at 01:58:20PM -0700, Sean Christopherson wrote:
> Skip the test if its target kernel/test file isn't available so that
> skipping a test that isn't supported for a given config doesn't require
> manually flagging the testcase in unittests.cfg.  This fixes "failures"
> on x86 with CONFIG_EFI=y due to some tests not being built for EFI, but
> not being annotated in x86/unittests.cfg.
> 
> Alternatively, testcases could be marked noefi (or efi-only), but that'd
> require more manual effort, and there's no obvious advantage to doing so.
> 
> Signed-off-by: Sean Christopherson <seanjc@google.com>
> ---
>  scripts/runtime.bash | 5 +++++
>  1 file changed, 5 insertions(+)
> 
> diff --git a/scripts/runtime.bash b/scripts/runtime.bash
> index ee229631..a94d940d 100644
> --- a/scripts/runtime.bash
> +++ b/scripts/runtime.bash
> @@ -150,6 +150,11 @@ function run()
>          done
>      fi
>  
> +    if [ ! -f "$kernel" ]; then
> +        print_result "SKIP" $testname "" "Test file '$kernel' not found";
> +        return 2;
> +    fi
> +

I see mkstandalone.sh already has something like this. There's still one
other place, though, which is print_testname(). Should we filter tests
from the listing that are missing their kernels?

Thanks,
drew

>      log=$(premature_failure) && {
>          skip=true
>          if [ "${CONFIG_EFI}" == "y" ]; then
> 
> base-commit: 72d110d8286baf1b355301cc8c8bdb42be2663fb
> -- 
> 2.49.0.1204.g71687c7c1d-goog
> 

Re: [kvm-unit-tests PATCH] runtime: Skip tests if the target "kernel" file doesn't exist

[Sean Christopherson]

On Fri, May 30, 2025, Andrew Jones wrote:
> On Thu, May 29, 2025 at 01:58:20PM -0700, Sean Christopherson wrote:
> > Skip the test if its target kernel/test file isn't available so that
> > skipping a test that isn't supported for a given config doesn't require
> > manually flagging the testcase in unittests.cfg.  This fixes "failures"
> > on x86 with CONFIG_EFI=y due to some tests not being built for EFI, but
> > not being annotated in x86/unittests.cfg.
> > 
> > Alternatively, testcases could be marked noefi (or efi-only), but that'd
> > require more manual effort, and there's no obvious advantage to doing so.
> > 
> > Signed-off-by: Sean Christopherson <seanjc@google.com>
> > ---
> >  scripts/runtime.bash | 5 +++++
> >  1 file changed, 5 insertions(+)
> > 
> > diff --git a/scripts/runtime.bash b/scripts/runtime.bash
> > index ee229631..a94d940d 100644
> > --- a/scripts/runtime.bash
> > +++ b/scripts/runtime.bash
> > @@ -150,6 +150,11 @@ function run()
> >          done
> >      fi
> >  
> > +    if [ ! -f "$kernel" ]; then
> > +        print_result "SKIP" $testname "" "Test file '$kernel' not found";
> > +        return 2;
> > +    fi
> > +
> 
> I see mkstandalone.sh already has something like this. There's still one
> other place, though, which is print_testname(). Should we filter tests
> from the listing that are missing their kernels?

Huh, TIL you can list testcases :-)

I would say no?  Because then listing testcases would depend on a successful
build, which would be annoying in a variety of scenarios.

It would also be weird to list testcases that are excluded based on e.g. arch,
but not list testcases that are effectively excluded via Makefile.

Re: [kvm-unit-tests PATCH] runtime: Skip tests if the target "kernel" file doesn't exist

[Andrew Jones]

On Fri, May 30, 2025 at 08:27:33AM -0700, Sean Christopherson wrote:
> On Fri, May 30, 2025, Andrew Jones wrote:
> > On Thu, May 29, 2025 at 01:58:20PM -0700, Sean Christopherson wrote:
> > > Skip the test if its target kernel/test file isn't available so that
> > > skipping a test that isn't supported for a given config doesn't require
> > > manually flagging the testcase in unittests.cfg.  This fixes "failures"
> > > on x86 with CONFIG_EFI=y due to some tests not being built for EFI, but
> > > not being annotated in x86/unittests.cfg.
> > > 
> > > Alternatively, testcases could be marked noefi (or efi-only), but that'd
> > > require more manual effort, and there's no obvious advantage to doing so.
> > > 
> > > Signed-off-by: Sean Christopherson <seanjc@google.com>
> > > ---
> > >  scripts/runtime.bash | 5 +++++
> > >  1 file changed, 5 insertions(+)
> > > 
> > > diff --git a/scripts/runtime.bash b/scripts/runtime.bash
> > > index ee229631..a94d940d 100644
> > > --- a/scripts/runtime.bash
> > > +++ b/scripts/runtime.bash
> > > @@ -150,6 +150,11 @@ function run()
> > >          done
> > >      fi
> > >  
> > > +    if [ ! -f "$kernel" ]; then
> > > +        print_result "SKIP" $testname "" "Test file '$kernel' not found";
> > > +        return 2;
> > > +    fi
> > > +
> > 
> > I see mkstandalone.sh already has something like this. There's still one
> > other place, though, which is print_testname(). Should we filter tests
> > from the listing that are missing their kernels?
> 
> Huh, TIL you can list testcases :-)
> 
> I would say no?  Because then listing testcases would depend on a successful
> build, which would be annoying in a variety of scenarios.
> 
> It would also be weird to list testcases that are excluded based on e.g. arch,
> but not list testcases that are effectively excluded via Makefile.

Sounds reasonable.

Reviewed-by: Andrew Jones <andrew.jones@linux.dev>

Thanks,
drew

Re: [kvm-unit-tests PATCH] runtime: Skip tests if the target "kernel" file doesn't exist

[Sean Christopherson]

On Thu, 29 May 2025 13:58:20 -0700, Sean Christopherson wrote:
> Skip the test if its target kernel/test file isn't available so that
> skipping a test that isn't supported for a given config doesn't require
> manually flagging the testcase in unittests.cfg.  This fixes "failures"
> on x86 with CONFIG_EFI=y due to some tests not being built for EFI, but
> not being annotated in x86/unittests.cfg.
> 
> Alternatively, testcases could be marked noefi (or efi-only), but that'd
> require more manual effort, and there's no obvious advantage to doing so.
> 
> [...]

Applied to kvm-x86 next, thanks!

[1/1] runtime: Skip tests if the target "kernel" file doesn't exist
      https://github.com/kvm-x86/kvm-unit-tests/commit/7f528c1b474c

--
https://github.com/kvm-x86/kvm-unit-tests/tree/next