[kvm-unit-tests RFC] Add ability to specify groups which won't be run by default

[kvm-unit-tests RFC] Add ability to specify groups which won't be run by default

[Suraj Jitindar Singh]

Hi,

I'm looking at adding a powerpc test to kvm-unit-tests which has a high
potential of bringing down the host.

Thus I would like to be able to specify that it not be run by default when
all tests are being run, that is the user has to specifically run that test
on its own to ensure that they are sure that they want to run it.

The way I was thinking of doing this is by having some prefix for the
groups parameter in the unittests.cfg file, for example an underscore,
which specifies that this test group not be run when no groups parameter is
passed to run_tests.sh (i.e. all tests are being run). That is the user
will have to specifically specify that individual test group at invocation
to have it run.

Would you be open to having me add this to the generic test infrastructure?

I am open to comments or any other way you would prefer this be done. This
just seemed like the easiest option to me.

Example of patch below using underscore prefix as condition.
---
 scripts/runtime.bash | 6 ++++++
 1 file changed, 6 insertions(+)

diff --git a/scripts/runtime.bash b/scripts/runtime.bash
index 0503cf0..fbd415c 100644
--- a/scripts/runtime.bash
+++ b/scripts/runtime.bash
@@ -52,6 +52,12 @@ function run()
         return
     fi
 
+    if grep -q "^_" <<<$groups && ( [ -z "$only_group" ] || ( [ -n \
+			    "$only_group" ] && ! grep -qw "$only_group" \
+			    <<<$groups) ); then
+	    return;
+    fi
+
     if [ -n "$arch" ] && [ "$arch" != "$ARCH" ]; then
         echo "`SKIP` $1 ($arch only)"
         return 2
-- 
2.5.5

Re: [kvm-unit-tests RFC] Add ability to specify groups which won't be run by default

[Andrew Jones]

On Thu, Aug 04, 2016 at 11:58:27AM +1000, Suraj Jitindar Singh wrote:
> Hi,
> 
> I'm looking at adding a powerpc test to kvm-unit-tests which has a high
> potential of bringing down the host.
> 
> Thus I would like to be able to specify that it not be run by default when
> all tests are being run, that is the user has to specifically run that test
> on its own to ensure that they are sure that they want to run it.
> 
> The way I was thinking of doing this is by having some prefix for the
> groups parameter in the unittests.cfg file, for example an underscore,
> which specifies that this test group not be run when no groups parameter is
> passed to run_tests.sh (i.e. all tests are being run). That is the user
> will have to specifically specify that individual test group at invocation
> to have it run.
> 
> Would you be open to having me add this to the generic test infrastructure?

Sounds good to me.

Thanks,
drew

Re: [kvm-unit-tests RFC] Add ability to specify groups which won't be run by default

[Paolo Bonzini]

On 04/08/2016 03:58, Suraj Jitindar Singh wrote:
> Hi,
> 
> I'm looking at adding a powerpc test to kvm-unit-tests which has a high
> potential of bringing down the host.
> 
> Thus I would like to be able to specify that it not be run by default when
> all tests are being run, that is the user has to specifically run that test
> on its own to ensure that they are sure that they want to run it.
> 
> The way I was thinking of doing this is by having some prefix for the
> groups parameter in the unittests.cfg file, for example an underscore,
> which specifies that this test group not be run when no groups parameter is
> passed to run_tests.sh (i.e. all tests are being run). That is the user
> will have to specifically specify that individual test group at invocation
> to have it run.

You can make a group "nodefault", and if this group is specified you
require that "$only_group" is non-empty in scripts/runtime.bash's run
function.

Paolo

> Would you be open to having me add this to the generic test infrastructure?
> 
> I am open to comments or any other way you would prefer this be done. This
> just seemed like the easiest option to me.
> 
> Example of patch below using underscore prefix as condition.
> ---
>  scripts/runtime.bash | 6 ++++++
>  1 file changed, 6 insertions(+)
> 
> diff --git a/scripts/runtime.bash b/scripts/runtime.bash
> index 0503cf0..fbd415c 100644
> --- a/scripts/runtime.bash
> +++ b/scripts/runtime.bash
> @@ -52,6 +52,12 @@ function run()
>          return
>      fi
>  
> +    if grep -q "^_" <<<$groups && ( [ -z "$only_group" ] || ( [ -n \
> +			    "$only_group" ] && ! grep -qw "$only_group" \
> +			    <<<$groups) ); then
> +	    return;
> +    fi
> +
>      if [ -n "$arch" ] && [ "$arch" != "$ARCH" ]; then
>          echo "`SKIP` $1 ($arch only)"
>          return 2
> 

Re: [kvm-unit-tests RFC] Add ability to specify groups which won't be run by default

[Suraj Jitindar Singh]

On 04/08/16 22:35, Paolo Bonzini wrote:
>
> On 04/08/2016 03:58, Suraj Jitindar Singh wrote:
>> Hi,
>>
>> I'm looking at adding a powerpc test to kvm-unit-tests which has a high
>> potential of bringing down the host.
>>
>> Thus I would like to be able to specify that it not be run by default when
>> all tests are being run, that is the user has to specifically run that test
>> on its own to ensure that they are sure that they want to run it.
>>
>> The way I was thinking of doing this is by having some prefix for the
>> groups parameter in the unittests.cfg file, for example an underscore,
>> which specifies that this test group not be run when no groups parameter is
>> passed to run_tests.sh (i.e. all tests are being run). That is the user
>> will have to specifically specify that individual test group at invocation
>> to have it run.
> You can make a group "nodefault", and if this group is specified you
> require that "$only_group" is non-empty in scripts/runtime.bash's run
> function.
>
> Paolo

This sounds like a good option, I'll go with that.

>
>> Would you be open to having me add this to the generic test infrastructure?
>>
>> I am open to comments or any other way you would prefer this be done. This
>> just seemed like the easiest option to me.
>>
>> Example of patch below using underscore prefix as condition.
>> ---
>>  scripts/runtime.bash | 6 ++++++
>>  1 file changed, 6 insertions(+)
>>
>> diff --git a/scripts/runtime.bash b/scripts/runtime.bash
>> index 0503cf0..fbd415c 100644
>> --- a/scripts/runtime.bash
>> +++ b/scripts/runtime.bash
>> @@ -52,6 +52,12 @@ function run()
>>          return
>>      fi
>>  
>> +    if grep -q "^_" <<<$groups && ( [ -z "$only_group" ] || ( [ -n \
>> +			    "$only_group" ] && ! grep -qw "$only_group" \
>> +			    <<<$groups) ); then
>> +	    return;
>> +    fi
>> +
>>      if [ -n "$arch" ] && [ "$arch" != "$ARCH" ]; then
>>          echo "`SKIP` $1 ($arch only)"
>>          return 2
>>