[PATCH, RFC] xfstests: fail test if it causes a kernel taint

[PATCH, RFC] xfstests: fail test if it causes a kernel taint

[Eric Sandeen]

This should, in theory, fail a test if it introduces one of
a handful of "serious" kernel taints.  I mask on a few taint
values because using an out of tree module or a non-GPL module
should never fail a test, for example.

Signed-off-by: Eric Sandeen <sandeen@redhat.com>
---

diff --git a/check b/check
index a79747e..a9cac4e 100755
--- a/check
+++ b/check
@@ -446,6 +445,11 @@ do
 	fi
 	rm -f core $seqres.notrun
 
+	if [ "$HOSTOS" == "Linux" ]; then
+		tainted=`sysctl -n kernel.tainted`
+		let "tainted &= $TAINT_FAIL"
+	fi
+
 	start=`_wallclock`
 	$timestamp && echo -n "	["`date "+%T"`"]"
 	[ ! -x $seq ] && chmod u+x $seq # ensure we can run it
@@ -507,6 +511,19 @@ do
 			 "entire diff)"
 		    err=true
 		fi
+
+		# See if this run tainted the kernel due to oops, etc
+		if [ "$HOSTOS" == "Linux" ]
+		then
+			tainted2=`sysctl -n kernel.tainted`
+			let "tainted2 &= $TAINT_FAIL"
+			if [ "$tainted" != "$tainted2" ]
+			then
+				echo " Kernel taint changed from $tainted to $tainted2."
+				echo " See dmesg for details."
+				err=true
+			fi
+		fi
 	    fi
 	fi
 
diff --git a/common/config b/common/config
index 67c1498..6483ded 100644
--- a/common/config
+++ b/common/config
@@ -82,6 +82,15 @@ export PWD=`pwd`
 export MALLOCLIB=${MALLOCLIB:=/usr/lib/libefence.a}
 export LOCAL_CONFIGURE_OPTIONS=${LOCAL_CONFIGURE_OPTIONS:=--enable-readline=yes}
 
+# kernel taint values on Linux (from kernel.taint sysctl)
+TAINT_MACHINE_CHECK=16
+TAINT_BAD_PAGE=32
+TAINT_DIED=128
+TAINT_WARNING=512
+# If a test introduces one of these taints we consider it to have failed
+let "TAINT_FAIL=( $TAINT_MACHINE_CHECK | $TAINT_BAD_PAGE | $TAINT_DIED | $TAINT_WARNING )"
+export TAINT_FAIL
+
 # $1 = prog to look for, $2* = default pathnames if not found in $PATH
 set_prog_path()
 {

_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs

Re: [PATCH, RFC] xfstests: fail test if it causes a kernel taint

[Dave Chinner]

On Mon, May 13, 2013 at 04:55:18PM -0500, Eric Sandeen wrote:
> This should, in theory, fail a test if it introduces one of
> a handful of "serious" kernel taints.  I mask on a few taint
> values because using an out of tree module or a non-GPL module
> should never fail a test, for example.
> 
> Signed-off-by: Eric Sandeen <sandeen@redhat.com>

I like the idea - it should catch lockdep failures and other such
problems.

> ---
> 
> diff --git a/check b/check
> index a79747e..a9cac4e 100755
> --- a/check
> +++ b/check
> @@ -446,6 +445,11 @@ do
>  	fi
>  	rm -f core $seqres.notrun
>  
> +	if [ "$HOSTOS" == "Linux" ]; then
> +		tainted=`sysctl -n kernel.tainted`
> +		let "tainted &= $TAINT_FAIL"
> +	fi
> +
>  	start=`_wallclock`
>  	$timestamp && echo -n "	["`date "+%T"`"]"
>  	[ ! -x $seq ] && chmod u+x $seq # ensure we can run it
> @@ -507,6 +511,19 @@ do
>  			 "entire diff)"
>  		    err=true
>  		fi
> +
> +		# See if this run tainted the kernel due to oops, etc
> +		if [ "$HOSTOS" == "Linux" ]
> +		then
> +			tainted2=`sysctl -n kernel.tainted`
> +			let "tainted2 &= $TAINT_FAIL"
> +			if [ "$tainted" != "$tainted2" ]
> +			then
> +				echo " Kernel taint changed from $tainted to $tainted2."
> +				echo " See dmesg for details."
> +				err=true
> +			fi

Should we dump the last 100 lines of dmesg into the output file
here? And then rely on the golden image match failing to fail the
test?

Cheers,

Dave.
-- 
Dave Chinner
david@fromorbit.com

_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs

Re: [PATCH, RFC] xfstests: fail test if it causes a kernel taint

[Eric Sandeen]

On 5/13/13 8:15 PM, Dave Chinner wrote:
> On Mon, May 13, 2013 at 04:55:18PM -0500, Eric Sandeen wrote:
>> This should, in theory, fail a test if it introduces one of
>> a handful of "serious" kernel taints.  I mask on a few taint
>> values because using an out of tree module or a non-GPL module
>> should never fail a test, for example.
>>
>> Signed-off-by: Eric Sandeen <sandeen@redhat.com>
> 
> I like the idea - it should catch lockdep failures and other such
> problems.
> 
>> ---
>>
>> diff --git a/check b/check
>> index a79747e..a9cac4e 100755
>> --- a/check
>> +++ b/check
>> @@ -446,6 +445,11 @@ do
>>  	fi
>>  	rm -f core $seqres.notrun
>>  
>> +	if [ "$HOSTOS" == "Linux" ]; then
>> +		tainted=`sysctl -n kernel.tainted`
>> +		let "tainted &= $TAINT_FAIL"
>> +	fi
>> +
>>  	start=`_wallclock`
>>  	$timestamp && echo -n "	["`date "+%T"`"]"
>>  	[ ! -x $seq ] && chmod u+x $seq # ensure we can run it
>> @@ -507,6 +511,19 @@ do
>>  			 "entire diff)"
>>  		    err=true
>>  		fi
>> +
>> +		# See if this run tainted the kernel due to oops, etc
>> +		if [ "$HOSTOS" == "Linux" ]
>> +		then
>> +			tainted2=`sysctl -n kernel.tainted`
>> +			let "tainted2 &= $TAINT_FAIL"
>> +			if [ "$tainted" != "$tainted2" ]
>> +			then
>> +				echo " Kernel taint changed from $tainted to $tainted2."
>> +				echo " See dmesg for details."
>> +				err=true
>> +			fi
> 
> Should we dump the last 100 lines of dmesg into the output file
> here? And then rely on the golden image match failing to fail the
> test?

Or into $seq.full . .  I dunno.  It's hard to know how much dmesg you'd need to get the right bits.

Dumping all of it into $seq.full might be best; it only trips when it's a new taint, so won't
fire for every test.

-Eric

> Cheers,
> 
> Dave.
> 

_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs

Re: [PATCH, RFC] xfstests: fail test if it causes a kernel taint

[Dave Chinner]

On Mon, May 13, 2013 at 09:00:45PM -0500, Eric Sandeen wrote:
> On 5/13/13 8:15 PM, Dave Chinner wrote:
> > On Mon, May 13, 2013 at 04:55:18PM -0500, Eric Sandeen wrote:
> >> This should, in theory, fail a test if it introduces one of
> >> a handful of "serious" kernel taints.  I mask on a few taint
> >> values because using an out of tree module or a non-GPL module
> >> should never fail a test, for example.
> >>
> >> Signed-off-by: Eric Sandeen <sandeen@redhat.com>
> > 
> > I like the idea - it should catch lockdep failures and other such
> > problems.
> > 
> >> ---
> >>
> >> diff --git a/check b/check
> >> index a79747e..a9cac4e 100755
> >> --- a/check
> >> +++ b/check
> >> @@ -446,6 +445,11 @@ do
> >>  	fi
> >>  	rm -f core $seqres.notrun
> >>  
> >> +	if [ "$HOSTOS" == "Linux" ]; then
> >> +		tainted=`sysctl -n kernel.tainted`
> >> +		let "tainted &= $TAINT_FAIL"
> >> +	fi
> >> +
> >>  	start=`_wallclock`
> >>  	$timestamp && echo -n "	["`date "+%T"`"]"
> >>  	[ ! -x $seq ] && chmod u+x $seq # ensure we can run it
> >> @@ -507,6 +511,19 @@ do
> >>  			 "entire diff)"
> >>  		    err=true
> >>  		fi
> >> +
> >> +		# See if this run tainted the kernel due to oops, etc
> >> +		if [ "$HOSTOS" == "Linux" ]
> >> +		then
> >> +			tainted2=`sysctl -n kernel.tainted`
> >> +			let "tainted2 &= $TAINT_FAIL"
> >> +			if [ "$tainted" != "$tainted2" ]
> >> +			then
> >> +				echo " Kernel taint changed from $tainted to $tainted2."
> >> +				echo " See dmesg for details."
> >> +				err=true
> >> +			fi
> > 
> > Should we dump the last 100 lines of dmesg into the output file
> > here? And then rely on the golden image match failing to fail the
> > test?
> 
> Or into $seq.full . .  I dunno.  It's hard to know how much dmesg you'd need to get the right bits.
> 
> Dumping all of it into $seq.full might be best; it only trips when it's a new taint, so won't
> fire for every test.

Only problem with $seqres.full is that it is internal to the test
and so many tests will not remove one that is there before the test
starts. Hence you'll get growing files the never get removed from
repeated failures. Even just dumping the above error message into
the output file would be sufficient, I think, as the 10 line diff
output will state it pretty clearly as the first difference....

Cheers,

Dave.
-- 
Dave Chinner
david@fromorbit.com

_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs

Re: [PATCH, RFC] xfstests: fail test if it causes a kernel taint

[Eric Sandeen]

On 5/13/13 10:15 PM, Dave Chinner wrote:
> On Mon, May 13, 2013 at 09:00:45PM -0500, Eric Sandeen wrote:
>> On 5/13/13 8:15 PM, Dave Chinner wrote:
>>> On Mon, May 13, 2013 at 04:55:18PM -0500, Eric Sandeen wrote:
>>>> This should, in theory, fail a test if it introduces one of
>>>> a handful of "serious" kernel taints.  I mask on a few taint
>>>> values because using an out of tree module or a non-GPL module
>>>> should never fail a test, for example.
>>>>
>>>> Signed-off-by: Eric Sandeen <sandeen@redhat.com>
>>>
>>> I like the idea - it should catch lockdep failures and other such
>>> problems.
>>>
>>>> ---
>>>>
>>>> diff --git a/check b/check
>>>> index a79747e..a9cac4e 100755
>>>> --- a/check
>>>> +++ b/check
>>>> @@ -446,6 +445,11 @@ do
>>>>  	fi
>>>>  	rm -f core $seqres.notrun
>>>>  
>>>> +	if [ "$HOSTOS" == "Linux" ]; then
>>>> +		tainted=`sysctl -n kernel.tainted`
>>>> +		let "tainted &= $TAINT_FAIL"
>>>> +	fi
>>>> +
>>>>  	start=`_wallclock`
>>>>  	$timestamp && echo -n "	["`date "+%T"`"]"
>>>>  	[ ! -x $seq ] && chmod u+x $seq # ensure we can run it
>>>> @@ -507,6 +511,19 @@ do
>>>>  			 "entire diff)"
>>>>  		    err=true
>>>>  		fi
>>>> +
>>>> +		# See if this run tainted the kernel due to oops, etc
>>>> +		if [ "$HOSTOS" == "Linux" ]
>>>> +		then
>>>> +			tainted2=`sysctl -n kernel.tainted`
>>>> +			let "tainted2 &= $TAINT_FAIL"
>>>> +			if [ "$tainted" != "$tainted2" ]
>>>> +			then
>>>> +				echo " Kernel taint changed from $tainted to $tainted2."
>>>> +				echo " See dmesg for details."
>>>> +				err=true
>>>> +			fi
>>>
>>> Should we dump the last 100 lines of dmesg into the output file
>>> here? And then rely on the golden image match failing to fail the
>>> test?
>>
>> Or into $seq.full . .  I dunno.  It's hard to know how much dmesg you'd need to get the right bits.
>>
>> Dumping all of it into $seq.full might be best; it only trips when it's a new taint, so won't
>> fire for every test.
> 
> Only problem with $seqres.full is that it is internal to the test
> and so many tests will not remove one that is there before the test
> starts. Hence you'll get growing files the never get removed from
> repeated failures. 

Oh, I see what you mean, yeah.

> Even just dumping the above error message into
> the output file would be sufficient, I think, as the 10 line diff
> output will state it pretty clearly as the first difference....

*shrug* I don't see the advantage of failing the test explicitly vs.
breaking it via output, but if you like that better I don't really mind,
I can change it to do that.

-Eric

> Cheers,
> 
> Dave.
> 

_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs

[PATCH, V2] xfstests: fail test if it causes a kernel taint

[Eric Sandeen]

This modification will fail a test if it introduces one of
a handful of "serious" kernel taints.  I mask on a few taint
values because using an out of tree module or a non-GPL module
should never fail a test, for example.

Signed-off-by: Eric Sandeen <sandeen@redhat.com>
---

Tested by modifying tests/generic/001 to modprobe a "staging"
driver, and adding the staging taint bit (1024) to the mask.

V2: fail test by modifying the test output so the golden
output check will fail, per dchinner's suggestion.

diff --git a/check b/check
index a79747e..7ba6696 100755
--- a/check
+++ b/check
@@ -446,6 +446,11 @@ do
 	fi
 	rm -f core $seqres.notrun
 
+	if [ "$HOSTOS" == "Linux" ]; then
+		tainted=`sysctl -n kernel.tainted`
+		let "tainted &= $TAINT_FAIL"
+	fi
+
 	start=`_wallclock`
 	$timestamp && echo -n "	["`date "+%T"`"]"
 	[ ! -x $seq ] && chmod u+x $seq # ensure we can run it
@@ -458,6 +463,19 @@ do
 	_fix_malloc <$tmp.rawout >$tmp.out
 	rm -f $tmp.rawout
 
+	# See if this run tainted the kernel due to oops, etc
+	# If so, break the test via the golden output file
+	if [ "$HOSTOS" == "Linux" ]
+	then
+		tainted2=`sysctl -n kernel.tainted`
+		let "tainted2 &= $TAINT_FAIL"
+		if [ "$tainted" != "$tainted2" ]
+		then
+			echo " Kernel taint changed from $tainted to $tainted2." >> $tmp.out
+			echo " See dmesg for details." >> $tmp.out
+		fi
+	fi
+
 	if [ -f core ]
 	then
 	    echo -n " [dumped core]"
diff --git a/common/config b/common/config
index 67c1498..6483ded 100644
--- a/common/config
+++ b/common/config
@@ -82,6 +82,15 @@ export PWD=`pwd`
 export MALLOCLIB=${MALLOCLIB:=/usr/lib/libefence.a}
 export LOCAL_CONFIGURE_OPTIONS=${LOCAL_CONFIGURE_OPTIONS:=--enable-readline=yes}
 
+# kernel taint values on Linux (from kernel.taint sysctl)
+TAINT_MACHINE_CHECK=16
+TAINT_BAD_PAGE=32
+TAINT_DIED=128
+TAINT_WARNING=512
+# If a test introduces one of these taints we consider it to have failed
+let "TAINT_FAIL=( $TAINT_MACHINE_CHECK | $TAINT_BAD_PAGE | $TAINT_DIED | $TAINT_WARNING )"
+export TAINT_FAIL
+
 # $1 = prog to look for, $2* = default pathnames if not found in $PATH
 set_prog_path()
 {

_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs

Re: [PATCH, V2] xfstests: fail test if it causes a kernel taint

[Eric Sandeen]

On 5/23/13 10:47 AM, Eric Sandeen wrote:
> This modification will fail a test if it introduces one of
> a handful of "serious" kernel taints.  I mask on a few taint
> values because using an out of tree module or a non-GPL module
> should never fail a test, for example.
> 
> Signed-off-by: Eric Sandeen <sandeen@redhat.com>
> ---
> 
> Tested by modifying tests/generic/001 to modprobe a "staging"
> driver, and adding the staging taint bit (1024) to the mask.
> 
> V2: fail test by modifying the test output so the golden
> output check will fail, per dchinner's suggestion.

Ping, needs re-review for V2 I guess.

Thanks,
-Eric

> diff --git a/check b/check
> index a79747e..7ba6696 100755
> --- a/check
> +++ b/check
> @@ -446,6 +446,11 @@ do
>  	fi
>  	rm -f core $seqres.notrun
>  
> +	if [ "$HOSTOS" == "Linux" ]; then
> +		tainted=`sysctl -n kernel.tainted`
> +		let "tainted &= $TAINT_FAIL"
> +	fi
> +
>  	start=`_wallclock`
>  	$timestamp && echo -n "	["`date "+%T"`"]"
>  	[ ! -x $seq ] && chmod u+x $seq # ensure we can run it
> @@ -458,6 +463,19 @@ do
>  	_fix_malloc <$tmp.rawout >$tmp.out
>  	rm -f $tmp.rawout
>  
> +	# See if this run tainted the kernel due to oops, etc
> +	# If so, break the test via the golden output file
> +	if [ "$HOSTOS" == "Linux" ]
> +	then
> +		tainted2=`sysctl -n kernel.tainted`
> +		let "tainted2 &= $TAINT_FAIL"
> +		if [ "$tainted" != "$tainted2" ]
> +		then
> +			echo " Kernel taint changed from $tainted to $tainted2." >> $tmp.out
> +			echo " See dmesg for details." >> $tmp.out
> +		fi
> +	fi
> +
>  	if [ -f core ]
>  	then
>  	    echo -n " [dumped core]"
> diff --git a/common/config b/common/config
> index 67c1498..6483ded 100644
> --- a/common/config
> +++ b/common/config
> @@ -82,6 +82,15 @@ export PWD=`pwd`
>  export MALLOCLIB=${MALLOCLIB:=/usr/lib/libefence.a}
>  export LOCAL_CONFIGURE_OPTIONS=${LOCAL_CONFIGURE_OPTIONS:=--enable-readline=yes}
>  
> +# kernel taint values on Linux (from kernel.taint sysctl)
> +TAINT_MACHINE_CHECK=16
> +TAINT_BAD_PAGE=32
> +TAINT_DIED=128
> +TAINT_WARNING=512
> +# If a test introduces one of these taints we consider it to have failed
> +let "TAINT_FAIL=( $TAINT_MACHINE_CHECK | $TAINT_BAD_PAGE | $TAINT_DIED | $TAINT_WARNING )"
> +export TAINT_FAIL
> +
>  # $1 = prog to look for, $2* = default pathnames if not found in $PATH
>  set_prog_path()
>  {
> 
> _______________________________________________
> xfs mailing list
> xfs@oss.sgi.com
> http://oss.sgi.com/mailman/listinfo/xfs
> 

_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs