[PATCH] check: use /var/tmp instead of /tmp

[PATCH] check: use /var/tmp instead of /tmp

[Leah Rumancik]

With the results recorded in /tmp, a crashing test results in a loss
of all prior test runs. Instead, record results in /var/tmp so that
it is possible to recover results from completed tests.

Also, make sure tmp.arglist is wiped before each run to avoid
accidentally rerunning tests.

Signed-off-by: Leah Rumancik <leah.rumancik@gmail.com>
---
 check | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/check b/check
index d2e51296..21c00524 100755
--- a/check
+++ b/check
@@ -4,7 +4,7 @@
 #
 # Control script for QA
 #
-tmp=/tmp/$$
+tmp=/var/tmp/$$
 status=0
 needwrap=true
 needsum=true
@@ -46,7 +46,7 @@ export DIFF_LENGTH=${DIFF_LENGTH:=10}
 # by default don't output timestamps
 timestamp=${TIMESTAMP:=false}
 
-rm -f $tmp.list $tmp.tmp $tmp.grep $here/$iam.out $tmp.xlist $tmp.report.*
+rm -f $tmp.list $tmp.tmp $tmp.grep $here/$iam.out $tmp.xlist $tmp.report.* $tmp.arglist
 
 SRC_GROUPS="generic shared"
 export SRC_DIR="tests"
-- 
2.39.0.rc1.256.g54fd8350bd-goog

Re: [PATCH] check: use /var/tmp instead of /tmp

[Darrick J. Wong]

On Mon, Dec 12, 2022 at 03:06:46PM -0800, Leah Rumancik wrote:
> With the results recorded in /tmp, a crashing test results in a loss
> of all prior test runs. Instead, record results in /var/tmp so that
> it is possible to recover results from completed tests.

RESULT_BASE=/some/stable/storage ./check <args> ?

> Also, make sure tmp.arglist is wiped before each run to avoid
> accidentally rerunning tests.
> 
> Signed-off-by: Leah Rumancik <leah.rumancik@gmail.com>
> ---
>  check | 4 ++--
>  1 file changed, 2 insertions(+), 2 deletions(-)
> 
> diff --git a/check b/check
> index d2e51296..21c00524 100755
> --- a/check
> +++ b/check
> @@ -4,7 +4,7 @@
>  #
>  # Control script for QA
>  #
> -tmp=/tmp/$$
> +tmp=/var/tmp/$$
>  status=0
>  needwrap=true
>  needsum=true
> @@ -46,7 +46,7 @@ export DIFF_LENGTH=${DIFF_LENGTH:=10}
>  # by default don't output timestamps
>  timestamp=${TIMESTAMP:=false}
>  
> -rm -f $tmp.list $tmp.tmp $tmp.grep $here/$iam.out $tmp.xlist $tmp.report.*
> +rm -f $tmp.list $tmp.tmp $tmp.grep $here/$iam.out $tmp.xlist $tmp.report.* $tmp.arglist

This part looks correct though.

--D

>  SRC_GROUPS="generic shared"
>  export SRC_DIR="tests"
> -- 
> 2.39.0.rc1.256.g54fd8350bd-goog
> 

Re: [PATCH] check: use /var/tmp instead of /tmp

[Leah Rumancik]

On Mon, Dec 12, 2022 at 03:18:49PM -0800, Darrick J. Wong wrote:
> On Mon, Dec 12, 2022 at 03:06:46PM -0800, Leah Rumancik wrote:
> > With the results recorded in /tmp, a crashing test results in a loss
> > of all prior test runs. Instead, record results in /var/tmp so that
> > it is possible to recover results from completed tests.
> 
> RESULT_BASE=/some/stable/storage ./check <args> ?

So I am interested in the passing test results in addition to the
failed/skipped tests. I thought just the non-passing tests stored
artifacts here, is there some way to also tell which tests passed for
tests that don't write to .full? I was hoping to just grab the xml
file that is being generated by _make_testcase_report as it also
has the test runtimes which would be nice to have.

> 
> > Also, make sure tmp.arglist is wiped before each run to avoid
> > accidentally rerunning tests.
> > 
> > Signed-off-by: Leah Rumancik <leah.rumancik@gmail.com>
> > ---
> >  check | 4 ++--
> >  1 file changed, 2 insertions(+), 2 deletions(-)
> > 
> > diff --git a/check b/check
> > index d2e51296..21c00524 100755
> > --- a/check
> > +++ b/check
> > @@ -4,7 +4,7 @@
> >  #
> >  # Control script for QA
> >  #
> > -tmp=/tmp/$$
> > +tmp=/var/tmp/$$
> >  status=0
> >  needwrap=true
> >  needsum=true
> > @@ -46,7 +46,7 @@ export DIFF_LENGTH=${DIFF_LENGTH:=10}
> >  # by default don't output timestamps
> >  timestamp=${TIMESTAMP:=false}
> >  
> > -rm -f $tmp.list $tmp.tmp $tmp.grep $here/$iam.out $tmp.xlist $tmp.report.*
> > +rm -f $tmp.list $tmp.tmp $tmp.grep $here/$iam.out $tmp.xlist $tmp.report.* $tmp.arglist
> 
> This part looks correct though.
> 
> --D

Thanks!
Leah

> >  SRC_GROUPS="generic shared"
> >  export SRC_DIR="tests"
> > -- 
> > 2.39.0.rc1.256.g54fd8350bd-goog
> > 

Re: [PATCH] check: use /var/tmp instead of /tmp

[Darrick J. Wong]

On Mon, Dec 12, 2022 at 05:56:54PM -0800, Leah Rumancik wrote:
> On Mon, Dec 12, 2022 at 03:18:49PM -0800, Darrick J. Wong wrote:
> > On Mon, Dec 12, 2022 at 03:06:46PM -0800, Leah Rumancik wrote:
> > > With the results recorded in /tmp, a crashing test results in a loss
> > > of all prior test runs. Instead, record results in /var/tmp so that
> > > it is possible to recover results from completed tests.
> > 
> > RESULT_BASE=/some/stable/storage ./check <args> ?
> 
> So I am interested in the passing test results in addition to the
> failed/skipped tests. I thought just the non-passing tests stored
> artifacts here, is there some way to also tell which tests passed for
> tests that don't write to .full? I was hoping to just grab the xml
> file that is being generated by _make_testcase_report as it also
> has the test runtimes which would be nice to have.

Oh, right, I forgot that the reporting only runs at the end of the
section, which means that (short of invading the fstests internal state)
there's no means to figure out where fstests was when it died, even if
you pointed RESULT_BASE at some nfs mount or something.

Does this help?

--D

From: Darrick J. Wong <djwong@kernel.org>
Subject: [PATCH] check: generate section reports between tests

Generate the section report between tests so that the summary report
always reflects the outcome of the most recent test.  This is useful for
fstests status reporting, for people who anxiously await results.

Signed-off-by: Darrick J. Wong <djwong@kernel.org>
---
 check |   10 ++++++++++
 1 file changed, 10 insertions(+)

diff --git a/check b/check
index d2e5129620..997547247f 100755
--- a/check
+++ b/check
@@ -841,6 +841,16 @@ function run_section()
 		fi
 		seqres="$REPORT_DIR/$seqnum"
 
+		# Generate the entire section report with whatever test results
+		# we have so far.  Leave the $sect_time parameter empty so that
+		# it's a little more obvious that this test run is incomplete.
+		if $do_report; then
+			local sect_now=`_wallclock`
+			_make_section_report "$section" "${#try[*]}" \
+					     "${#bad[*]}" "${#notrun[*]}" \
+					     "" &> /dev/null
+		fi
+
 		mkdir -p $RESULT_DIR
 		rm -f ${RESULT_DIR}/require_scratch*
 		rm -f ${RESULT_DIR}/require_test*

Re: [PATCH] check: use /var/tmp instead of /tmp

[Leah Rumancik]

On Mon, Dec 12, 2022 at 06:53:27PM -0800, Darrick J. Wong wrote:
> 
> Does this help?

This should work for me! I'll go ahead and resend just the rm
tmp.arglist change.

Thanks,
Leah

> 
> --D
> 
> From: Darrick J. Wong <djwong@kernel.org>
> Subject: [PATCH] check: generate section reports between tests
> 
> Generate the section report between tests so that the summary report
> always reflects the outcome of the most recent test.  This is useful for
> fstests status reporting, for people who anxiously await results.
> 
> Signed-off-by: Darrick J. Wong <djwong@kernel.org>
> ---
>  check |   10 ++++++++++
>  1 file changed, 10 insertions(+)
> 
> diff --git a/check b/check
> index d2e5129620..997547247f 100755
> --- a/check
> +++ b/check
> @@ -841,6 +841,16 @@ function run_section()
>  		fi
>  		seqres="$REPORT_DIR/$seqnum"
>  
> +		# Generate the entire section report with whatever test results
> +		# we have so far.  Leave the $sect_time parameter empty so that
> +		# it's a little more obvious that this test run is incomplete.
> +		if $do_report; then
> +			local sect_now=`_wallclock`
> +			_make_section_report "$section" "${#try[*]}" \
> +					     "${#bad[*]}" "${#notrun[*]}" \
> +					     "" &> /dev/null
> +		fi
> +
>  		mkdir -p $RESULT_DIR
>  		rm -f ${RESULT_DIR}/require_scratch*
>  		rm -f ${RESULT_DIR}/require_test*