Re: [PATCH] fstests: report: always save the dmesg as system-err if KEEP_DMESG is set

["Theodore Ts'o" <tytso@mit.edu>]

On Tue, Dec 20, 2022 at 06:45:27AM +0800, Qu Wenruo wrote:
> 
> 
> On the other hand, separating dmesg from its test cases means extra parsing
> to bind the log section to certain test runs.
> 
> In fact, I'm not even sure if possible to do that.
> E.g. on the same host with different kernel/fstests configs.

Well, I just have my test runner framework drop a file which contains
a single line[2] in the /etc/rsyslog.d directory, and then kill -HUP
rsyslogd.  When I'm done, I'll reconfigure rsyslog and HUP it again,
and then tar up the results directory, and save that as part of my
test artifacts for that particular test run.

[2]   *.*;auth,authpriv.none		-/results/syslog

(This could probably be "kern.*" instead, but occasionally it's useful
to see what other kind of background noise might be running on the
test machine that might be disturbing the test run.)

The /results directory will contain a copy of the current kernel's
build config file, the arguments to the xfstests run, the xfstests
results directory with all of the NNN.full and NNN.out.bad files, and
of course, the syslog file.  I also have things like a copy of
/proc/slabinfo before and after the run; and I have support for hook
scripts that can enable/disable ftrace with various tracepoints and
which will drop the ftrace logs in the /results directory.  (I don't
have support for drgn or BPF scripts yet, but one of these days...)

Anyway, my test framework will compress the /results directory and
save it away (in my case, in Google Cloud Store).  For a full ext4
test run (twelve "-g auto" runs with different fs configs), the
compressed test artifact tarball will be about 23 megabytes.  In
contrast, the xunut-quiet XML file will be about 80k compressed, and
the test run summary which lands in my e-mail inbox is less than 4k. :-)

	      	       	  	 - Ted

P.S.  This is a recent test run on btrfs using my test environment.
I'm curious whether any of these test failures look familiar to you.
(Some of them may very well be issues in how I set up the btrfs pool
of block devices, which is a relatively new feature in gce-xfstests.
What are you seeing when you run -g auto on btrfs using the default
file system config?)

TESTRUNID: tytso-20221212170803
KERNEL:    kernel 6.1.0-xfstests #2 SMP PREEMPT_DYNAMIC Mon Dec 12 16:09:40 EST 2022 x86_64
CMDLINE:   -c btrfs/4k -g auto
CPUS:      2
MEM:       7680

btrfs/4k: 953 tests, 21 failures, 176 skipped, 12135 seconds
  Failures: btrfs/006 btrfs/012 btrfs/049 btrfs/100 btrfs/162 btrfs/163
    btrfs/184 btrfs/192 btrfs/196 btrfs/197 btrfs/218 btrfs/219 btrfs/235
    btrfs/254 btrfs/277 btrfs/291 generic/455 generic/457
  Flaky: btrfs/028: 40% (2/5)   generic/475: 60% (3/5)
    shared/298: 20% (1/5)

FSTESTIMG: gce-xfstests/xfstests-amd64-202211301035
FSTESTPRJ: gce-xfstests
FSTESTVER: blktests 068bd2a (Fri, 18 Nov 2022 08:38:35 +0900)  
FSTESTVER: fio  fio-3.31 (Tue, 9 Aug 2022 14:41:25 -0600)
FSTESTVER: fsverity v1.5 (Sun, 6 Feb 2022 10:59:13 -0800)
FSTESTVER: ima-evm-utils v1.3.2 (Wed, 28 Oct 2020 13:18:08 -0400)
FSTESTVER: nvme-cli v1.16 (Thu, 11 Nov 2021 13:09:06 -0800)
FSTESTVER: quota  v4.05-52-gf7e24ee (Tue, 1 Nov 2022 11:45:06 +0100)
FSTESTVER: util-linux v2.38.1 (Thu, 4 Aug 2022 11:06:21 +0200) 
FSTESTVER: xfsprogs v6.0.0 (Mon, 14 Nov 2022 12:06:23 +0100)   
FSTESTVER: xfstests-bld 920c0e32 (Mon, 28 Nov 2022 00:37:48 -0500)
FSTESTVER: xfstests v2022.11.27-8-g3c178050c (Wed, 30 Nov 2022 10:25:39 -0500)
FSTESTVER: zz_build-distro bullseye
FSTESTCFG: btrfs/4k
FSTESTSET: -g auto
FSTESTOPT: aex
GCE ID:    1186623270044702885


					- Ted