Re: [PATCH] fstests: report: add arch and kernel version info into testsuite attributes

["Darrick J. Wong" <djwong@kernel.org>]

On Fri, Dec 16, 2022 at 03:05:43PM +0800, Qu Wenruo wrote:
> Although "testcase" tags contain the "timestamp" element, for day-0
> testing it can be hard to relate the timestamp to the tested kernel
> version.
> 
> Thus this patch will add a "kernel" element to the "testcase" tag, to
> indicate the kernel version we're running.
> Paired with CONFIG_LOCALVERSION_AUTO=y config, it will easily show the
> kernel commit we're testing.
> 
> Since we're here, also add a "arch" element, as there are more and more
> aarch64 boards (From RK3399 to Apple M1) able to finish fstests in an
> acceptable duration, we can no longer assume x86_64 as our only
> platform.
> 
> Signed-off-by: Qu Wenruo <wqu@suse.com>
> ---
>  common/report | 2 +-
>  1 file changed, 1 insertion(+), 1 deletion(-)
> 
> diff --git a/common/report b/common/report
> index 4a747f8d..92586527 100644
> --- a/common/report
> +++ b/common/report
> @@ -49,7 +49,7 @@ _xunit_make_section_report()
>  		date_time=$(date +"%F %T")
>  	fi
>  	local stats="failures=\"$bad_count\" skipped=\"$notrun_count\" tests=\"$tests_count\" time=\"$sect_time\""
> -	local hw_info="hostname=\"$HOST\" timestamp=\"${date_time/ /T}\" "
> +	local hw_info="hostname=\"$HOST\" timestamp=\"${date_time/ /T}\" arch=\"$(uname -m)\" kernel=\"$(uname -r)\""
>  	echo "<testsuite name=\"xfstests\" $stats  $hw_info >" >> $REPORT_DIR/result.xml

The original commit that added this report format was f9fde7db2f
("report: Add xunit format report generator").  Dmitry Monakhov's commit
message points out that the xml being emitted was "xunit/junit":

    Footnotes:
    [1] https://xunit.github.io/docs/format-xml-v2.html
    [2] http://help.catchsoftware.com/display/ET/JUnit+Format

The first link is now dead, but the second link contains enough
information to find the current junit xml format:

[1] https://raw.githubusercontent.com/windyroad/JUnit-Schema/master/JUnit.xsd

Note that the xunit project appears to have diverged their report
format:

[2] https://xunit.net/docs/format-xml-v2

(Or perhaps there were multiple things called xunit?)

Either way, it's pretty obvious from common/report code that the "xunit"
code is still emitting junit xml files.  I think it's important that
fstests should continue to follow that schema, so that these files can
be fed into test dashboards (yes I have one) that consume this file
format.

Regrettably, the schema does not provide for @arch or @kernel attributes
hanging off the <testsuite> element, so it's not a good idea to add
things that a strict parser could reject.

That said, I think it's important to record the architecture and kernel.
Probably even more attributes than that.  The junit xml schema provides
for arbitrary string properties to be attached to reports; would you
mind putting these there instead?

(I want to add a few more properties now that people have started
talking about reporting again... ;))

	# Properties
	echo -e "\t<properties>" >> $REPORT_DIR/result.xml
	echo -e "\t\t<property name=\"arch\" value=\"$(uname -m)\"/>" >> $REPORT_DIR/result.xml
	echo -e "\t\t<property name=\"kernel\" value=\"$(uname -r)\"/>" >> $REPORT_DIR/result.xml
	for p in "${REPORT_ENV_LIST[@]}"; do
		_xunit_add_property "$p"
	done
	echo -e "\t</properties>" >> $REPORT_DIR/result.xml

--D

>  
>  	# Properties
> -- 
> 2.38.0
>