From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from mx2.suse.de ([195.135.220.15]:48775 "EHLO mx2.suse.de" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1754871AbeD3UsN (ORCPT ); Mon, 30 Apr 2018 16:48:13 -0400 Subject: Re: [PATCH] check: annotate good and expunged tests in results References: <20180412213838.28929-1-jeffm@suse.com> <20180424093646.GA11384@desktop> <20180427112356.GP11384@desktop> <20180427170009.GT27853@wotan.suse.de> <20180428024545.GR11384@desktop> From: Jeff Mahoney Message-ID: <33a80b8a-fe26-faee-9704-d1bb2c933a3a@suse.com> Date: Mon, 30 Apr 2018 16:48:08 -0400 MIME-Version: 1.0 In-Reply-To: <20180428024545.GR11384@desktop> Content-Type: text/plain; charset=utf-8 Content-Language: en-US Content-Transfer-Encoding: 8bit Sender: fstests-owner@vger.kernel.org To: Eryu Guan , "Luis R. Rodriguez" Cc: fstests@vger.kernel.org, nborisov@suse.com, fdmanana@suse.com List-ID: On 4/27/18 10:45 PM, Eryu Guan wrote: > On Fri, Apr 27, 2018 at 05:00:09PM +0000, Luis R. Rodriguez wrote: >> On Fri, Apr 27, 2018 at 07:23:56PM +0800, Eryu Guan wrote: >>> >>> But the test run & pass info is already available from the check output >>> and the test result summary at the end of check. Is that sufficient for >>> you? Also, we already have mechanism to generate a test report in xunit >>> format, i.e. ./check -R xunit -g auto, which includes results for passed >>> & failed & notrun tests. >> >> Do we have a way to parse the results *after* a run? For instance, >> if you forgot -R xunit ? > > There's a tools/compare-failures script that takes the outputs of check > as inputs and compares the results. But, TBH, I never run it after > reviewing it.. Perhaps it could be enhanced somehow. Yeah, that takes check output. Without capturing it at runtime, it can't compare anything. The XML report may do most of what I want provided we can enable it by default and write it someplace safe rather than clean it up automatically when the test run is interrupted. I already have test code that extends it to output expunged tests and to add an explicit element. It saves the timestamps already, so that's a plus. -Jeff -- Jeff Mahoney SUSE Labs