Re: [igt-dev] [Intel-gfx] [ANNOUNCEMENT] Documenting tests with igt_describe()

From: Petri Latvala <petri.latvala@intel.com>
To: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Cc: igt-dev@lists.freedesktop.org, intel-gfx@lists.freedesktop.org
Subject: Re: [igt-dev] [Intel-gfx] [ANNOUNCEMENT] Documenting tests with igt_describe()
Date: Tue, 19 Nov 2019 14:55:45 +0200	[thread overview]
Message-ID: <20191119125545.GB25209@platvala-desk.ger.corp.intel.com> (raw)
In-Reply-To: <59485e6b-e06f-29c7-14d4-30de572e9022@intel.com>

On Tue, Nov 19, 2019 at 02:37:17PM +0200, Lionel Landwerlin wrote:
> On 08/11/2019 11:04, Arkadiusz Hiler wrote:
> > On Thu, Nov 07, 2019 at 09:09:34PM +0000, Chris Wilson wrote:
> > > Quoting Arkadiusz Hiler (2019-11-07 17:38:20)
> > > > We don't want you to translate C into English, we want you to provide a bit of
> > > > that extra information that you would have put in the comments anyway.
> > > The comments should exist and are _inline_ with the code.
> > And I encourage doing so. Detailed comments what this particular
> > non-obvious chunk of code is doing are always welcome.
> > 
> > > In all the examples of igt_describe() I have seen, it is nowhere near
> > > the code so is useless; the information conveyed does not assist
> > > anyone in diagnosing or debugging the problem, so I yet to understand
> > > how it helps.
> > They are supposed to document not the implementation but what is the
> > grand idea behind a given subtest, so they are on a subtest level (or a
> > group of subtests), which is our basic testing unit - this is what fails
> > in CI, this is what you execute locally to reproduce the issue.
> > 
> > Can you truly debug an issue or understand what the failure means
> > without knowing what the test is supposed to prove?
> > 
> > A lot of people here have struggled with this and often it takes a lot
> > of time and requires advanced archeology with `git blame` hoping that
> > there is at least one detailed enough commit message in there.
> > 
> > If not then talking to people and relying on our tribal knowledge is
> > required.
> > 
> > As I have mentioned - getting the bigger picture from implementation
> > details is hard. I get that you are not affected by this, but please do
> > not deny the others.
> 
> 
> I kind of agree with Chris that I don't find that additional macro useful
> from the point of view of reading a particular test.
> 
> A comment above the test function seems more appropriate, at least you don't
> have to look at 2 different places to find out about a particular test.
> 
> 
> Unless there is some untold goal here, like producing some kind of report in
> an automated way.

Like this one? https://drm.pages.freedesktop.org/igt-gpu-tools/igt-kms-tests.html#kms_chamelium

-- 
Petri Latvala
_______________________________________________
igt-dev mailing list
igt-dev@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/igt-dev