Re: [igt-dev] [PATCH i-g-t] meson: Disable testplan build by default

[Mauro Carvalho Chehab <mauro.chehab@linux.intel.com>]

On Wed, 12 Apr 2023 10:33:39 +0300
Ville Syrjälä <ville.syrjala@linux.intel.com> wrote:

> On Wed, Apr 12, 2023 at 09:26:29AM +0200, Mauro Carvalho Chehab wrote:
> > On Tue, 11 Apr 2023 14:02:05 +0300
> > Ville Syrjala <ville.syrjala@linux.intel.com> wrote:
> > 
> > > From: Ville Syrjälä <ville.syrjala@linux.intel.com>
> > 
> > > No real idea what this "testplan" is and why it lives
> > > in igt. Seems to be build some xe documentation, and based
> > > on the name is maybe some manager level stuff? Surely
> > > this is nothing that matters for normal people (esp. those
> > > outside Intel).
> > 
> > No. Basically, testplan contains documentation for the tests.
> > Currently, it contains 100% of the documentation from Xe tests.
> > We may end implementing it for i915 as well.
> > 
> > There is even a plan to generate testlists like xe-fast-feedback.testlist
> > directly from documentation. So, it is important to have the tests properly
> > documented, as otherwise they won't be executed by CI in the future.
> > 
> > > 
> > > The main problem here being that it is hideously slow to
> > > build, making life miserable for everyone. Flip the
> > > default to disabled and let those that need this enable
> > > it themselves.
> > 
> > Generating documents takes ~150ms. What makes it slow is not the doc
> > generation itself, but a validation logic that checks if the test 
> > documentation was updated as tests got added/renamed/removed. 
> > It currently uses igt_runner, pointing to the documented tests. 
> > As right now just Xe tests uses it, it will call the runner with:
> > 
> > 	$ igt_runner -L -t igt@xe ${builddir}/tests
> 
> That is not the slow thing. The slow thing is some python stuff.

This was part of the slow logic. The other part is to run regular
expressions to check test names. This also needed some optimization. 

Just sent a patch series using re.compile() to speed regexes, with
a replacement patch for igt_runner.

It could be possible to optimize it even further with multithreading
but as it is now taking 500ms to run (against 12 seconds before that on
a i7 notebook using python 3.11), it seems good enough to me after the
changes.

Patch series sent: https://patchwork.freedesktop.org/series/116379/

Regards,
Mauro