Re: [igt-dev] [PATCH i-g-t] meson: Disable testplan build by default

["Ville Syrjälä" <ville.syrjala@linux.intel.com>]

On Wed, Apr 12, 2023 at 02:54:26PM +0200, Mauro Carvalho Chehab wrote:
> On Wed, 12 Apr 2023 10:33:39 +0300
> Ville Syrj�l� <ville.syrjala@linux.intel.com> wrote:
> 
> > On Wed, Apr 12, 2023 at 09:26:29AM +0200, Mauro Carvalho Chehab wrote:
> > > On Tue, 11 Apr 2023 14:02:05 +0300
> > > Ville Syrjala <ville.syrjala@linux.intel.com> wrote:
> > > 
> > > > From: Ville Syrj�l� <ville.syrjala@linux.intel.com>
> > > 
> > > > No real idea what this "testplan" is and why it lives
> > > > in igt. Seems to be build some xe documentation, and based
> > > > on the name is maybe some manager level stuff? Surely
> > > > this is nothing that matters for normal people (esp. those
> > > > outside Intel).
> > > 
> > > No. Basically, testplan contains documentation for the tests.
> > > Currently, it contains 100% of the documentation from Xe tests.
> > > We may end implementing it for i915 as well.
> > > 
> > > There is even a plan to generate testlists like xe-fast-feedback.testlist
> > > directly from documentation. So, it is important to have the tests properly
> > > documented, as otherwise they won't be executed by CI in the future.
> > > 
> > > > 
> > > > The main problem here being that it is hideously slow to
> > > > build, making life miserable for everyone. Flip the
> > > > default to disabled and let those that need this enable
> > > > it themselves.
> > > 
> > > Generating documents takes ~150ms. What makes it slow is not the doc
> > > generation itself, but a validation logic that checks if the test 
> > > documentation was updated as tests got added/renamed/removed. 
> > > It currently uses igt_runner, pointing to the documented tests. 
> > > As right now just Xe tests uses it, it will call the runner with:
> > > 
> > > 	$ igt_runner -L -t igt@xe ${builddir}/tests
> > 
> > That is not the slow thing. The slow thing is some python stuff.
> 
> This was part of the slow logic. The other part is to run regular
> expressions to check test names. This also needed some optimization. 
> 
> Just sent a patch series using re.compile() to speed regexes, with
> a replacement patch for igt_runner.
> 
> It could be possible to optimize it even further with multithreading
> but as it is now taking 500ms to run (against 12 seconds before that on
> a i7 notebook using python 3.11), it seems good enough to me after the
> changes.
> 
> Patch series sent: https://patchwork.freedesktop.org/series/116379/

That is definitely much improved. Thanks.

Still feels a bit too slow to keep enabled
during normal developement work though.

$ time git rebase -x 'ninja -Cbuild' HEAD~10

-Dtestplan=disabled
real    0m9,699s
user    0m26,422s
sys     0m2,518s

-Dtestplan=auto
real    0m46,020s
user    1m5,102s
sys     0m4,191s

At least for me that that would still exceed
my attention span.

But seems fast enough for a one off test build 
before sending out patches/pushing.

-- 
Ville Syrj�l�
Intel