Re: [OE-core] [PATCH] oeqa/sdk/sdktest: Add test discovery across layers

[Richard Purdie <richard.purdie@linuxfoundation.org>]

On Thu, 2025-04-17 at 12:54 +0000, Thune Tran via lists.openembedded.org wrote:
> > On Wed, 2025-04-16 at 21:39 -0700, Chuck Wolber wrote:
> > > On Wed, Apr 16, 2025 at 3:07 PM Richard Purdie via
> > > lists.openembedded.org
> > > <richard.purdie=linuxfoundation.org@lists.openembedded.org> wrote:
> > > 
> > > %< SNIP $<
> > > > 
> > > > I suspect this causes some regressions in our automated testing
> > > > unfortunately:
> > > > 
> > > > buildtools:
> > > > https://autobuilder.yoctoproject.org/valkyrie/#/builders/43/builds/1
> > > > 396
> > > > 
> > > > meta-mingw:
> > > > https://autobuilder.yoctoproject.org/valkyrie/#/builders/7/builds/14
> > > > 30
> > > > 
> > > 
> > > 
> > > Would it make more sense to add
> > > self.context_executor_class.default_cases
> > > to the list of paths returned by get_sdk_paths?
> > 
> > meta/recipes-core/meta/buildtools-tarball.bb does:
> > 
> >     cases_path =
> > os.path.join(os.path.abspath(os.path.dirname(oeqa.sdk.testsdk.__file__)),
> > d.getVar("TESTSDK_CASES"))
> >     testsdk.context_executor_class.default_cases = cases_path
> > mingw does things in a different place, setting:
> > 
> >     default_cases = [os.path.join(os.path.abspath(os.path.dirname(__file__)),
> > 'cases')]
> > 
> > in meta-mingw/lib/oeqa/sdkmingw/contex.py
> > 
> > I think we need to add something which represents the name of the
> > directory for cases, i.e. "sdk", "sdkming", "sdk/buildtools-cases" and
> > then allow this to be passed into your function so regardless, it allows
> > customisation within a layer.
> We can match the test directory pattern (sdk, sdkming, sdk/buildtools-cases)  
> found in default_cases. The test discovery will then be based on the matched pattern.
> 

We could do that but in this case I think it might be better to improve
the code and make it a bit cleaner if we can, even if that is a little
more invasive.

This area of the code could do with simplification/cleanup...

Cheers,

Richard