Re: [OE-core] [PATCH 0/2] image specific configuration with oeqa runtime tests

[Mikko Rapeli <mikko.rapeli@linaro.org>]

Hi,

On Thu, Nov 17, 2022 at 03:22:20PM +0100, Alexandre Belloni wrote:
> Hello,
> 
> With this two patches, I have multiple new warnings on the autobuilders
> for qemuarm and qemuarm-alt
> 
> https://autobuilder.yoctoproject.org/typhoon/#/builders/53/builds/6185/steps/13/logs/stdio
> https://autobuilder.yoctoproject.org/typhoon/#/builders/110/builds/5064/steps/12/logs/stdio

WARNING: core-image-sato-sdk-1.0-r0 do_testimage: No ignore list found for this machine and no valid testimage_data.json, using defaults

I can change these to be info messages to be at info level. Previously
these were only in the test output log, not in the bitbake output.
I think they need to be in both.

Cheers,

-Mikko

> On 17/11/2022 09:12:21+0200, Mikko Rapeli wrote:
> > Many runtime tests would need customization for different
> > machines and images. Currently some tests like parselogs.py are hard
> > coding machine specific exceptions into the test itself. I think these
> > machine specific exceptions fit better as image specific ones, since a
> > single machine config can generate multiple images which behave
> > differently. Thus create a "testimage_data.json" file format which image
> > recipes can deploy. This is then used by tests like parselogs.py to find
> > the image specific exception list.
> > 
> > Same approach would fit other runtime tests too. For example systemd
> > tests could include a test case which checks that an image specific list of
> > services are running.
> > 
> > I don't know how this data storage would be used with SDK or selftests,
> > but maybe it could work there too with some small tweaks.
> > 
> > Mikko Rapeli (2):
> >   oeqa: add utils/data.py with get_data() function
> >   oeqa parselogs.py: use get_data() to fetch image specific error list
> > 
> >  meta/lib/oeqa/runtime/cases/parselogs.py | 17 +++++++---
> >  meta/lib/oeqa/utils/data.py              | 41 ++++++++++++++++++++++++
> >  2 files changed, 54 insertions(+), 4 deletions(-)
> >  create mode 100644 meta/lib/oeqa/utils/data.py
> > 
> > -- 
> > 2.34.1
> > 
> 
> > 
> > -=-=-=-=-=-=-=-=-=-=-=-
> > Links: You receive all messages sent to this group.
> > View/Reply Online (#173404): https://lists.openembedded.org/g/openembedded-core/message/173404
> > Mute This Topic: https://lists.openembedded.org/mt/95085492/3617179
> > Group Owner: openembedded-core+owner@lists.openembedded.org
> > Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [alexandre.belloni@bootlin.com]
> > -=-=-=-=-=-=-=-=-=-=-=-
> > 
> 
> 
> -- 
> Alexandre Belloni, co-owner and COO, Bootlin
> Embedded Linux and Kernel engineering
> https://bootlin.com