Re: [PATCH 3/4] test/py: Show info about module-loading

[Tom Rini <trini@konsulko.com>]

[-- Attachment #1: Type: text/plain, Size: 4063 bytes --]

On Sat, Jan 25, 2025 at 04:39:19PM -0700, Simon Glass wrote:
> Hi Tom,
> 
> On Sat, 25 Jan 2025 at 16:02, Tom Rini <trini@konsulko.com> wrote:
> >
> > On Sat, Jan 25, 2025 at 04:00:05PM -0700, Simon Glass wrote:
> > > Hi Tom,
> > >
> > > On Sat, 25 Jan 2025 at 14:43, Tom Rini <trini@konsulko.com> wrote:
> > > >
> > > > On Sat, Jan 25, 2025 at 02:31:38PM -0700, Simon Glass wrote:
> > > > > It is sometimes tricky to figure out what modules test.py is loading
> > > > > when it starts up. The result can be a silent failure with no clue as to
> > > > > what when wrong.
> > > > >
> > > > > Add a section which lists the modules loaded as well as those not
> > > > > found.
> > > > >
> > > > > Signed-off-by: Simon Glass <sjg@chromium.org>
> > > > > ---
> > > > >
> > > > >  test/py/conftest.py | 33 ++++++++++++++++++++-------------
> > > > >  1 file changed, 20 insertions(+), 13 deletions(-)
> > > > >
> > > > > diff --git a/test/py/conftest.py b/test/py/conftest.py
> > > > > index 5096c20ed65..23435b7db3c 100644
> > > > > --- a/test/py/conftest.py
> > > > > +++ b/test/py/conftest.py
> > > > > @@ -286,19 +286,26 @@ def pytest_configure(config):
> > > > >      ubconfig = ArbitraryAttributeContainer()
> > > > >      ubconfig.brd = dict()
> > > > >      ubconfig.env = dict()
> > > > > -
> > > > > -    modules = [
> > > > > -        (ubconfig.brd, 'u_boot_board_' + board_type_filename),
> > > > > -        (ubconfig.env, 'u_boot_boardenv_' + board_type_filename),
> > > > > -        (ubconfig.env, 'u_boot_boardenv_' + board_type_filename + '_' +
> > > > > -            board_identity_filename),
> > > > > -    ]
> > > > > -    for (dict_to_fill, module_name) in modules:
> > > > > -        try:
> > > > > -            module = __import__(module_name)
> > > > > -        except ImportError:
> > > > > -            continue
> > > > > -        dict_to_fill.update(module.__dict__)
> > > > > +    not_found = []
> > > > > +
> > > > > +    with log.section('Loading lab modules', 'load_modules'):
> > > > > +        modules = [
> > > > > +            (ubconfig.brd, 'u_boot_board_' + board_type_filename),
> > > > > +            (ubconfig.env, 'u_boot_boardenv_' + board_type_filename),
> > > > > +            (ubconfig.env, 'u_boot_boardenv_' + board_type_filename + '_' +
> > > > > +                board_identity_filename),
> > > > > +        ]
> > > > > +        for (dict_to_fill, module_name) in modules:
> > > > > +            try:
> > > > > +                module = __import__(module_name)
> > > > > +            except ImportError:
> > > > > +                not_found.append(module_name)
> > > > > +                continue
> > > > > +            dict_to_fill.update(module.__dict__)
> > > > > +            log.info(f"Loaded {module}")
> > > > > +
> > > > > +        if not_found:
> > > > > +            log.warning(f"Failed to find modules: {' '.join(not_found)}")
> > > > >
> > > > >      ubconfig.buildconfig = dict()
> > > >
> > > > We won't see this warning normally, only if some level of verbosity is
> > > > used, yes? It's normal to not have both u_boot_boardenv files for
> > > > example.
> > >
> > > There is a '[-] Section: Loading lab modules' section so you only see
> > > it if you open that.
> > >
> > > I feel that test.py is a bit cryptic at present.
> >
> > I don't follow you, sorry. Open what where?
> 
> I mean that the section is collapsed unless you click to open it (when
> viewed in a web browser).

Oh, right,
https://u-boot.source-pages.denx.de/-/u-boot/-/jobs/1005206/artifacts/test-log.html
and similar. Yes, anything you want to improve to that part is fine with
me. I occasionally look at
https://source.denx.de/u-boot/u-boot/-/pipelines/24337/test_report?job_name=coreboot+test.py
for example but just generally make sure CI saves those reports off for
people / tooling that wants to consume them.
https://source.denx.de/u-boot/u-boot/-/jobs/1005206 is what I pay most
attention to (and similar output locally).

-- 
Tom

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 659 bytes --]