Re: [OE-core] [PATCH v9 4/5] wic: move canned *wks files

[Trevor Woerner <twoerner@gmail.com>]

On Sat 2026-04-04 @ 07:36:05 PM, Richard Purdie wrote:
> On Sat, 2026-04-04 at 11:46 -0400, Trevor Woerner wrote:
> > On Sat 2026-04-04 @ 08:27:13 AM, Richard Purdie wrote:
> > > On Fri, 2026-04-03 at 18:23 -0400, Trevor Woerner wrote:
> > > > On Fri 2026-04-03 @ 10:13:00 PM, Richard Purdie wrote:
> > > > > On Fri, 2026-04-03 at 14:35 -0400, Trevor Woerner via lists.openembedded.org wrote:
> > > > > > When "wic create ..." is invoked with a bare *wks name (i.e. without the
> > > > > > `.wks` extension), wic calls engine.py:find_canned_images() to find the
> > > > > > fully qualified *wks file. This function searches every directory formed by:
> > > > > >     - permutating all BBLAYERS with `/wic`
> > > > > >     - permutating all BBLAYERS with `/scripts/lib/wic/canned-wks`
> > > > > >     - checking `<scripts_path>/lib/wic/canned-wks`
> > > > > > Where `<scripts_path>` is the directory containing the wic program.
> > > > > 
> > > > > It doesn't. I just looked at the code and it uses BBPATH. That can be
> > > > > similar to BBLAYERS but it is different and the commit messages really
> > > > > should refer to BBPATH.
> > > > 
> > > > In the oe-selftest wic tests, most of the "wic create ..." commands
> > > > are called with bare *wks files (i.e. *wks files without the `.wks`
> > > > extension). When this happens, as "wic create..." is called, it starts
> > > > by running the code found here:
> > > > 
> > > > 	https://git.openembedded.org/openembedded-core/tree/scripts/wic#n206
> > > > 
> > > > which calls:
> > > > 
> > > > 	https://git.openembedded.org/openembedded-core/tree/scripts/lib/wic/engine.py#n46
> > > > 
> > > > which follows the algorithm that I have described above.
> > > > 
> > > > So for the oe-selftests to pass I wanted to move them.
> > > 
> > > You are right, it is using BBLAYERS. This is very confusing since the
> > > wks files are using BBPATH, which is similar but subtly different. 
> > > 
> > > Is there any reason it can't use BBPATH to match?
> > > 
> > > I agree the files need to move but I'd like them to end up in
> > > meta/files/wic, not meta/wic. Whilst it complicates the migration a
> > > little, I think it would be best to move them once and get them into
> > > the right place.
> > > 
> > > The questions are then firstly, how to do that simply, then secondly,
> > > how to clean up the rest of this (and when to do that).
> > > 
> > > Adding files/wic into the search path is trivial. Deprecating the
> > > existing search paths with appropriate warnings is a bit trickier and
> > > there is the question of when we should do that, before or after the
> > > current release. "after" would seem more obvious, but it will mean two
> > > different behaviours before and after release and with a standalone
> > > tool, this does become harder.
> > 
> > I'm only guessing here, but it looks like someone wrote "wic create" to
> > run two different ways: with the wks file specified with or without the
> > ".wks" extension.
> > 
> > If ".wks" is not given (e.g. "wic create directdisk-gpt ...", which is
> > what occurs in the oe-selftest tests, i.e. when wic is run independently
> > without being part of a bitbake build) then wic invokes its own search
> > algorithm to try to find the wks file using BBLAYERS and script_dir. In
> > this case the wks file is specified without the ".wks" and no path
> > information is given on the cmdline. It is up to wic to find the actual
> > wks file itself.
> > 
> > If the ".wks" is included (which is what happens when wic is called as
> > part of a bitbake build and therefore the image_types_wic.bbclass is
> > used) then the class adds the ".wks" at the end, and also gives wic the
> > path to the wks file which it finds using WKS_SEARCH_PATH which is based
> > on BBPATH and COREBASE.
> > 
> > Assuming the persona of someone who wants to use this new, independent
> > wic tool, and who knows nothing about The Yocto Project or bitbake, I
> > think having wic look in "magical" places for the wks file would be hard
> > to understand and surprising. As an independent tool I think the user
> > should provide the path to the actual wks file (with the ".wks"
> > extension) and if that file can't be found as specified on the cmdline,
> > wic should simply fail with an error.
> > 
> > As part of making wic an independent tool, I think wic's code to search
> > for a wks file if the ".wks" is not provided should be removed. This
> > means that, as part of the transition, I would have to modify each "wic
> > create" test in oe-core's oe-selftests. I'm fine with that. That could
> > be done cleanly if I write a little function to find the wks file in
> > the wic.py oe-selftest itself using the same logic as is used in the
> > image_types_wic.bbclass.
> > 
> > To summarize: in my opinion, wic shouldn't have search logic. From wic's
> > point of view the wks file should be specified on the cmdline in a way
> > that wic will find the file the user wants to use (or not find it).
> > The oe-selftests should be updated to use the same search logic from
> > the image_types_wic.bbclass so that when it invokes "wic create" it is
> > providing wic with the path to a wks file that has a ".wks" extension
> > (which is how bitbake invokes wic) instead of specifying a bare wks file
> > and hoping wic will find it.
> > 
> > I'm happy to add "files/wic" as another location for
> > image_types_wic.bbclass to search for wks files.
> 
> I guess the first question is are we going to take this for wrynose or
> not? I'm not particularly happy with the files a top level wic
> directory, I think it needs to be meta/files/wic. To make that work, we
> need to tweak the search paths in wic and in image_types_wic.bbclass.
> If we can resolve that issue, we're probably good to move forward with
> merging.
> 
> I agree that the search path logic should be tidied up. Whether you can
> provide a search path to wic or whether that has to be done in metadata
> is an open question. I do see your perspective, equally, if you are
> using wic within OE, having some search knowledge may be ok. I would
> want to see it basically using files/wic from BBPATH  with everything
> else removed. There is a question of how you warn users about old
> paths. This piece is too late for the release though.
> 
> So I'd propose we add files/wic to the search path, move the files
> there and that lets us at least get some of the changes into the
> release?

Sounds perfect, thanks!

> Cheers,
> 
> Richard
>