Re: [OE-core] [PATCH v9 4/5] wic: move canned *wks files

[Richard Purdie <richard.purdie@linuxfoundation.org>]

On Sat, 2026-04-04 at 11:46 -0400, Trevor Woerner wrote:
> On Sat 2026-04-04 @ 08:27:13 AM, Richard Purdie wrote:
> > On Fri, 2026-04-03 at 18:23 -0400, Trevor Woerner wrote:
> > > On Fri 2026-04-03 @ 10:13:00 PM, Richard Purdie wrote:
> > > > On Fri, 2026-04-03 at 14:35 -0400, Trevor Woerner via lists.openembedded.org wrote:
> > > > > When "wic create ..." is invoked with a bare *wks name (i.e. without the
> > > > > `.wks` extension), wic calls engine.py:find_canned_images() to find the
> > > > > fully qualified *wks file. This function searches every directory formed by:
> > > > >     - permutating all BBLAYERS with `/wic`
> > > > >     - permutating all BBLAYERS with `/scripts/lib/wic/canned-wks`
> > > > >     - checking `<scripts_path>/lib/wic/canned-wks`
> > > > > Where `<scripts_path>` is the directory containing the wic program.
> > > > 
> > > > It doesn't. I just looked at the code and it uses BBPATH. That can be
> > > > similar to BBLAYERS but it is different and the commit messages really
> > > > should refer to BBPATH.
> > > 
> > > In the oe-selftest wic tests, most of the "wic create ..." commands
> > > are called with bare *wks files (i.e. *wks files without the `.wks`
> > > extension). When this happens, as "wic create..." is called, it starts
> > > by running the code found here:
> > > 
> > > 	https://git.openembedded.org/openembedded-core/tree/scripts/wic#n206
> > > 
> > > which calls:
> > > 
> > > 	https://git.openembedded.org/openembedded-core/tree/scripts/lib/wic/engine.py#n46
> > > 
> > > which follows the algorithm that I have described above.
> > > 
> > > So for the oe-selftests to pass I wanted to move them.
> > 
> > You are right, it is using BBLAYERS. This is very confusing since the
> > wks files are using BBPATH, which is similar but subtly different. 
> > 
> > Is there any reason it can't use BBPATH to match?
> > 
> > I agree the files need to move but I'd like them to end up in
> > meta/files/wic, not meta/wic. Whilst it complicates the migration a
> > little, I think it would be best to move them once and get them into
> > the right place.
> > 
> > The questions are then firstly, how to do that simply, then secondly,
> > how to clean up the rest of this (and when to do that).
> > 
> > Adding files/wic into the search path is trivial. Deprecating the
> > existing search paths with appropriate warnings is a bit trickier and
> > there is the question of when we should do that, before or after the
> > current release. "after" would seem more obvious, but it will mean two
> > different behaviours before and after release and with a standalone
> > tool, this does become harder.
> 
> I'm only guessing here, but it looks like someone wrote "wic create" to
> run two different ways: with the wks file specified with or without the
> ".wks" extension.
> 
> If ".wks" is not given (e.g. "wic create directdisk-gpt ...", which is
> what occurs in the oe-selftest tests, i.e. when wic is run independently
> without being part of a bitbake build) then wic invokes its own search
> algorithm to try to find the wks file using BBLAYERS and script_dir. In
> this case the wks file is specified without the ".wks" and no path
> information is given on the cmdline. It is up to wic to find the actual
> wks file itself.
> 
> If the ".wks" is included (which is what happens when wic is called as
> part of a bitbake build and therefore the image_types_wic.bbclass is
> used) then the class adds the ".wks" at the end, and also gives wic the
> path to the wks file which it finds using WKS_SEARCH_PATH which is based
> on BBPATH and COREBASE.
> 
> Assuming the persona of someone who wants to use this new, independent
> wic tool, and who knows nothing about The Yocto Project or bitbake, I
> think having wic look in "magical" places for the wks file would be hard
> to understand and surprising. As an independent tool I think the user
> should provide the path to the actual wks file (with the ".wks"
> extension) and if that file can't be found as specified on the cmdline,
> wic should simply fail with an error.
> 
> As part of making wic an independent tool, I think wic's code to search
> for a wks file if the ".wks" is not provided should be removed. This
> means that, as part of the transition, I would have to modify each "wic
> create" test in oe-core's oe-selftests. I'm fine with that. That could
> be done cleanly if I write a little function to find the wks file in
> the wic.py oe-selftest itself using the same logic as is used in the
> image_types_wic.bbclass.
> 
> To summarize: in my opinion, wic shouldn't have search logic. From wic's
> point of view the wks file should be specified on the cmdline in a way
> that wic will find the file the user wants to use (or not find it).
> The oe-selftests should be updated to use the same search logic from
> the image_types_wic.bbclass so that when it invokes "wic create" it is
> providing wic with the path to a wks file that has a ".wks" extension
> (which is how bitbake invokes wic) instead of specifying a bare wks file
> and hoping wic will find it.
> 
> I'm happy to add "files/wic" as another location for
> image_types_wic.bbclass to search for wks files.

I guess the first question is are we going to take this for wrynose or
not? I'm not particularly happy with the files a top level wic
directory, I think it needs to be meta/files/wic. To make that work, we
need to tweak the search paths in wic and in image_types_wic.bbclass.
If we can resolve that issue, we're probably good to move forward with
merging.

I agree that the search path logic should be tidied up. Whether you can
provide a search path to wic or whether that has to be done in metadata
is an open question. I do see your perspective, equally, if you are
using wic within OE, having some search knowledge may be ok. I would
want to see it basically using files/wic from BBPATH  with everything
else removed. There is a question of how you warn users about old
paths. This piece is too late for the release though.

So I'd propose we add files/wic to the search path, move the files
there and that lets us at least get some of the changes into the
release?

Cheers,

Richard