Re: [bitbake-devel] [RFC PATCH 0/6] fetch2: add support for implicit urls

["Mathieu Dubois-Briand" <mathieu.dubois-briand@bootlin.com>]

On Tue Sep 2, 2025 at 8:55 AM CEST, Stefan Herbrechtsmeier via lists.openembedded.org wrote:
> From: Stefan Herbrechtsmeier <stefan.herbrechtsmeier@weidmueller.com>
>
> The patch series add support for implicit URLs inside the fetcher. The
> implicit URLs could be defined inside a source like a version control
> system (git submodule) or a lock file (package-lock.json, cargo.lock or
> go.sum). The integration of implicit URLs beside explicit URLs
> simplifies the fetcher classes and avoid bugs because of iterations
> between the Fetch and FetchMethod classes.
>
> The series remove most methods inside the gitsm fetcher and only leaves
> the parsing of the git submodules and the unpack functionality. It
> allows the gitsm fetcher to use the premirror only feature. The current
> implementation leads to problems because the download of the git
> submodules is triggered via the download method which is called deeply
> inside the fetcher code.
>
>
> Stefan Herbrechtsmeier (6):
>   fetch2: rename u to url in Fetch class
>   fetch2: call functions within loops of Fetch class
>   fetch2: add helper to get urldata in Fetch class
>   fetch2: add support for implicit urls
>   fetch2: gitsm: use implicit urls feature
>   tests: fetch: add test case for gitsm implicit local paths
>
>  lib/bb/fetch2/__init__.py | 128 +++++++++++++++++++++++++++-----------
>  lib/bb/fetch2/gitsm.py    |  46 ++------------
>  lib/bb/tests/fetch.py     |  12 ++++
>  3 files changed, 109 insertions(+), 77 deletions(-)

Hi Stefan,

I know it is just an RFC so far, but I did launch a build on the
autobuilder. It was mostly correct but fails a selftest:

ERROR: git-submodule-test-1.0-r0 do_ar_mirror: Error executing a python function in exec_func_python() autogenerated:
...
File: '/srv/pokybuild/yocto-worker/oe-selftest-debian/build/bitbake/lib/bb/fetch2/__init__.py', lineno: 2102, function: expand_urldata
...
Exception: UnboundLocalError: cannot access local variable 'urldata' where it is not associated with a value
...
2025-09-03 06:21:46,098 - oe-selftest - INFO - archiver.Archiver.test_archiver_mode_mirror_gitsm (subunit.RemotedTestCase)
2025-09-03 06:21:46,099 - oe-selftest - INFO -  ... FAIL

And a similar error with
archiver.Archiver.test_archiver_mode_mirror_gitsm_shallow.

Thanks,
Mathieu

-- 
Mathieu Dubois-Briand, Bootlin
Embedded Linux and Kernel engineering
https://bootlin.com