[RFC PATCH 0/4] Clean AUTOREV usage, add revision check for every SCM URI

[RFC PATCH 0/4] Clean AUTOREV usage, add revision check for every SCM URI

[Corentin Guillevic]

This patch series introduces an additional "insane" test to check whether every
URI within the SRC_URI variable has a revision. This is the purpose of the last
two patches.

However, the new test raises a "deep" issue when AUTOREV is set:
"AUTOREV/SRCPV set too late for the fetcher to work properly". This occurs
because the test accesses to the SRCREV variable, at the line:

if d.getVar('SRCREV') == "INVALID":

The following self-tests can raise this issue (if my test is enabled):

$ oe-selftest -Kv -r bblayers.BitbakeLayers \
    -r pokybleeding.PokyBleeding.test_poky_bleeding_autorev \
    -r devtool.DevtoolUpgradeTests

Or a shorter one:

$ recipetool --color=auto create --devtool -o /tmp/devtool4819z5wk \
    'http://downloads.yoctoproject.org/mirror/sources/i2c-tools-3.1.2.tar.bz2' \
    -x ${BUILDDIR}/workspace/sources/devtoolsrcdcviuvn9

The common point is AUTOREV, which is used as default value by devtool and
recipetool. When AUTOREV is parsed, this triggers the call to the get_autorev()
function (see [1]) which sets the internal __BBAUTOREV_SEEN variable.

If the associated URI is a SCM one, there is no problem. Otherwise, the above
error will occur (see [2]) because the fetcher code doesn't take AUTOREV into
account (see [3]).

Why has this error never occurred before? In my opinion, it is related to how
devtool/recipetool parse and run their crafted recipes. Firstly, these tools
unconditionally set AUTOREV as default value for srcrev. Despite this value is
not written into the temporary recipe if SRC_URI doesn't have a SCM URI, the
tools seem to keep it in memory.

Next, the crafted recipe is run. Without my patch series, no error is raised
because I think there is no use of SRCREV in the execution flow. With my insane
test only, which uses SRCREV, the test code is parsed and expanded. As SRCREV
is being used, its value AUTOREV is expanded and leads to function
get_autorev(). As the URI is no a SCM one, no code handles AUTOREV and the
above error occurs.

This is why the first two patches: their purpose is to set srcrev to AUTOREV
only if the submitted URI is a SCM one. These patches resolve, the
"AUTOREV/SRCPV set too late for the fetcher to work properly" error disappears,
as AUTOREV is only set when required.

I sent this patch series as RFC because I need feedback on this issue. What
are your thoughts on the error and my fix? Is my theory correct about
devtool/recipetool keeping some variables in memory before running the crafted
recipe?

Thank you

Corentin

- [1] https://git.openembedded.org/bitbake/tree/lib/bb/fetch2/__init__.py
- [2] Function finalize() from https://git.openembedded.org/bitbake/tree/lib/bb/parse/ast.py
- [3] Function setup_revisions() from https://git.openembedded.org/bitbake/tree/lib/bb/fetch2/__init__.py, only called from SCM fetchers

Corentin Guillevic (4):
  insane.bbclass: check revision for every git URI
  layer.conf: enable test for revision alongside a git URI
  recipetool: create: use default AUTOREV only for SCM URI
  devtool: use default AUTOREV only for SCM URI

 meta/classes-global/insane.bbclass | 27 +++++++++++++++++++++++++++
 meta/conf/layer.conf               |  3 +++
 scripts/lib/devtool/upgrade.py     |  7 ++++++-
 scripts/lib/recipetool/create.py   |  7 ++++++-
 4 files changed, 42 insertions(+), 2 deletions(-)

-- 
2.51.0

[RFC PATCH 1/4] insane.bbclass: check revision for every git URI

[Corentin Guillevic]

Every git URI into SRC_URI variable should have a revision. However, if
the revision is missing (SRCREV), BitBake will perform a query on the
remote repository every time it parses a recipe.

This check will raise an error if a git URI is not provided alongside a revision.

Signed-off-by: Corentin Guillevic <corentin.guillevic@smile.fr>
---
 meta/classes-global/insane.bbclass | 27 +++++++++++++++++++++++++++
 1 file changed, 27 insertions(+)

diff --git a/meta/classes-global/insane.bbclass b/meta/classes-global/insane.bbclass
index aeffee7983..349da3a65d 100644
--- a/meta/classes-global/insane.bbclass
+++ b/meta/classes-global/insane.bbclass
@@ -1493,7 +1493,34 @@ python do_recipe_qa() {
                 error_msg = "%s: invalid PACKAGECONFIG(s): %s" % (pn, " ".join(sorted(invalid_pkgconfigs)))
                 oe.qa.handle_error("invalid-packageconfig", error_msg, d)
 
+    def test_git_missing_srcrev(pn, d):
+        sha1_re = re.compile(r'^[0-9a-f]{40}$')
+        for uri in d.getVar('SRC_URI').split():
+            # Get tokens for the current URI
+            params = bb.fetch2.decodeurl(uri)
+
+            if params[0] != 'git':
+                continue
+
+            name = params[5].get('name', '')
+            rev = params[5].get('rev', '')
+
+            # Revision is provided as a 'rev' parameter
+            if rev:
+                # Prevent any revision that doesn't look like a SHA-1
+                if not sha1_re.match(rev or ''):
+                    oe.qa.handle_error("missing-srcrev", "%s: ;rev=%s URL parameter doesn't look like a SHA-1" % (pn, rev), d)
+            # git URI has a "name" parameter
+            elif name:
+                rev = d.getVar('SRCREV_' + name)
+                if not rev:
+                    oe.qa.handle_error("missing-srcrev", "%s: no revision (%s) defined for URI %s" % (pn, 'SRCREV_' + name, uri), d)
+            else:
+                if d.getVar('SRCREV') == "INVALID":
+                    oe.qa.handle_error("missing-srcrev", "%s: no revision (SRCREV) defined for URI %s" % (pn, uri), d)
+
     pn = d.getVar('PN')
+    test_git_missing_srcrev(pn, d)
     test_naming(pn, d)
     test_missing_metadata(pn, d)
     test_missing_maintainer(pn, d)
-- 
2.51.0

[RFC PATCH 2/4] layer.conf: enable test for revision alongside a git URI

[Corentin Guillevic]

As meta is the main layer, its recipes should fully comply with revisions
for git URIs. Enable the check for this layer only.

Signed-off-by: Corentin Guillevic <corentin.guillevic@smile.fr>
---
 meta/conf/layer.conf | 3 +++
 1 file changed, 3 insertions(+)

diff --git a/meta/conf/layer.conf b/meta/conf/layer.conf
index ba25ca3029..1f95867165 100644
--- a/meta/conf/layer.conf
+++ b/meta/conf/layer.conf
@@ -19,6 +19,9 @@ BBLAYERS_LAYERINDEX_NAME_core = "openembedded-core"
 # Set a variable to get to the top of the metadata location
 COREBASE = '${@os.path.normpath("${LAYERDIR}/../")}'
 
+# Raise an error if no revision is provided with a git URI
+ERROR_QA:append:layer-core = " missing-srcrev"
+
 # opkg-utils is for update-alternatives :(
 SIGGEN_EXCLUDERECIPES_ABISAFE += " \
   sysvinit-inittab \
-- 
2.51.0

[RFC PATCH 3/4] recipetool: create: use default AUTOREV only for SCM URI

[Corentin Guillevic]

AUTOREV shoud only be used for SCM URIs (Git, SVN...). However, the
default revision is set to AUTOREV regardless of the URI scheme.

To fix this, set AUTOREV as the default value only if the source URI
supports SRCREV.

Signed-off-by: Corentin Guillevic <corentin.guillevic@smile.fr>
---
 scripts/lib/recipetool/create.py | 7 ++++++-
 1 file changed, 6 insertions(+), 1 deletion(-)

diff --git a/scripts/lib/recipetool/create.py b/scripts/lib/recipetool/create.py
index ef0ba974a9..d785f042ee 100644
--- a/scripts/lib/recipetool/create.py
+++ b/scripts/lib/recipetool/create.py
@@ -419,7 +419,7 @@ def create_recipe(args):
     tempsrc = ''
     source = args.source
     srcsubdir = ''
-    srcrev = '${AUTOREV}'
+    srcrev = ''
     srcbranch = ''
     scheme = ''
     storeTagName = ''
@@ -470,6 +470,11 @@ def create_recipe(args):
             # so we need to extract to a subdirectory
             fetchuri += ';subdir=${BPN}'
         srcuri = fetchuri
+
+        # Define AUTOREV as default value only if the source URI supports SRCREV
+        if srcuri and srcuri.startswith(('gitsm://', 'git://', 'hg://', 'svn://')):
+            srcrev = '${AUTOREV}'
+
         rev_re = re.compile(';rev=([^;]+)')
         res = rev_re.search(srcuri)
         if res:
-- 
2.51.0

[RFC PATCH 4/4] devtool: use default AUTOREV only for SCM URI

[Corentin Guillevic]

Same than recipetool, the default revision is set to AUTOREV regardless
of the URI scheme. AUTOREV is now the default value only if the source URI
supports SRCREV.

Signed-off-by: Corentin Guillevic <corentin.guillevic@smile.fr>
---
 scripts/lib/devtool/upgrade.py | 7 ++++++-
 1 file changed, 6 insertions(+), 1 deletion(-)

diff --git a/scripts/lib/devtool/upgrade.py b/scripts/lib/devtool/upgrade.py
index 8930fde5d6..34e23c2c45 100644
--- a/scripts/lib/devtool/upgrade.py
+++ b/scripts/lib/devtool/upgrade.py
@@ -157,7 +157,12 @@ def _get_uri(rd):
             break
     if not srcuri:
         raise DevtoolError('Unable to find non-local entry in SRC_URI')
-    srcrev = '${AUTOREV}'
+
+    if srcuri and srcuri.startswith(('gitsm://', 'git://', 'hg://', 'svn://')):
+        srcrev = '${AUTOREV}'
+    else:
+        srcrev = None
+
     if '://' in srcuri:
         # Fetch a URL
         rev_re = re.compile(';rev=([^;]+)')
-- 
2.51.0

Re: [OE-core] [RFC PATCH 0/4] Clean AUTOREV usage, add revision check for every SCM URI

[Mathieu Dubois-Briand]

On Wed Mar 4, 2026 at 11:43 AM CET, Corentin Guillevic via lists.openembedded.org wrote:
> This patch series introduces an additional "insane" test to check whether every
> URI within the SRC_URI variable has a revision. This is the purpose of the last
> two patches.
>
> However, the new test raises a "deep" issue when AUTOREV is set:
> "AUTOREV/SRCPV set too late for the fetcher to work properly". This occurs
> because the test accesses to the SRCREV variable, at the line:
>
> if d.getVar('SRCREV') == "INVALID":
>
> The following self-tests can raise this issue (if my test is enabled):
>
> $ oe-selftest -Kv -r bblayers.BitbakeLayers \
>     -r pokybleeding.PokyBleeding.test_poky_bleeding_autorev \
>     -r devtool.DevtoolUpgradeTests
>
> Or a shorter one:
>
> $ recipetool --color=auto create --devtool -o /tmp/devtool4819z5wk \
>     'http://downloads.yoctoproject.org/mirror/sources/i2c-tools-3.1.2.tar.bz2' \
>     -x ${BUILDDIR}/workspace/sources/devtoolsrcdcviuvn9
>
> The common point is AUTOREV, which is used as default value by devtool and
> recipetool. When AUTOREV is parsed, this triggers the call to the get_autorev()
> function (see [1]) which sets the internal __BBAUTOREV_SEEN variable.
>
> If the associated URI is a SCM one, there is no problem. Otherwise, the above
> error will occur (see [2]) because the fetcher code doesn't take AUTOREV into
> account (see [3]).
>
> Why has this error never occurred before? In my opinion, it is related to how
> devtool/recipetool parse and run their crafted recipes. Firstly, these tools
> unconditionally set AUTOREV as default value for srcrev. Despite this value is
> not written into the temporary recipe if SRC_URI doesn't have a SCM URI, the
> tools seem to keep it in memory.
>
> Next, the crafted recipe is run. Without my patch series, no error is raised
> because I think there is no use of SRCREV in the execution flow. With my insane
> test only, which uses SRCREV, the test code is parsed and expanded. As SRCREV
> is being used, its value AUTOREV is expanded and leads to function
> get_autorev(). As the URI is no a SCM one, no code handles AUTOREV and the
> above error occurs.
>
> This is why the first two patches: their purpose is to set srcrev to AUTOREV
> only if the submitted URI is a SCM one. These patches resolve, the
> "AUTOREV/SRCPV set too late for the fetcher to work properly" error disappears,
> as AUTOREV is only set when required.
>
> I sent this patch series as RFC because I need feedback on this issue. What
> are your thoughts on the error and my fix? Is my theory correct about
> devtool/recipetool keeping some variables in memory before running the crafted
> recipe?
>
> Thank you
>
> Corentin
>

Hi Corentin,

Thanks for the update. I know this is still an RFC, but I still picked
it for a run on the autobuilder.

I noted just a single test fail, that seems linked with this series:

2026-03-10 16:29:38,455 - oe-selftest - INFO - devtool.DevtoolAddTests.test_devtool_add_git_local (subunit.RemotedTestCase)
2026-03-10 16:29:38,457 - oe-selftest - INFO -  ... FAIL
...
ERROR: Traceback (most recent call last):
  File "/srv/pokybuild/yocto-worker/oe-selftest-armhost/build/layers/bitbake/lib/bb/data_smart.py", line 465, in expandWithRefs
    s = __expand_python_regexp__.sub(varparse.python_sub, s)
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/srv/pokybuild/yocto-worker/oe-selftest-armhost/build/layers/bitbake/lib/bb/data_smart.py", line 154, in python_sub
    value = utils.better_eval(codeobj, DataContext(self.d), {'d' : self.d})
            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/srv/pokybuild/yocto-worker/oe-selftest-armhost/build/layers/bitbake/lib/bb/utils.py", line 489, in better_eval
    return eval(source, ctx, locals)
           ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "Var <fetcher_hashes_dummyfunc[vardepvalue]>", line 1, in <module>
  File "/srv/pokybuild/yocto-worker/oe-selftest-armhost/build/layers/bitbake/lib/bb/fetch2/__init__.py", line 853, in get_hashvalue
    pkgv, revs = _get_srcrev(d, method_name=method_name)
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/srv/pokybuild/yocto-worker/oe-selftest-armhost/build/layers/bitbake/lib/bb/fetch2/__init__.py", line 798, in _get_srcrev
    fetcher = Fetch(d.getVar('SRC_URI').split(), d)
              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/srv/pokybuild/yocto-worker/oe-selftest-armhost/build/layers/bitbake/lib/bb/fetch2/__init__.py", line 1821, in __init__
    self.ud[url] = FetchData(url, d, localonly)
                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/srv/pokybuild/yocto-worker/oe-selftest-armhost/build/layers/bitbake/lib/bb/fetch2/__init__.py", line 1357, in __init__
    self.method.urldata_init(self, d)
  File "/srv/pokybuild/yocto-worker/oe-selftest-armhost/build/layers/bitbake/lib/bb/fetch2/git.py", line 257, in urldata_init
    ud.setup_revisions(d)
  File "/srv/pokybuild/yocto-worker/oe-selftest-armhost/build/layers/bitbake/lib/bb/fetch2/__init__.py", line 1390, in setup_revisions
    self.revision = srcrev_internal_helper(self, d, self.name)
                    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/srv/pokybuild/yocto-worker/oe-selftest-armhost/build/layers/bitbake/lib/bb/fetch2/__init__.py", line 1251, in srcrev_internal_helper
    raise FetchError("Please set a valid SRCREV for url %s (possible key names are %s, or use a ;rev=X URL parameter)" % (str(attempts), ud.url), ud.url)
bb.fetch2.FetchError: Fetcher failure for URL: 'git://git.yoctoproject.org/git/dbus-wait;protocol=https;branch=master'. Please set a valid SRCREV for url ['SRCREV_default:pn-dbus-wait', 'SRCREV_default', 'SRCREV:pn-dbus-wait', 'SRCREV'] (possible key names are git://git.yoctoproject.org/git/dbus-wait;protocol=https;branch=master, or use a ;rev=X URL parameter)


https://autobuilder.yoctoproject.org/valkyrie/#/builders/23/builds/3490
https://autobuilder.yoctoproject.org/valkyrie/#/builders/35/builds/3371
https://autobuilder.yoctoproject.org/valkyrie/#/builders/48/builds/3261

Thanks,
Mathieu

-- 
Mathieu Dubois-Briand, Bootlin
Embedded Linux and Kernel engineering
https://bootlin.com