[PATCH 0/1] base.bbclass: Fix the recipe parsing race issue

[PATCH 0/1] base.bbclass: Fix the recipe parsing race issue

[Tony Rex]

It seems that the race condition happens when doing do_fetch on really small repos
and it can be difficult to reproduce so below the other case with floating tag/branch
that is easier to verify.

To verify the floating tag/branch issue have s simple recipe hello_1.0.bb and set the SRC_URI to
SRC_URI = "git://<repository>;protocol=https;user=${USER};branch=master;tag=v1.0"

run command bitbake hello everything builds.
 
Apply the patch

run command bitbake hello
ERROR: hello-1.0-r1 do_fetch: Bitbake Fetcher Error: FetchError("Recipe uses a floating tag/branch 'v1.0' for repo '<repository>' without a fixed SRCREV yet doesn't call bb.fetch2.get_srcrev() (use SRCPV in PV for OE).", None)
ERROR: Logfile of failure stored in: hello/1.0/temp/log.do_fetch.3289932
ERROR: Task (hello/hello_1.0.bb:do_fetch) failed with exit code '1'

Tony Rex (1):
  base.bbclass: Fix the recipe parsing race issue

 meta/classes-global/base.bbclass | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

[PATCH 1/1] base.bbclass: Fix the recipe parsing race issue

[Tony Rex]

There seems to be some settings that are not done in time in
base.bbclass when setting up the do_fetch function which leads
to parsing issues of the recipes:

bb.data_smart.ExpansionError: Failure expanding variable
fetcher_hashes_dummyfunc[vardepvalue]

Also by fixing the build error the floating tag/branch build error
is taking effect again:
FetchError("Recipe uses a floating tag/branch

Signed-off-by: Tony Rex <tony.rex@ericsson.com>
---
 meta/classes-global/base.bbclass | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/meta/classes-global/base.bbclass b/meta/classes-global/base.bbclass
index 6be1f5c2df..c955bb861f 100644
--- a/meta/classes-global/base.bbclass
+++ b/meta/classes-global/base.bbclass
@@ -147,7 +147,7 @@ def setup_hosttools_dir(dest, toolsvar, d, fatal=True):
 python fetcher_hashes_dummyfunc() {
     return
 }
-fetcher_hashes_dummyfunc[vardepvalue] = "${@bb.fetch.get_hashvalue(d)}"
+fetcher_hashes_dummyfunc[vardepvalue] := "${@bb.fetch.get_hashvalue(d)}"
 
 addtask fetch
 do_fetch[dirs] = "${DL_DIR}"

Re: [OE-core] [PATCH 1/1] base.bbclass: Fix the recipe parsing race issue

[Alexander Kanavin]

I don't understand. Does this patch trigger the fail or fix it? If it
fixes the fail, how can the fail be observed?

Alex

On Thu, 14 Aug 2025 at 21:05, Tony Rex via lists.openembedded.org
<tony.rex=ericsson.com@lists.openembedded.org> wrote:
>
> There seems to be some settings that are not done in time in
> base.bbclass when setting up the do_fetch function which leads
> to parsing issues of the recipes:
>
> bb.data_smart.ExpansionError: Failure expanding variable
> fetcher_hashes_dummyfunc[vardepvalue]
>
> Also by fixing the build error the floating tag/branch build error
> is taking effect again:
> FetchError("Recipe uses a floating tag/branch
>
> Signed-off-by: Tony Rex <tony.rex@ericsson.com>
> ---
>  meta/classes-global/base.bbclass | 2 +-
>  1 file changed, 1 insertion(+), 1 deletion(-)
>
> diff --git a/meta/classes-global/base.bbclass b/meta/classes-global/base.bbclass
> index 6be1f5c2df..c955bb861f 100644
> --- a/meta/classes-global/base.bbclass
> +++ b/meta/classes-global/base.bbclass
> @@ -147,7 +147,7 @@ def setup_hosttools_dir(dest, toolsvar, d, fatal=True):
>  python fetcher_hashes_dummyfunc() {
>      return
>  }
> -fetcher_hashes_dummyfunc[vardepvalue] = "${@bb.fetch.get_hashvalue(d)}"
> +fetcher_hashes_dummyfunc[vardepvalue] := "${@bb.fetch.get_hashvalue(d)}"
>
>  addtask fetch
>  do_fetch[dirs] = "${DL_DIR}"
>
> -=-=-=-=-=-=-=-=-=-=-=-
> Links: You receive all messages sent to this group.
> View/Reply Online (#221870): https://lists.openembedded.org/g/openembedded-core/message/221870
> Mute This Topic: https://lists.openembedded.org/mt/114706198/1686489
> Group Owner: openembedded-core+owner@lists.openembedded.org
> Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [alex.kanavin@gmail.com]
> -=-=-=-=-=-=-=-=-=-=-=-
>

Re: [OE-core] [PATCH 1/1] base.bbclass: Fix the recipe parsing race issue

[Tony Rex]

Hi Alex!

It fixes the faulty parsing and resolving variables in recipes and as a side effect the floating tag/branch protection is working as intended again.

I put the verification of the floating tag/branch and a way to test it in the cover-letter ( [PATCH 0/1] base.bbclass: Fix the recipe parsing race issue):

It seems that the race condition happens when doing do_fetch on really small repos and it can be difficult to reproduce so below the other case with floating tag/branch that is easier to verify.

To verify the floating tag/branch issue have s simple recipe hello_1.0.bb and set the SRC_URI to SRC_URI = "git://<repository>;protocol=https;user=${USER};branch=master;tag=v1.0"

run command bitbake hello everything builds.
 
Apply the patch

run command bitbake hello
ERROR: hello-1.0-r1 do_fetch: Bitbake Fetcher Error: FetchError("Recipe uses a floating tag/branch 'v1.0' for repo '<repository>' without a fixed SRCREV yet doesn't call bb.fetch2.get_srcrev() (use SRCPV in PV for OE).", None)
ERROR: Logfile of failure stored in: hello/1.0/temp/log.do_fetch.3289932
ERROR: Task (hello/hello_1.0.bb:do_fetch) failed with exit code '1'

 Tony

-----Original Message-----
From: Alexander Kanavin <alex.kanavin@gmail.com> 
Sent: Friday, 15 August 2025 11:05
To: Tony Rex <tony.rex@ericsson.com>
Cc: openembedded-core@lists.openembedded.org
Subject: Re: [OE-core] [PATCH 1/1] base.bbclass: Fix the recipe parsing race issue

[You don't often get email from alex.kanavin@gmail.com. Learn why this is important at https://aka.ms/LearnAboutSenderIdentification ]

I don't understand. Does this patch trigger the fail or fix it? If it fixes the fail, how can the fail be observed?

Alex

On Thu, 14 Aug 2025 at 21:05, Tony Rex via lists.openembedded.org <tony.rex=ericsson.com@lists.openembedded.org> wrote:
>
> There seems to be some settings that are not done in time in 
> base.bbclass when setting up the do_fetch function which leads to 
> parsing issues of the recipes:
>
> bb.data_smart.ExpansionError: Failure expanding variable 
> fetcher_hashes_dummyfunc[vardepvalue]
>
> Also by fixing the build error the floating tag/branch build error is 
> taking effect again:
> FetchError("Recipe uses a floating tag/branch
>
> Signed-off-by: Tony Rex <tony.rex@ericsson.com>
> ---
>  meta/classes-global/base.bbclass | 2 +-
>  1 file changed, 1 insertion(+), 1 deletion(-)
>
> diff --git a/meta/classes-global/base.bbclass 
> b/meta/classes-global/base.bbclass
> index 6be1f5c2df..c955bb861f 100644
> --- a/meta/classes-global/base.bbclass
> +++ b/meta/classes-global/base.bbclass
> @@ -147,7 +147,7 @@ def setup_hosttools_dir(dest, toolsvar, d, fatal=True):
>  python fetcher_hashes_dummyfunc() {
>      return
>  }
> -fetcher_hashes_dummyfunc[vardepvalue] = "${@bb.fetch.get_hashvalue(d)}"
> +fetcher_hashes_dummyfunc[vardepvalue] := "${@bb.fetch.get_hashvalue(d)}"
>
>  addtask fetch
>  do_fetch[dirs] = "${DL_DIR}"
>
> 
>

Re: [OE-core] [PATCH 0/1] base.bbclass: Fix the recipe parsing race issue

[Mathieu Dubois-Briand]

On Thu Aug 14, 2025 at 7:23 PM CEST, Tony Rex via lists.openembedded.org wrote:
> It seems that the race condition happens when doing do_fetch on really small repos
> and it can be difficult to reproduce so below the other case with floating tag/branch
> that is easier to verify.
>
> To verify the floating tag/branch issue have s simple recipe hello_1.0.bb and set the SRC_URI to
> SRC_URI = "git://<repository>;protocol=https;user=${USER};branch=master;tag=v1.0"
>
> run command bitbake hello everything builds.
>  
> Apply the patch
>
> run command bitbake hello
> ERROR: hello-1.0-r1 do_fetch: Bitbake Fetcher Error: FetchError("Recipe uses a floating tag/branch 'v1.0' for repo '<repository>' without a fixed SRCREV yet doesn't call bb.fetch2.get_srcrev() (use SRCPV in PV for OE).", None)
> ERROR: Logfile of failure stored in: hello/1.0/temp/log.do_fetch.3289932
> ERROR: Task (hello/hello_1.0.bb:do_fetch) failed with exit code '1'
>
> Tony Rex (1):
>   base.bbclass: Fix the recipe parsing race issue
>
>  meta/classes-global/base.bbclass | 2 +-
>  1 file changed, 1 insertion(+), 1 deletion(-)

Hi Tony,

Thanks for your patch.

It looks like this is breaking some builds with the following error:

oeqa.utils.subprocesstweak.OETestCalledProcessError: Command '. /srv/pokybuild/yocto-worker/qemux86-64/build/build/tmp/work/qemux86_64-poky-linux/core-image-sato/1.0/testsdkext/environment-setup-core2-64-poky-linux > /dev/null; devtool add kernel-module-hello-world https://git.yoctoproject.org/git/kernel-module-hello-world;' returned non-zero exit status 1.
...
ERROR: Bitbake Fetcher Error: FetchError("Recipe uses a floating tag/branch 'master' for repo 'git.yoctoproject.org/git/kernel-module-hello-world' without a fixed SRCREV yet doesn't call bb.fetch2.get_srcrev() (use SRCPV in PV for OE).", None)

https://autobuilder.yoctoproject.org/valkyrie/#/builders/68/builds/2284
https://autobuilder.yoctoproject.org/valkyrie/#/builders/30/builds/2199
https://autobuilder.yoctoproject.org/valkyrie/#/builders/16/builds/2247

Can you have a look at this error please?

Best regards,
Mathieu

-- 
Mathieu Dubois-Briand, Bootlin
Embedded Linux and Kernel engineering
https://bootlin.com

Re: [OE-core] [PATCH 0/1] base.bbclass: Fix the recipe parsing race issue

[Tony Rex]

Hi Mathieu!

I think that this is how it is intended to work, bitbake does not allow floating tag/branch anymore.
Please correct me if I'm wrong @Richard Purdie (who implemented this for security reasons I think.)

The patch that I have makes sure that the parsing and setting of the variables in the recipe are done before executing the do_fetch command,
and as a side effect the floating tag/branch protection is working as intended again.

How does the recipe look like?
devtool add kernel-module-hello-world https://git.yoctoproject.org/git/kernel-module-hello-world
The one I get looks like this:

cat build/workspace/recipes/kernel-module-hello-world/kernel-module-hello-world_git.bb
# Recipe created by recipetool
# This is the basis of a recipe and may need further editing in order to be fully functional.
# (Feel free to remove these comments when editing.)

# WARNING: the following LICENSE and LIC_FILES_CHKSUM values are best guesses - it is
# your responsibility to verify that the values are complete and correct.
#
# The following license files were not able to be identified and are
# represented as "Unknown" below, you will need to check them yourself:
#   LICENSE
LICENSE = "Unknown"
LIC_FILES_CHKSUM = "file://LICENSE;md5=7b785be6433a2e4af6a641cc7f0330d1"

SRC_URI = "git://git.yoctoproject.org/git/kernel-module-hello-world;protocol=https;branch=master"

# Modify these as desired
PV = "1.0+git"
SRCREV = "b66f0f2bd559a1df81fc1bda240eba07a2ecdd2b"

S = "${WORKDIR}/git"

inherit module

EXTRA_OEMAKE:append:task-install = " -C ${STAGING_KERNEL_DIR} M=${S}"
EXTRA_OEMAKE += "KERNEL_DIR=${STAGING_KERNEL_DIR}"

And it contains SRCREV = "b66f0f2bd559a1df81fc1bda240eba07a2ecdd2b"

Tony

-----Original Message-----
From: Mathieu Dubois-Briand <mathieu.dubois-briand@bootlin.com>
Sent: Friday, 15 August 2025 11:26
To: Tony Rex <tony.rex@ericsson.com>; openembedded-core@lists.openembedded.org
Subject: Re: [OE-core] [PATCH 0/1] base.bbclass: Fix the recipe parsing race issue

[You don't often get email from mathieu.dubois-briand@bootlin.com. Learn why this is important at https://aka.ms/LearnAboutSenderIdentification ]

On Thu Aug 14, 2025 at 7:23 PM CEST, Tony Rex via lists.openembedded.org wrote:
> It seems that the race condition happens when doing do_fetch on really
> small repos and it can be difficult to reproduce so below the other
> case with floating tag/branch that is easier to verify.
>
> To verify the floating tag/branch issue have s simple recipe
> hello_1.0.bb and set the SRC_URI to SRC_URI = "git://<repository>;protocol=https;user=${USER};branch=master;tag=v1.0"
>
> run command bitbake hello everything builds.
>
> Apply the patch
>
> run command bitbake hello
> ERROR: hello-1.0-r1 do_fetch: Bitbake Fetcher Error:
> FetchError("Recipe uses a floating tag/branch 'v1.0' for repo
> '<repository>' without a fixed SRCREV yet doesn't call
> bb.fetch2.get_srcrev() (use SRCPV in PV for OE).", None)
> ERROR: Logfile of failure stored in:
> hello/1.0/temp/log.do_fetch.3289932
> ERROR: Task (hello/hello_1.0.bb:do_fetch) failed with exit code '1'
>
> Tony Rex (1):
>   base.bbclass: Fix the recipe parsing race issue
>
>  meta/classes-global/base.bbclass | 2 +-
>  1 file changed, 1 insertion(+), 1 deletion(-)

Hi Tony,

Thanks for your patch.

It looks like this is breaking some builds with the following error:

oeqa.utils.subprocesstweak.OETestCalledProcessError: Command '. /srv/pokybuild/yocto-worker/qemux86-64/build/build/tmp/work/qemux86_64-poky-linux/core-image-sato/1.0/testsdkext/environment-setup-core2-64-poky-linux > /dev/null; devtool add kernel-module-hello-world https://git.yoctoproject.org/git/kernel-module-hello-world;' returned non-zero exit status 1.
...
ERROR: Bitbake Fetcher Error: FetchError("Recipe uses a floating tag/branch 'master' for repo 'git.yoctoproject.org/git/kernel-module-hello-world' without a fixed SRCREV yet doesn't call bb.fetch2.get_srcrev() (use SRCPV in PV for OE).", None)

https://autobuilder.yoctoproject.org/valkyrie/#/builders/68/builds/2284
https://autobuilder.yoctoproject.org/valkyrie/#/builders/30/builds/2199
https://autobuilder.yoctoproject.org/valkyrie/#/builders/16/builds/2247

Can you have a look at this error please?

Best regards,
Mathieu

--
Mathieu Dubois-Briand, Bootlin
Embedded Linux and Kernel engineering
https://bootlin.com/

Re: [OE-core] [PATCH 0/1] base.bbclass: Fix the recipe parsing race issue

[Alexander Kanavin]

You need to make things a little bit easier for maintainers and try to
save their time.

I would suggest that you provide clear steps to observe the issue.
Attach the recipe file that would demonstrate the issue, and provide
clear steps to trigger it. What is confusing right now is that you say
here's a recipe that builds ok, but then you 'apply the patch' and
then there's some kind of failure. So which patch needs to be applied?

Alex

On Fri, 15 Aug 2025 at 17:29, Tony Rex via lists.openembedded.org
<tony.rex=ericsson.com@lists.openembedded.org> wrote:
>
> Hi Mathieu!
>
> I think that this is how it is intended to work, bitbake does not allow floating tag/branch anymore.
> Please correct me if I'm wrong @Richard Purdie (who implemented this for security reasons I think.)
>
> The patch that I have makes sure that the parsing and setting of the variables in the recipe are done before executing the do_fetch command,
> and as a side effect the floating tag/branch protection is working as intended again.
>
> How does the recipe look like?
> devtool add kernel-module-hello-world https://git.yoctoproject.org/git/kernel-module-hello-world
> The one I get looks like this:
>
> cat build/workspace/recipes/kernel-module-hello-world/kernel-module-hello-world_git.bb
> # Recipe created by recipetool
> # This is the basis of a recipe and may need further editing in order to be fully functional.
> # (Feel free to remove these comments when editing.)
>
> # WARNING: the following LICENSE and LIC_FILES_CHKSUM values are best guesses - it is
> # your responsibility to verify that the values are complete and correct.
> #
> # The following license files were not able to be identified and are
> # represented as "Unknown" below, you will need to check them yourself:
> #   LICENSE
> LICENSE = "Unknown"
> LIC_FILES_CHKSUM = "file://LICENSE;md5=7b785be6433a2e4af6a641cc7f0330d1"
>
> SRC_URI = "git://git.yoctoproject.org/git/kernel-module-hello-world;protocol=https;branch=master"
>
> # Modify these as desired
> PV = "1.0+git"
> SRCREV = "b66f0f2bd559a1df81fc1bda240eba07a2ecdd2b"
>
> S = "${WORKDIR}/git"
>
> inherit module
>
> EXTRA_OEMAKE:append:task-install = " -C ${STAGING_KERNEL_DIR} M=${S}"
> EXTRA_OEMAKE += "KERNEL_DIR=${STAGING_KERNEL_DIR}"
>
> And it contains SRCREV = "b66f0f2bd559a1df81fc1bda240eba07a2ecdd2b"
>
> Tony
>
> -----Original Message-----
> From: Mathieu Dubois-Briand <mathieu.dubois-briand@bootlin.com>
> Sent: Friday, 15 August 2025 11:26
> To: Tony Rex <tony.rex@ericsson.com>; openembedded-core@lists.openembedded.org
> Subject: Re: [OE-core] [PATCH 0/1] base.bbclass: Fix the recipe parsing race issue
>
> [You don't often get email from mathieu.dubois-briand@bootlin.com. Learn why this is important at https://aka.ms/LearnAboutSenderIdentification ]
>
> On Thu Aug 14, 2025 at 7:23 PM CEST, Tony Rex via lists.openembedded.org wrote:
> > It seems that the race condition happens when doing do_fetch on really
> > small repos and it can be difficult to reproduce so below the other
> > case with floating tag/branch that is easier to verify.
> >
> > To verify the floating tag/branch issue have s simple recipe
> > hello_1.0.bb and set the SRC_URI to SRC_URI = "git://<repository>;protocol=https;user=${USER};branch=master;tag=v1.0"
> >
> > run command bitbake hello everything builds.
> >
> > Apply the patch
> >
> > run command bitbake hello
> > ERROR: hello-1.0-r1 do_fetch: Bitbake Fetcher Error:
> > FetchError("Recipe uses a floating tag/branch 'v1.0' for repo
> > '<repository>' without a fixed SRCREV yet doesn't call
> > bb.fetch2.get_srcrev() (use SRCPV in PV for OE).", None)
> > ERROR: Logfile of failure stored in:
> > hello/1.0/temp/log.do_fetch.3289932
> > ERROR: Task (hello/hello_1.0.bb:do_fetch) failed with exit code '1'
> >
> > Tony Rex (1):
> >   base.bbclass: Fix the recipe parsing race issue
> >
> >  meta/classes-global/base.bbclass | 2 +-
> >  1 file changed, 1 insertion(+), 1 deletion(-)
>
> Hi Tony,
>
> Thanks for your patch.
>
> It looks like this is breaking some builds with the following error:
>
> oeqa.utils.subprocesstweak.OETestCalledProcessError: Command '. /srv/pokybuild/yocto-worker/qemux86-64/build/build/tmp/work/qemux86_64-poky-linux/core-image-sato/1.0/testsdkext/environment-setup-core2-64-poky-linux > /dev/null; devtool add kernel-module-hello-world https://git.yoctoproject.org/git/kernel-module-hello-world;' returned non-zero exit status 1.
> ...
> ERROR: Bitbake Fetcher Error: FetchError("Recipe uses a floating tag/branch 'master' for repo 'git.yoctoproject.org/git/kernel-module-hello-world' without a fixed SRCREV yet doesn't call bb.fetch2.get_srcrev() (use SRCPV in PV for OE).", None)
>
> https://autobuilder.yoctoproject.org/valkyrie/#/builders/68/builds/2284
> https://autobuilder.yoctoproject.org/valkyrie/#/builders/30/builds/2199
> https://autobuilder.yoctoproject.org/valkyrie/#/builders/16/builds/2247
>
> Can you have a look at this error please?
>
> Best regards,
> Mathieu
>
> --
> Mathieu Dubois-Briand, Bootlin
> Embedded Linux and Kernel engineering
> https://bootlin.com/
>
>
> -=-=-=-=-=-=-=-=-=-=-=-
> Links: You receive all messages sent to this group.
> View/Reply Online (#221976): https://lists.openembedded.org/g/openembedded-core/message/221976
> Mute This Topic: https://lists.openembedded.org/mt/114706197/1686489
> Group Owner: openembedded-core+owner@lists.openembedded.org
> Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [alex.kanavin@gmail.com]
> -=-=-=-=-=-=-=-=-=-=-=-
>

Re: [OE-core] [PATCH 0/1] base.bbclass: Fix the recipe parsing race issue

[Tony Rex]

Hi Alex and Mathieu!

Ok, so the patch fixes a bug and then the floating tag/branch function is in place again.
The floating tag/branch function is broken due to the bug and the patch fixes it.

The mail from Mathieu below might not fully reflect the issue here, since it seems to create a recipe with the help of devtool.

To verify:
1. add layers/oe-core/meta-selftest to conf/bblayers.conf
2. run bitbake gitrepotest -c fetch
result:
NOTE: Tasks Summary: Attempted 2 tasks of which 0 didn't need to be rerun and all succeeded.
3. run bitbake gitrepotest -c cleanall
4. edit ../layers/oe-core/meta-selftest/recipes-test/gitrepotest/gitrepotest.bb
add tag=2.12 in SRC_URI
SRC_URI = "git://git.yoctoproject.org/git/matchbox-panel-2;branch=master;tag=2.12;protocol=https \
and comment out SRCREV
#SRCREV = "f82ca3f42510fb3ef10f598b393eb373a2c34ca7"
5. run bitbake gitrepotest -c fetch
result:
NOTE: Tasks Summary: Attempted 2 tasks of which 0 didn't need to be rerun and all succeeded.
The fetch goes through although this is not allowed according to the floating tag/branch error statement if not d.getVar("__BBSRCREV_SEEN"): in git.py.
6. run bitbake gitrepotest -c cleanall
7. apply patch 0001-base.bbclass-Fix-the-recipe-parsing-race-issue.patch
8. run bitbake gitrepotest -c fetch
result:
ERROR: gitrepotest-1.0-r0 do_fetch: Bitbake Fetcher Error: FetchError("Recipe uses a floating tag/branch '2.12' for repo 'git.yoctoproject.org/git/matchbox-panel-2' without a fixed SRCREV yet doesn't call bb.fetch2.get_srcrev() (use SRCPV in PV for OE).", None)
ERROR: Logfile of failure stored in: .../build/tmp-glibc/work/cortexa53-wrs-linux/gitrepotest/1.0/temp/log.do_fetch.1254999
ERROR: Task (.../layers/oe-core/meta-selftest/recipes-test/gitrepotest/gitrepotest.bb:do_fetch) failed with exit code '1'
NOTE: Tasks Summary: Attempted 2 tasks of which 0 didn't need to be rerun and 1 failed.

Error is the expected result when only having a tag set.

Tony

-----Original Message-----
From: Alexander Kanavin <alex.kanavin@gmail.com>
Sent: Tuesday, 19 August 2025 10:53
To: Tony Rex <tony.rex@ericsson.com>
Cc: Mathieu Dubois-Briand <mathieu.dubois-briand@bootlin.com>; openembedded-core@lists.openembedded.org
Subject: Re: [OE-core] [PATCH 0/1] base.bbclass: Fix the recipe parsing race issue

[You don't often get email from alex.kanavin@gmail.com. Learn why this is important at https://aka.ms/LearnAboutSenderIdentification ]

You need to make things a little bit easier for maintainers and try to save their time.

I would suggest that you provide clear steps to observe the issue.
Attach the recipe file that would demonstrate the issue, and provide clear steps to trigger it. What is confusing right now is that you say here's a recipe that builds ok, but then you 'apply the patch' and then there's some kind of failure. So which patch needs to be applied?

Alex

On Fri, 15 Aug 2025 at 17:29, Tony Rex via lists.openembedded.org <tony.rex=ericsson.com@lists.openembedded.org> wrote:
>
> Hi Mathieu!
>
> I think that this is how it is intended to work, bitbake does not allow floating tag/branch anymore.
> Please correct me if I'm wrong @Richard Purdie (who implemented this
> for security reasons I think.)
>
> The patch that I have makes sure that the parsing and setting of the
> variables in the recipe are done before executing the do_fetch command, and as a side effect the floating tag/branch protection is working as intended again.
>
> How does the recipe look like?
> devtool add kernel-module-hello-world
> https://git/.
> yoctoproject.org%2Fgit%2Fkernel-module-hello-world&data=05%7C02%7Ctony
> .rex%40ericsson.com%7Cde1ae2aceeda470b381608dddefdd730%7C92e84cebfbfd4
> 7abbe52080c6b87953f%7C0%7C0%7C638911904005151487%7CUnknown%7CTWFpbGZsb
> 3d8eyJFbXB0eU1hcGkiOnRydWUsIlYiOiIwLjAuMDAwMCIsIlAiOiJXaW4zMiIsIkFOIjo
> iTWFpbCIsIldUIjoyfQ%3D%3D%7C0%7C%7C%7C&sdata=jyCicRQT%2FLtb1YxiFHKgw5B
> nNktN4t9IciiyxtsEwO4%3D&reserved=0
> The one I get looks like this:
>
> cat
> build/workspace/recipes/kernel-module-hello-world/kernel-module-hello-
> world_git.bb
> # Recipe created by recipetool
> # This is the basis of a recipe and may need further editing in order to be fully functional.
> # (Feel free to remove these comments when editing.)
>
> # WARNING: the following LICENSE and LIC_FILES_CHKSUM values are best
> guesses - it is # your responsibility to verify that the values are complete and correct.
> #
> # The following license files were not able to be identified and are #
> represented as "Unknown" below, you will need to check them yourself:
> #   LICENSE
> LICENSE = "Unknown"
> LIC_FILES_CHKSUM = "file://LICENSE;md5=7b785be6433a2e4af6a641cc7f0330d1"
>
> SRC_URI = "git://git.yoctoproject.org/git/kernel-module-hello-world;protocol=https;branch=master"
>
> # Modify these as desired
> PV = "1.0+git"
> SRCREV = "b66f0f2bd559a1df81fc1bda240eba07a2ecdd2b"
>
> S = "${WORKDIR}/git"
>
> inherit module
>
> EXTRA_OEMAKE:append:task-install = " -C ${STAGING_KERNEL_DIR} M=${S}"
> EXTRA_OEMAKE += "KERNEL_DIR=${STAGING_KERNEL_DIR}"
>
> And it contains SRCREV = "b66f0f2bd559a1df81fc1bda240eba07a2ecdd2b"
>
> Tony
>
> -----Original Message-----
> From: Mathieu Dubois-Briand <mathieu.dubois-briand@bootlin.com>
> Sent: Friday, 15 August 2025 11:26
> To: Tony Rex <tony.rex@ericsson.com>;
> openembedded-core@lists.openembedded.org
> Subject: Re: [OE-core] [PATCH 0/1] base.bbclass: Fix the recipe
> parsing race issue
>
> [You don't often get email from mathieu.dubois-briand@bootlin.com.
> Learn why this is important at
> https://aka.ms/LearnAboutSenderIdentification ]
>
> On Thu Aug 14, 2025 at 7:23 PM CEST, Tony Rex via lists.openembedded.org wrote:
> > It seems that the race condition happens when doing do_fetch on
> > really small repos and it can be difficult to reproduce so below the
> > other case with floating tag/branch that is easier to verify.
> >
> > To verify the floating tag/branch issue have s simple recipe
> > hello_1.0.bb and set the SRC_URI to SRC_URI = "git://<repository>;protocol=https;user=${USER};branch=master;tag=v1.0"
> >
> > run command bitbake hello everything builds.
> >
> > Apply the patch
> >
> > run command bitbake hello
> > ERROR: hello-1.0-r1 do_fetch: Bitbake Fetcher Error:
> > FetchError("Recipe uses a floating tag/branch 'v1.0' for repo
> > '<repository>' without a fixed SRCREV yet doesn't call
> > bb.fetch2.get_srcrev() (use SRCPV in PV for OE).", None)
> > ERROR: Logfile of failure stored in:
> > hello/1.0/temp/log.do_fetch.3289932
> > ERROR: Task (hello/hello_1.0.bb:do_fetch) failed with exit code '1'
> >
> > Tony Rex (1):
> >   base.bbclass: Fix the recipe parsing race issue
> >
> >  meta/classes-global/base.bbclass | 2 +-
> >  1 file changed, 1 insertion(+), 1 deletion(-)
>
> Hi Tony,
>
> Thanks for your patch.
>
> It looks like this is breaking some builds with the following error:
>
> oeqa.utils.subprocesstweak.OETestCalledProcessError: Command '. /srv/pokybuild/yocto-worker/qemux86-64/build/build/tmp/work/qemux86_64-poky-linux/core-image-sato/1.0/testsdkext/environment-setup-core2-64-poky-linux > /dev/null; devtool add kernel-module-hello-world https://git.yoctoproject.org/git/kernel-module-hello-world;' returned non-zero exit status 1.
> ...
> ERROR: Bitbake Fetcher Error: FetchError("Recipe uses a floating
> tag/branch 'master' for repo
> 'git.yoctoproject.org/git/kernel-module-hello-world' without a fixed
> SRCREV yet doesn't call bb.fetch2.get_srcrev() (use SRCPV in PV for
> OE).", None)
>
> https://auto/
> builder.yoctoproject.org%2Fvalkyrie%2F%23%2Fbuilders%2F68%2Fbuilds%2F2
> 284&data=05%7C02%7Ctony.rex%40ericsson.com%7Cde1ae2aceeda470b381608ddd
> efdd730%7C92e84cebfbfd47abbe52080c6b87953f%7C0%7C0%7C63891190400520900
> 3%7CUnknown%7CTWFpbGZsb3d8eyJFbXB0eU1hcGkiOnRydWUsIlYiOiIwLjAuMDAwMCIs
> IlAiOiJXaW4zMiIsIkFOIjoiTWFpbCIsIldUIjoyfQ%3D%3D%7C0%7C%7C%7C&sdata=mP
> UXMdb%2FYOyGIm0rnncaWgcqtwkUIh8364IP3eYDPe4%3D&reserved=0
> https://auto/
> builder.yoctoproject.org%2Fvalkyrie%2F%23%2Fbuilders%2F30%2Fbuilds%2F2
> 199&data=05%7C02%7Ctony.rex%40ericsson.com%7Cde1ae2aceeda470b381608ddd
> efdd730%7C92e84cebfbfd47abbe52080c6b87953f%7C0%7C0%7C63891190400522743
> 2%7CUnknown%7CTWFpbGZsb3d8eyJFbXB0eU1hcGkiOnRydWUsIlYiOiIwLjAuMDAwMCIs
> IlAiOiJXaW4zMiIsIkFOIjoiTWFpbCIsIldUIjoyfQ%3D%3D%7C0%7C%7C%7C&sdata=R0
> HFameGl2LmIVMT9ml%2BDMnj%2FMbG6SW0%2F3yNZHQu2js%3D&reserved=0
> https://auto/
> builder.yoctoproject.org%2Fvalkyrie%2F%23%2Fbuilders%2F16%2Fbuilds%2F2
> 247&data=05%7C02%7Ctony.rex%40ericsson.com%7Cde1ae2aceeda470b381608ddd
> efdd730%7C92e84cebfbfd47abbe52080c6b87953f%7C0%7C0%7C63891190400524538
> 7%7CUnknown%7CTWFpbGZsb3d8eyJFbXB0eU1hcGkiOnRydWUsIlYiOiIwLjAuMDAwMCIs
> IlAiOiJXaW4zMiIsIkFOIjoiTWFpbCIsIldUIjoyfQ%3D%3D%7C0%7C%7C%7C&sdata=NS
> FhEupwMkgqUAhpRgFbKnyFl%2FsY%2B4UJMhVQHhqOaNM%3D&reserved=0
>
> Can you have a look at this error please?
>
> Best regards,
> Mathieu
>
> --
> Mathieu Dubois-Briand, Bootlin
> Embedded Linux and Kernel engineering
> https://boot/
> lin.com%2F&data=05%7C02%7Ctony.rex%40ericsson.com%7Cde1ae2aceeda470b38
> 1608dddefdd730%7C92e84cebfbfd47abbe52080c6b87953f%7C0%7C0%7C6389119040
> 05263327%7CUnknown%7CTWFpbGZsb3d8eyJFbXB0eU1hcGkiOnRydWUsIlYiOiIwLjAuM
> DAwMCIsIlAiOiJXaW4zMiIsIkFOIjoiTWFpbCIsIldUIjoyfQ%3D%3D%7C0%7C%7C%7C&s
> data=oyi7YWsXDVHn7QB4fDdG1sBp7gQsfGWcJTWtX1CjAw4%3D&reserved=0
>
>
> 
>

Re: [OE-core] [PATCH 0/1] base.bbclass: Fix the recipe parsing race issue

[Alexander Kanavin]

On Tue, 19 Aug 2025 at 23:10, Tony Rex <tony.rex@ericsson.com> wrote:
> Ok, so the patch fixes a bug and then the floating tag/branch function is in place again.
> The floating tag/branch function is broken due to the bug and the patch fixes it.

Right. But this is different to a 'recipe parsing race issue'. Does
this patch fix the problem?

https://git.yoctoproject.org/poky/commit/?id=3d451f3452643a6fcaa7ff9345a6f955392daec3

Alex

Re: [OE-core] [PATCH 0/1] base.bbclass: Fix the recipe parsing race issue

[Tony Rex]

No the patch does not seem to fix the problem, I see the same behavior regarding the floating tag/branch, but the patch I provided makes the floating tag/branch guard work.

I'm sorry that I only seem able to reproduce the floating tag/branch issue and not the race condition issue, which I have so far only manage to reproduce and expose locally, but I'm looking into that as well and trying to come up with a way to catch it in a more common scenario.

Tony

-----Original Message-----
From: Alexander Kanavin <alex.kanavin@gmail.com>
Sent: Wednesday, 20 August 2025 09:16
To: Tony Rex <tony.rex@ericsson.com>
Cc: Mathieu Dubois-Briand <mathieu.dubois-briand@bootlin.com>; openembedded-core@lists.openembedded.org
Subject: Re: [OE-core] [PATCH 0/1] base.bbclass: Fix the recipe parsing race issue

On Tue, 19 Aug 2025 at 23:10, Tony Rex <tony.rex@ericsson.com> wrote:
> Ok, so the patch fixes a bug and then the floating tag/branch function is in place again.
> The floating tag/branch function is broken due to the bug and the patch fixes it.

Right. But this is different to a 'recipe parsing race issue'. Does this patch fix the problem?

https://git.yoctoproject.org/poky/commit/?id=3d451f3452643a6fcaa7ff9345a6f955392daec3

Alex

Re: [OE-core] [PATCH 0/1] base.bbclass: Fix the recipe parsing race issue

[Gyorgy Sarvari]

On 8/19/25 23:10, Tony Rex via lists.openembedded.org wrote:
> Hi Alex and Mathieu!
>
> Ok, so the patch fixes a bug and then the floating tag/branch function is in place again.
> The floating tag/branch function is broken due to the bug and the patch fixes it.
>
> The mail from Mathieu below might not fully reflect the issue here, since it seems to create a recipe with the help of devtool.
>
> To verify:
> 1. add layers/oe-core/meta-selftest to conf/bblayers.conf
> 2. run bitbake gitrepotest -c fetch
> result:
> NOTE: Tasks Summary: Attempted 2 tasks of which 0 didn't need to be rerun and all succeeded.
> 3. run bitbake gitrepotest -c cleanall
> 4. edit ../layers/oe-core/meta-selftest/recipes-test/gitrepotest/gitrepotest.bb
> add tag=2.12 in SRC_URI
> SRC_URI = "git://git.yoctoproject.org/git/matchbox-panel-2;branch=master;tag=2.12;protocol=https \
> and comment out SRCREV
> #SRCREV = "f82ca3f42510fb3ef10f598b393eb373a2c34ca7"
> 5. run bitbake gitrepotest -c fetch
> result:
> NOTE: Tasks Summary: Attempted 2 tasks of which 0 didn't need to be rerun and all succeeded.
> The fetch goes through although this is not allowed according to the floating tag/branch error statement if not d.getVar("__BBSRCREV_SEEN"): in git.py.

Could you point to a commit/email/etc about this "no floating tags
allowed" statement? AFAIK tags without specifying revision is allowed
and supported. Not recommended, due to mandatory network access, and
because git tags are not immutable, but I'm surprised to hear that they
are not supposed to work (which of course can also mean that I just
missed a memo)

> 6. run bitbake gitrepotest -c cleanall
> 7. apply patch 0001-base.bbclass-Fix-the-recipe-parsing-race-issue.patch
> 8. run bitbake gitrepotest -c fetch
> result:
> ERROR: gitrepotest-1.0-r0 do_fetch: Bitbake Fetcher Error: FetchError("Recipe uses a floating tag/branch '2.12' for repo 'git.yoctoproject.org/git/matchbox-panel-2' without a fixed SRCREV yet doesn't call bb.fetch2.get_srcrev() (use SRCPV in PV for OE).", None)
> ERROR: Logfile of failure stored in: .../build/tmp-glibc/work/cortexa53-wrs-linux/gitrepotest/1.0/temp/log.do_fetch.1254999
> ERROR: Task (.../layers/oe-core/meta-selftest/recipes-test/gitrepotest/gitrepotest.bb:do_fetch) failed with exit code '1'
> NOTE: Tasks Summary: Attempted 2 tasks of which 0 didn't need to be rerun and 1 failed.
>
> Error is the expected result when only having a tag set.
>
> Tony
>
> -----Original Message-----
> From: Alexander Kanavin <alex.kanavin@gmail.com>
> Sent: Tuesday, 19 August 2025 10:53
> To: Tony Rex <tony.rex@ericsson.com>
> Cc: Mathieu Dubois-Briand <mathieu.dubois-briand@bootlin.com>; openembedded-core@lists.openembedded.org
> Subject: Re: [OE-core] [PATCH 0/1] base.bbclass: Fix the recipe parsing race issue
>
> [You don't often get email from alex.kanavin@gmail.com. Learn why this is important at https://aka.ms/LearnAboutSenderIdentification ]
>
> You need to make things a little bit easier for maintainers and try to save their time.
>
> I would suggest that you provide clear steps to observe the issue.
> Attach the recipe file that would demonstrate the issue, and provide clear steps to trigger it. What is confusing right now is that you say here's a recipe that builds ok, but then you 'apply the patch' and then there's some kind of failure. So which patch needs to be applied?
>
> Alex
>
> On Fri, 15 Aug 2025 at 17:29, Tony Rex via lists.openembedded.org <tony.rex=ericsson.com@lists.openembedded.org> wrote:
>> Hi Mathieu!
>>
>> I think that this is how it is intended to work, bitbake does not allow floating tag/branch anymore.
>> Please correct me if I'm wrong @Richard Purdie (who implemented this
>> for security reasons I think.)
>>
>> The patch that I have makes sure that the parsing and setting of the
>> variables in the recipe are done before executing the do_fetch command, and as a side effect the floating tag/branch protection is working as intended again.
>>
>> How does the recipe look like?
>> devtool add kernel-module-hello-world
>> https://git/.
>> yoctoproject.org%2Fgit%2Fkernel-module-hello-world&data=05%7C02%7Ctony
>> .rex%40ericsson.com%7Cde1ae2aceeda470b381608dddefdd730%7C92e84cebfbfd4
>> 7abbe52080c6b87953f%7C0%7C0%7C638911904005151487%7CUnknown%7CTWFpbGZsb
>> 3d8eyJFbXB0eU1hcGkiOnRydWUsIlYiOiIwLjAuMDAwMCIsIlAiOiJXaW4zMiIsIkFOIjo
>> iTWFpbCIsIldUIjoyfQ%3D%3D%7C0%7C%7C%7C&sdata=jyCicRQT%2FLtb1YxiFHKgw5B
>> nNktN4t9IciiyxtsEwO4%3D&reserved=0
>> The one I get looks like this:
>>
>> cat
>> build/workspace/recipes/kernel-module-hello-world/kernel-module-hello-
>> world_git.bb
>> # Recipe created by recipetool
>> # This is the basis of a recipe and may need further editing in order to be fully functional.
>> # (Feel free to remove these comments when editing.)
>>
>> # WARNING: the following LICENSE and LIC_FILES_CHKSUM values are best
>> guesses - it is # your responsibility to verify that the values are complete and correct.
>> #
>> # The following license files were not able to be identified and are #
>> represented as "Unknown" below, you will need to check them yourself:
>> #   LICENSE
>> LICENSE = "Unknown"
>> LIC_FILES_CHKSUM = "file://LICENSE;md5=7b785be6433a2e4af6a641cc7f0330d1"
>>
>> SRC_URI = "git://git.yoctoproject.org/git/kernel-module-hello-world;protocol=https;branch=master"
>>
>> # Modify these as desired
>> PV = "1.0+git"
>> SRCREV = "b66f0f2bd559a1df81fc1bda240eba07a2ecdd2b"
>>
>> S = "${WORKDIR}/git"
>>
>> inherit module
>>
>> EXTRA_OEMAKE:append:task-install = " -C ${STAGING_KERNEL_DIR} M=${S}"
>> EXTRA_OEMAKE += "KERNEL_DIR=${STAGING_KERNEL_DIR}"
>>
>> And it contains SRCREV = "b66f0f2bd559a1df81fc1bda240eba07a2ecdd2b"
>>
>> Tony
>>
>> -----Original Message-----
>> From: Mathieu Dubois-Briand <mathieu.dubois-briand@bootlin.com>
>> Sent: Friday, 15 August 2025 11:26
>> To: Tony Rex <tony.rex@ericsson.com>;
>> openembedded-core@lists.openembedded.org
>> Subject: Re: [OE-core] [PATCH 0/1] base.bbclass: Fix the recipe
>> parsing race issue
>>
>> [You don't often get email from mathieu.dubois-briand@bootlin.com.
>> Learn why this is important at
>> https://aka.ms/LearnAboutSenderIdentification ]
>>
>> On Thu Aug 14, 2025 at 7:23 PM CEST, Tony Rex via lists.openembedded.org wrote:
>>> It seems that the race condition happens when doing do_fetch on
>>> really small repos and it can be difficult to reproduce so below the
>>> other case with floating tag/branch that is easier to verify.
>>>
>>> To verify the floating tag/branch issue have s simple recipe
>>> hello_1.0.bb and set the SRC_URI to SRC_URI = "git://<repository>;protocol=https;user=${USER};branch=master;tag=v1.0"
>>>
>>> run command bitbake hello everything builds.
>>>
>>> Apply the patch
>>>
>>> run command bitbake hello
>>> ERROR: hello-1.0-r1 do_fetch: Bitbake Fetcher Error:
>>> FetchError("Recipe uses a floating tag/branch 'v1.0' for repo
>>> '<repository>' without a fixed SRCREV yet doesn't call
>>> bb.fetch2.get_srcrev() (use SRCPV in PV for OE).", None)
>>> ERROR: Logfile of failure stored in:
>>> hello/1.0/temp/log.do_fetch.3289932
>>> ERROR: Task (hello/hello_1.0.bb:do_fetch) failed with exit code '1'
>>>
>>> Tony Rex (1):
>>>   base.bbclass: Fix the recipe parsing race issue
>>>
>>>  meta/classes-global/base.bbclass | 2 +-
>>>  1 file changed, 1 insertion(+), 1 deletion(-)
>> Hi Tony,
>>
>> Thanks for your patch.
>>
>> It looks like this is breaking some builds with the following error:
>>
>> oeqa.utils.subprocesstweak.OETestCalledProcessError: Command '. /srv/pokybuild/yocto-worker/qemux86-64/build/build/tmp/work/qemux86_64-poky-linux/core-image-sato/1.0/testsdkext/environment-setup-core2-64-poky-linux > /dev/null; devtool add kernel-module-hello-world https://git.yoctoproject.org/git/kernel-module-hello-world;' returned non-zero exit status 1.
>> ...
>> ERROR: Bitbake Fetcher Error: FetchError("Recipe uses a floating
>> tag/branch 'master' for repo
>> 'git.yoctoproject.org/git/kernel-module-hello-world' without a fixed
>> SRCREV yet doesn't call bb.fetch2.get_srcrev() (use SRCPV in PV for
>> OE).", None)
>>
>> https://auto/
>> builder.yoctoproject.org%2Fvalkyrie%2F%23%2Fbuilders%2F68%2Fbuilds%2F2
>> 284&data=05%7C02%7Ctony.rex%40ericsson.com%7Cde1ae2aceeda470b381608ddd
>> efdd730%7C92e84cebfbfd47abbe52080c6b87953f%7C0%7C0%7C63891190400520900
>> 3%7CUnknown%7CTWFpbGZsb3d8eyJFbXB0eU1hcGkiOnRydWUsIlYiOiIwLjAuMDAwMCIs
>> IlAiOiJXaW4zMiIsIkFOIjoiTWFpbCIsIldUIjoyfQ%3D%3D%7C0%7C%7C%7C&sdata=mP
>> UXMdb%2FYOyGIm0rnncaWgcqtwkUIh8364IP3eYDPe4%3D&reserved=0
>> https://auto/
>> builder.yoctoproject.org%2Fvalkyrie%2F%23%2Fbuilders%2F30%2Fbuilds%2F2
>> 199&data=05%7C02%7Ctony.rex%40ericsson.com%7Cde1ae2aceeda470b381608ddd
>> efdd730%7C92e84cebfbfd47abbe52080c6b87953f%7C0%7C0%7C63891190400522743
>> 2%7CUnknown%7CTWFpbGZsb3d8eyJFbXB0eU1hcGkiOnRydWUsIlYiOiIwLjAuMDAwMCIs
>> IlAiOiJXaW4zMiIsIkFOIjoiTWFpbCIsIldUIjoyfQ%3D%3D%7C0%7C%7C%7C&sdata=R0
>> HFameGl2LmIVMT9ml%2BDMnj%2FMbG6SW0%2F3yNZHQu2js%3D&reserved=0
>> https://auto/
>> builder.yoctoproject.org%2Fvalkyrie%2F%23%2Fbuilders%2F16%2Fbuilds%2F2
>> 247&data=05%7C02%7Ctony.rex%40ericsson.com%7Cde1ae2aceeda470b381608ddd
>> efdd730%7C92e84cebfbfd47abbe52080c6b87953f%7C0%7C0%7C63891190400524538
>> 7%7CUnknown%7CTWFpbGZsb3d8eyJFbXB0eU1hcGkiOnRydWUsIlYiOiIwLjAuMDAwMCIs
>> IlAiOiJXaW4zMiIsIkFOIjoiTWFpbCIsIldUIjoyfQ%3D%3D%7C0%7C%7C%7C&sdata=NS
>> FhEupwMkgqUAhpRgFbKnyFl%2FsY%2B4UJMhVQHhqOaNM%3D&reserved=0
>>
>> Can you have a look at this error please?
>>
>> Best regards,
>> Mathieu
>>
>> --
>> Mathieu Dubois-Briand, Bootlin
>> Embedded Linux and Kernel engineering
>> https://boot/
>> lin.com%2F&data=05%7C02%7Ctony.rex%40ericsson.com%7Cde1ae2aceeda470b38
>> 1608dddefdd730%7C92e84cebfbfd47abbe52080c6b87953f%7C0%7C0%7C6389119040
>> 05263327%7CUnknown%7CTWFpbGZsb3d8eyJFbXB0eU1hcGkiOnRydWUsIlYiOiIwLjAuM
>> DAwMCIsIlAiOiJXaW4zMiIsIkFOIjoiTWFpbCIsIldUIjoyfQ%3D%3D%7C0%7C%7C%7C&s
>> data=oyi7YWsXDVHn7QB4fDdG1sBp7gQsfGWcJTWtX1CjAw4%3D&reserved=0
>>
>>
>>
>>
>>
>> -=-=-=-=-=-=-=-=-=-=-=-
>> Links: You receive all messages sent to this group.
>> View/Reply Online (#222149): https://lists.openembedded.org/g/openembedded-core/message/222149
>> Mute This Topic: https://lists.openembedded.org/mt/114706197/6084445
>> Group Owner: openembedded-core+owner@lists.openembedded.org
>> Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [skandigraun@gmail.com]
>> -=-=-=-=-=-=-=-=-=-=-=-
>>

Re: [OE-core] [PATCH 0/1] base.bbclass: Fix the recipe parsing race issue

[Tony Rex]

It was introduced by this commit:
https://git.yoctoproject.org/poky/commit/?id=ebfa1700f41b3411aec040144605166c35b8dd14

Maybe "not allowed" is a strong wording, but that was how I interpreted it. Sorry, my bad if I misinterpreted it.

Tony

-----Original Message-----
From: Gyorgy Sarvari <skandigraun@gmail.com>
Sent: Wednesday, 20 August 2025 12:18
To: Tony Rex <tony.rex@ericsson.com>; Alexander Kanavin <alex.kanavin@gmail.com>
Cc: Mathieu Dubois-Briand <mathieu.dubois-briand@bootlin.com>; openembedded-core@lists.openembedded.org
Subject: Re: [OE-core] [PATCH 0/1] base.bbclass: Fix the recipe parsing race issue

[You don't often get email from skandigraun@gmail.com. Learn why this is important at https://aka.ms/LearnAboutSenderIdentification ]

On 8/19/25 23:10, Tony Rex via lists.openembedded.org wrote:
> Hi Alex and Mathieu!
>
> Ok, so the patch fixes a bug and then the floating tag/branch function is in place again.
> The floating tag/branch function is broken due to the bug and the patch fixes it.
>
> The mail from Mathieu below might not fully reflect the issue here, since it seems to create a recipe with the help of devtool.
>
> To verify:
> 1. add layers/oe-core/meta-selftest to conf/bblayers.conf 2. run
> bitbake gitrepotest -c fetch
> result:
> NOTE: Tasks Summary: Attempted 2 tasks of which 0 didn't need to be rerun and all succeeded.
> 3. run bitbake gitrepotest -c cleanall 4. edit
> ../layers/oe-core/meta-selftest/recipes-test/gitrepotest/gitrepotest.b
> b
> add tag=2.12 in SRC_URI
> SRC_URI =
> "git://git.yoctoproject.org/git/matchbox-panel-2;branch=master;tag=2.1
> 2;protocol=https \ and comment out SRCREV #SRCREV = "f82ca3f42510fb3ef10f598b393eb373a2c34ca7"
> 5. run bitbake gitrepotest -c fetch
> result:
> NOTE: Tasks Summary: Attempted 2 tasks of which 0 didn't need to be rerun and all succeeded.
> The fetch goes through although this is not allowed according to the floating tag/branch error statement if not d.getVar("__BBSRCREV_SEEN"): in git.py.

Could you point to a commit/email/etc about this "no floating tags allowed" statement? AFAIK tags without specifying revision is allowed and supported. Not recommended, due to mandatory network access, and because git tags are not immutable, but I'm surprised to hear that they are not supposed to work (which of course can also mean that I just missed a memo)

> 6. run bitbake gitrepotest -c cleanall 7. apply patch
> 0001-base.bbclass-Fix-the-recipe-parsing-race-issue.patch
> 8. run bitbake gitrepotest -c fetch
> result:
> ERROR: gitrepotest-1.0-r0 do_fetch: Bitbake Fetcher Error:
> FetchError("Recipe uses a floating tag/branch '2.12' for repo
> 'git.yoctoproject.org/git/matchbox-panel-2' without a fixed SRCREV yet
> doesn't call bb.fetch2.get_srcrev() (use SRCPV in PV for OE).", None)
> ERROR: Logfile of failure stored in:
> .../build/tmp-glibc/work/cortexa53-wrs-linux/gitrepotest/1.0/temp/log.
> do_fetch.1254999
> ERROR: Task (.../layers/oe-core/meta-selftest/recipes-test/gitrepotest/gitrepotest.bb:do_fetch) failed with exit code '1'
> NOTE: Tasks Summary: Attempted 2 tasks of which 0 didn't need to be rerun and 1 failed.
>
> Error is the expected result when only having a tag set.
>
> Tony
>
> -----Original Message-----
> From: Alexander Kanavin <alex.kanavin@gmail.com>
> Sent: Tuesday, 19 August 2025 10:53
> To: Tony Rex <tony.rex@ericsson.com>
> Cc: Mathieu Dubois-Briand <mathieu.dubois-briand@bootlin.com>;
> openembedded-core@lists.openembedded.org
> Subject: Re: [OE-core] [PATCH 0/1] base.bbclass: Fix the recipe
> parsing race issue
>
> [You don't often get email from alex.kanavin@gmail.com. Learn why this
> is important at https://aka.ms/LearnAboutSenderIdentification ]
>
> You need to make things a little bit easier for maintainers and try to save their time.
>
> I would suggest that you provide clear steps to observe the issue.
> Attach the recipe file that would demonstrate the issue, and provide clear steps to trigger it. What is confusing right now is that you say here's a recipe that builds ok, but then you 'apply the patch' and then there's some kind of failure. So which patch needs to be applied?
>
> Alex
>
> On Fri, 15 Aug 2025 at 17:29, Tony Rex via lists.openembedded.org <tony.rex=ericsson.com@lists.openembedded.org> wrote:
>> Hi Mathieu!
>>
>> I think that this is how it is intended to work, bitbake does not allow floating tag/branch anymore.
>> Please correct me if I'm wrong @Richard Purdie (who implemented this
>> for security reasons I think.)
>>
>> The patch that I have makes sure that the parsing and setting of the
>> variables in the recipe are done before executing the do_fetch command, and as a side effect the floating tag/branch protection is working as intended again.
>>
>> How does the recipe look like?
>> devtool add kernel-module-hello-world https://git/.
>> yoctoproject.org%2Fgit%2Fkernel-module-hello-world&data=05%7C02%7Cton
>> y
>> .rex%40ericsson.com%7Cde1ae2aceeda470b381608dddefdd730%7C92e84cebfbfd
>> 4
>> 7abbe52080c6b87953f%7C0%7C0%7C638911904005151487%7CUnknown%7CTWFpbGZs
>> b
>> 3d8eyJFbXB0eU1hcGkiOnRydWUsIlYiOiIwLjAuMDAwMCIsIlAiOiJXaW4zMiIsIkFOIj
>> o
>> iTWFpbCIsIldUIjoyfQ%3D%3D%7C0%7C%7C%7C&sdata=jyCicRQT%2FLtb1YxiFHKgw5
>> B
>> nNktN4t9IciiyxtsEwO4%3D&reserved=0
>> The one I get looks like this:
>>
>> cat
>> build/workspace/recipes/kernel-module-hello-world/kernel-module-hello
>> -
>> world_git.bb
>> # Recipe created by recipetool
>> # This is the basis of a recipe and may need further editing in order to be fully functional.
>> # (Feel free to remove these comments when editing.)
>>
>> # WARNING: the following LICENSE and LIC_FILES_CHKSUM values are best
>> guesses - it is # your responsibility to verify that the values are complete and correct.
>> #
>> # The following license files were not able to be identified and are
>> # represented as "Unknown" below, you will need to check them yourself:
>> #   LICENSE
>> LICENSE = "Unknown"
>> LIC_FILES_CHKSUM = "file://LICENSE;md5=7b785be6433a2e4af6a641cc7f0330d1"
>>
>> SRC_URI = "git://git.yoctoproject.org/git/kernel-module-hello-world;protocol=https;branch=master"
>>
>> # Modify these as desired
>> PV = "1.0+git"
>> SRCREV = "b66f0f2bd559a1df81fc1bda240eba07a2ecdd2b"
>>
>> S = "${WORKDIR}/git"
>>
>> inherit module
>>
>> EXTRA_OEMAKE:append:task-install = " -C ${STAGING_KERNEL_DIR} M=${S}"
>> EXTRA_OEMAKE += "KERNEL_DIR=${STAGING_KERNEL_DIR}"
>>
>> And it contains SRCREV = "b66f0f2bd559a1df81fc1bda240eba07a2ecdd2b"
>>
>> Tony
>>
>> -----Original Message-----
>> From: Mathieu Dubois-Briand <mathieu.dubois-briand@bootlin.com>
>> Sent: Friday, 15 August 2025 11:26
>> To: Tony Rex <tony.rex@ericsson.com>;
>> openembedded-core@lists.openembedded.org
>> Subject: Re: [OE-core] [PATCH 0/1] base.bbclass: Fix the recipe
>> parsing race issue
>>
>> [You don't often get email from mathieu.dubois-briand@bootlin.com.
>> Learn why this is important at
>> https://aka.ms/LearnAboutSenderIdentification ]
>>
>> On Thu Aug 14, 2025 at 7:23 PM CEST, Tony Rex via lists.openembedded.org wrote:
>>> It seems that the race condition happens when doing do_fetch on
>>> really small repos and it can be difficult to reproduce so below the
>>> other case with floating tag/branch that is easier to verify.
>>>
>>> To verify the floating tag/branch issue have s simple recipe
>>> hello_1.0.bb and set the SRC_URI to SRC_URI = "git://<repository>;protocol=https;user=${USER};branch=master;tag=v1.0"
>>>
>>> run command bitbake hello everything builds.
>>>
>>> Apply the patch
>>>
>>> run command bitbake hello
>>> ERROR: hello-1.0-r1 do_fetch: Bitbake Fetcher Error:
>>> FetchError("Recipe uses a floating tag/branch 'v1.0' for repo
>>> '<repository>' without a fixed SRCREV yet doesn't call
>>> bb.fetch2.get_srcrev() (use SRCPV in PV for OE).", None)
>>> ERROR: Logfile of failure stored in:
>>> hello/1.0/temp/log.do_fetch.3289932
>>> ERROR: Task (hello/hello_1.0.bb:do_fetch) failed with exit code '1'
>>>
>>> Tony Rex (1):
>>>   base.bbclass: Fix the recipe parsing race issue
>>>
>>>  meta/classes-global/base.bbclass | 2 +-
>>>  1 file changed, 1 insertion(+), 1 deletion(-)
>> Hi Tony,
>>
>> Thanks for your patch.
>>
>> It looks like this is breaking some builds with the following error:
>>
>> oeqa.utils.subprocesstweak.OETestCalledProcessError: Command '. /srv/pokybuild/yocto-worker/qemux86-64/build/build/tmp/work/qemux86_64-poky-linux/core-image-sato/1.0/testsdkext/environment-setup-core2-64-poky-linux > /dev/null; devtool add kernel-module-hello-world https://git.yoctoproject.org/git/kernel-module-hello-world;' returned non-zero exit status 1.
>> ...
>> ERROR: Bitbake Fetcher Error: FetchError("Recipe uses a floating
>> tag/branch 'master' for repo
>> 'git.yoctoproject.org/git/kernel-module-hello-world' without a fixed
>> SRCREV yet doesn't call bb.fetch2.get_srcrev() (use SRCPV in PV for
>> OE).", None)
>>
>> https://auto/
>> builder.yoctoproject.org%2Fvalkyrie%2F%23%2Fbuilders%2F68%2Fbuilds%2F
>> 2
>> 284&data=05%7C02%7Ctony.rex%40ericsson.com%7Cde1ae2aceeda470b381608dd
>> d
>> efdd730%7C92e84cebfbfd47abbe52080c6b87953f%7C0%7C0%7C6389119040052090
>> 0
>> 3%7CUnknown%7CTWFpbGZsb3d8eyJFbXB0eU1hcGkiOnRydWUsIlYiOiIwLjAuMDAwMCI
>> s
>> IlAiOiJXaW4zMiIsIkFOIjoiTWFpbCIsIldUIjoyfQ%3D%3D%7C0%7C%7C%7C&sdata=m
>> P
>> UXMdb%2FYOyGIm0rnncaWgcqtwkUIh8364IP3eYDPe4%3D&reserved=0
>> https://auto/
>> builder.yoctoproject.org%2Fvalkyrie%2F%23%2Fbuilders%2F30%2Fbuilds%2F
>> 2
>> 199&data=05%7C02%7Ctony.rex%40ericsson.com%7Cde1ae2aceeda470b381608dd
>> d
>> efdd730%7C92e84cebfbfd47abbe52080c6b87953f%7C0%7C0%7C6389119040052274
>> 3
>> 2%7CUnknown%7CTWFpbGZsb3d8eyJFbXB0eU1hcGkiOnRydWUsIlYiOiIwLjAuMDAwMCI
>> s
>> IlAiOiJXaW4zMiIsIkFOIjoiTWFpbCIsIldUIjoyfQ%3D%3D%7C0%7C%7C%7C&sdata=R
>> 0
>> HFameGl2LmIVMT9ml%2BDMnj%2FMbG6SW0%2F3yNZHQu2js%3D&reserved=0
>> https://auto/
>> builder.yoctoproject.org%2Fvalkyrie%2F%23%2Fbuilders%2F16%2Fbuilds%2F
>> 2
>> 247&data=05%7C02%7Ctony.rex%40ericsson.com%7Cde1ae2aceeda470b381608dd
>> d
>> efdd730%7C92e84cebfbfd47abbe52080c6b87953f%7C0%7C0%7C6389119040052453
>> 8
>> 7%7CUnknown%7CTWFpbGZsb3d8eyJFbXB0eU1hcGkiOnRydWUsIlYiOiIwLjAuMDAwMCI
>> s
>> IlAiOiJXaW4zMiIsIkFOIjoiTWFpbCIsIldUIjoyfQ%3D%3D%7C0%7C%7C%7C&sdata=N
>> S
>> FhEupwMkgqUAhpRgFbKnyFl%2FsY%2B4UJMhVQHhqOaNM%3D&reserved=0
>>
>> Can you have a look at this error please?
>>
>> Best regards,
>> Mathieu
>>
>> --
>> Mathieu Dubois-Briand, Bootlin
>> Embedded Linux and Kernel engineering https://boot/
>> lin.com%2F&data=05%7C02%7Ctony.rex%40ericsson.com%7Cde1ae2aceeda470b3
>> 8
>> 1608dddefdd730%7C92e84cebfbfd47abbe52080c6b87953f%7C0%7C0%7C638911904
>> 0
>> 05263327%7CUnknown%7CTWFpbGZsb3d8eyJFbXB0eU1hcGkiOnRydWUsIlYiOiIwLjAu
>> M
>> DAwMCIsIlAiOiJXaW4zMiIsIkFOIjoiTWFpbCIsIldUIjoyfQ%3D%3D%7C0%7C%7C%7C&
>> s
>> data=oyi7YWsXDVHn7QB4fDdG1sBp7gQsfGWcJTWtX1CjAw4%3D&reserved=0
>>
>>
>>
>>
>>
>> 
>>

Re: [OE-core] [PATCH 0/1] base.bbclass: Fix the recipe parsing race issue

[Gyorgy Sarvari]

On 8/20/25 13:53, Tony Rex wrote:
> It was introduced by this commit:
> https://git.yoctoproject.org/poky/commit/?id=ebfa1700f41b3411aec040144605166c35b8dd14
>
> Maybe "not allowed" is a strong wording, but that was how I interpreted it. Sorry, my bad if I misinterpreted it.

In my understanding (which could be wrong), this is more to avoid
working on a repository, where the exact work-revision can't be
determined. But in your example, it can be determined, if your patch is
not applied.

Some extra ramblings, which may or may not be unadulterated garbage from
me: Your initial commit message mentioned an expansion error. I looked
into that a bit, and is it possible that the repository that is causing
this issue has some access rate limit? The only thing that can fail
sometimes in that code path seems to be "git ls-remote".

Now, when I noticed this, I compiled a git version for myself that
randomly throws an error with "git ls-remote" command, and when I used
it with bitbake, I got this:

ERROR: ExpansionError during parsing
/home/meee/stuff/ptest-images/poky/meta/recipes-core/ell/mytest_get.bb
bb.data_smart.ExpansionError: Failure expanding variable
fetcher_hashes_dummyfunc[vardepvalue], expression was
<etc etc etc>

Is this the same error you got?

>
> Tony
>
> -----Original Message-----
> From: Gyorgy Sarvari <skandigraun@gmail.com>
> Sent: Wednesday, 20 August 2025 12:18
> To: Tony Rex <tony.rex@ericsson.com>; Alexander Kanavin <alex.kanavin@gmail.com>
> Cc: Mathieu Dubois-Briand <mathieu.dubois-briand@bootlin.com>; openembedded-core@lists.openembedded.org
> Subject: Re: [OE-core] [PATCH 0/1] base.bbclass: Fix the recipe parsing race issue
>
> [You don't often get email from skandigraun@gmail.com. Learn why this is important at https://aka.ms/LearnAboutSenderIdentification ]
>
> On 8/19/25 23:10, Tony Rex via lists.openembedded.org wrote:
>> Hi Alex and Mathieu!
>>
>> Ok, so the patch fixes a bug and then the floating tag/branch function is in place again.
>> The floating tag/branch function is broken due to the bug and the patch fixes it.
>>
>> The mail from Mathieu below might not fully reflect the issue here, since it seems to create a recipe with the help of devtool.
>>
>> To verify:
>> 1. add layers/oe-core/meta-selftest to conf/bblayers.conf 2. run
>> bitbake gitrepotest -c fetch
>> result:
>> NOTE: Tasks Summary: Attempted 2 tasks of which 0 didn't need to be rerun and all succeeded.
>> 3. run bitbake gitrepotest -c cleanall 4. edit
>> ../layers/oe-core/meta-selftest/recipes-test/gitrepotest/gitrepotest.b
>> b
>> add tag=2.12 in SRC_URI
>> SRC_URI =
>> "git://git.yoctoproject.org/git/matchbox-panel-2;branch=master;tag=2.1
>> 2;protocol=https \ and comment out SRCREV #SRCREV = "f82ca3f42510fb3ef10f598b393eb373a2c34ca7"
>> 5. run bitbake gitrepotest -c fetch
>> result:
>> NOTE: Tasks Summary: Attempted 2 tasks of which 0 didn't need to be rerun and all succeeded.
>> The fetch goes through although this is not allowed according to the floating tag/branch error statement if not d.getVar("__BBSRCREV_SEEN"): in git.py.
> Could you point to a commit/email/etc about this "no floating tags allowed" statement? AFAIK tags without specifying revision is allowed and supported. Not recommended, due to mandatory network access, and because git tags are not immutable, but I'm surprised to hear that they are not supposed to work (which of course can also mean that I just missed a memo)
>
>> 6. run bitbake gitrepotest -c cleanall 7. apply patch
>> 0001-base.bbclass-Fix-the-recipe-parsing-race-issue.patch
>> 8. run bitbake gitrepotest -c fetch
>> result:
>> ERROR: gitrepotest-1.0-r0 do_fetch: Bitbake Fetcher Error:
>> FetchError("Recipe uses a floating tag/branch '2.12' for repo
>> 'git.yoctoproject.org/git/matchbox-panel-2' without a fixed SRCREV yet
>> doesn't call bb.fetch2.get_srcrev() (use SRCPV in PV for OE).", None)
>> ERROR: Logfile of failure stored in:
>> .../build/tmp-glibc/work/cortexa53-wrs-linux/gitrepotest/1.0/temp/log.
>> do_fetch.1254999
>> ERROR: Task (.../layers/oe-core/meta-selftest/recipes-test/gitrepotest/gitrepotest.bb:do_fetch) failed with exit code '1'
>> NOTE: Tasks Summary: Attempted 2 tasks of which 0 didn't need to be rerun and 1 failed.
>>
>> Error is the expected result when only having a tag set.
>>
>> Tony
>>
>> -----Original Message-----
>> From: Alexander Kanavin <alex.kanavin@gmail.com>
>> Sent: Tuesday, 19 August 2025 10:53
>> To: Tony Rex <tony.rex@ericsson.com>
>> Cc: Mathieu Dubois-Briand <mathieu.dubois-briand@bootlin.com>;
>> openembedded-core@lists.openembedded.org
>> Subject: Re: [OE-core] [PATCH 0/1] base.bbclass: Fix the recipe
>> parsing race issue
>>
>> [You don't often get email from alex.kanavin@gmail.com. Learn why this
>> is important at https://aka.ms/LearnAboutSenderIdentification ]
>>
>> You need to make things a little bit easier for maintainers and try to save their time.
>>
>> I would suggest that you provide clear steps to observe the issue.
>> Attach the recipe file that would demonstrate the issue, and provide clear steps to trigger it. What is confusing right now is that you say here's a recipe that builds ok, but then you 'apply the patch' and then there's some kind of failure. So which patch needs to be applied?
>>
>> Alex
>>
>> On Fri, 15 Aug 2025 at 17:29, Tony Rex via lists.openembedded.org <tony.rex=ericsson.com@lists.openembedded.org> wrote:
>>> Hi Mathieu!
>>>
>>> I think that this is how it is intended to work, bitbake does not allow floating tag/branch anymore.
>>> Please correct me if I'm wrong @Richard Purdie (who implemented this
>>> for security reasons I think.)
>>>
>>> The patch that I have makes sure that the parsing and setting of the
>>> variables in the recipe are done before executing the do_fetch command, and as a side effect the floating tag/branch protection is working as intended again.
>>>
>>> How does the recipe look like?
>>> devtool add kernel-module-hello-world https://git/.
>>> yoctoproject.org%2Fgit%2Fkernel-module-hello-world&data=05%7C02%7Cton
>>> y
>>> .rex%40ericsson.com%7Cde1ae2aceeda470b381608dddefdd730%7C92e84cebfbfd
>>> 4
>>> 7abbe52080c6b87953f%7C0%7C0%7C638911904005151487%7CUnknown%7CTWFpbGZs
>>> b
>>> 3d8eyJFbXB0eU1hcGkiOnRydWUsIlYiOiIwLjAuMDAwMCIsIlAiOiJXaW4zMiIsIkFOIj
>>> o
>>> iTWFpbCIsIldUIjoyfQ%3D%3D%7C0%7C%7C%7C&sdata=jyCicRQT%2FLtb1YxiFHKgw5
>>> B
>>> nNktN4t9IciiyxtsEwO4%3D&reserved=0
>>> The one I get looks like this:
>>>
>>> cat
>>> build/workspace/recipes/kernel-module-hello-world/kernel-module-hello
>>> -
>>> world_git.bb
>>> # Recipe created by recipetool
>>> # This is the basis of a recipe and may need further editing in order to be fully functional.
>>> # (Feel free to remove these comments when editing.)
>>>
>>> # WARNING: the following LICENSE and LIC_FILES_CHKSUM values are best
>>> guesses - it is # your responsibility to verify that the values are complete and correct.
>>> #
>>> # The following license files were not able to be identified and are
>>> # represented as "Unknown" below, you will need to check them yourself:
>>> #   LICENSE
>>> LICENSE = "Unknown"
>>> LIC_FILES_CHKSUM = "file://LICENSE;md5=7b785be6433a2e4af6a641cc7f0330d1"
>>>
>>> SRC_URI = "git://git.yoctoproject.org/git/kernel-module-hello-world;protocol=https;branch=master"
>>>
>>> # Modify these as desired
>>> PV = "1.0+git"
>>> SRCREV = "b66f0f2bd559a1df81fc1bda240eba07a2ecdd2b"
>>>
>>> S = "${WORKDIR}/git"
>>>
>>> inherit module
>>>
>>> EXTRA_OEMAKE:append:task-install = " -C ${STAGING_KERNEL_DIR} M=${S}"
>>> EXTRA_OEMAKE += "KERNEL_DIR=${STAGING_KERNEL_DIR}"
>>>
>>> And it contains SRCREV = "b66f0f2bd559a1df81fc1bda240eba07a2ecdd2b"
>>>
>>> Tony
>>>
>>> -----Original Message-----
>>> From: Mathieu Dubois-Briand <mathieu.dubois-briand@bootlin.com>
>>> Sent: Friday, 15 August 2025 11:26
>>> To: Tony Rex <tony.rex@ericsson.com>;
>>> openembedded-core@lists.openembedded.org
>>> Subject: Re: [OE-core] [PATCH 0/1] base.bbclass: Fix the recipe
>>> parsing race issue
>>>
>>> [You don't often get email from mathieu.dubois-briand@bootlin.com.
>>> Learn why this is important at
>>> https://aka.ms/LearnAboutSenderIdentification ]
>>>
>>> On Thu Aug 14, 2025 at 7:23 PM CEST, Tony Rex via lists.openembedded.org wrote:
>>>> It seems that the race condition happens when doing do_fetch on
>>>> really small repos and it can be difficult to reproduce so below the
>>>> other case with floating tag/branch that is easier to verify.
>>>>
>>>> To verify the floating tag/branch issue have s simple recipe
>>>> hello_1.0.bb and set the SRC_URI to SRC_URI = "git://<repository>;protocol=https;user=${USER};branch=master;tag=v1.0"
>>>>
>>>> run command bitbake hello everything builds.
>>>>
>>>> Apply the patch
>>>>
>>>> run command bitbake hello
>>>> ERROR: hello-1.0-r1 do_fetch: Bitbake Fetcher Error:
>>>> FetchError("Recipe uses a floating tag/branch 'v1.0' for repo
>>>> '<repository>' without a fixed SRCREV yet doesn't call
>>>> bb.fetch2.get_srcrev() (use SRCPV in PV for OE).", None)
>>>> ERROR: Logfile of failure stored in:
>>>> hello/1.0/temp/log.do_fetch.3289932
>>>> ERROR: Task (hello/hello_1.0.bb:do_fetch) failed with exit code '1'
>>>>
>>>> Tony Rex (1):
>>>>   base.bbclass: Fix the recipe parsing race issue
>>>>
>>>>  meta/classes-global/base.bbclass | 2 +-
>>>>  1 file changed, 1 insertion(+), 1 deletion(-)
>>> Hi Tony,
>>>
>>> Thanks for your patch.
>>>
>>> It looks like this is breaking some builds with the following error:
>>>
>>> oeqa.utils.subprocesstweak.OETestCalledProcessError: Command '. /srv/pokybuild/yocto-worker/qemux86-64/build/build/tmp/work/qemux86_64-poky-linux/core-image-sato/1.0/testsdkext/environment-setup-core2-64-poky-linux > /dev/null; devtool add kernel-module-hello-world https://git.yoctoproject.org/git/kernel-module-hello-world;' returned non-zero exit status 1.
>>> ...
>>> ERROR: Bitbake Fetcher Error: FetchError("Recipe uses a floating
>>> tag/branch 'master' for repo
>>> 'git.yoctoproject.org/git/kernel-module-hello-world' without a fixed
>>> SRCREV yet doesn't call bb.fetch2.get_srcrev() (use SRCPV in PV for
>>> OE).", None)
>>>
>>> https://auto/
>>> builder.yoctoproject.org%2Fvalkyrie%2F%23%2Fbuilders%2F68%2Fbuilds%2F
>>> 2
>>> 284&data=05%7C02%7Ctony.rex%40ericsson.com%7Cde1ae2aceeda470b381608dd
>>> d
>>> efdd730%7C92e84cebfbfd47abbe52080c6b87953f%7C0%7C0%7C6389119040052090
>>> 0
>>> 3%7CUnknown%7CTWFpbGZsb3d8eyJFbXB0eU1hcGkiOnRydWUsIlYiOiIwLjAuMDAwMCI
>>> s
>>> IlAiOiJXaW4zMiIsIkFOIjoiTWFpbCIsIldUIjoyfQ%3D%3D%7C0%7C%7C%7C&sdata=m
>>> P
>>> UXMdb%2FYOyGIm0rnncaWgcqtwkUIh8364IP3eYDPe4%3D&reserved=0
>>> https://auto/
>>> builder.yoctoproject.org%2Fvalkyrie%2F%23%2Fbuilders%2F30%2Fbuilds%2F
>>> 2
>>> 199&data=05%7C02%7Ctony.rex%40ericsson.com%7Cde1ae2aceeda470b381608dd
>>> d
>>> efdd730%7C92e84cebfbfd47abbe52080c6b87953f%7C0%7C0%7C6389119040052274
>>> 3
>>> 2%7CUnknown%7CTWFpbGZsb3d8eyJFbXB0eU1hcGkiOnRydWUsIlYiOiIwLjAuMDAwMCI
>>> s
>>> IlAiOiJXaW4zMiIsIkFOIjoiTWFpbCIsIldUIjoyfQ%3D%3D%7C0%7C%7C%7C&sdata=R
>>> 0
>>> HFameGl2LmIVMT9ml%2BDMnj%2FMbG6SW0%2F3yNZHQu2js%3D&reserved=0
>>> https://auto/
>>> builder.yoctoproject.org%2Fvalkyrie%2F%23%2Fbuilders%2F16%2Fbuilds%2F
>>> 2
>>> 247&data=05%7C02%7Ctony.rex%40ericsson.com%7Cde1ae2aceeda470b381608dd
>>> d
>>> efdd730%7C92e84cebfbfd47abbe52080c6b87953f%7C0%7C0%7C6389119040052453
>>> 8
>>> 7%7CUnknown%7CTWFpbGZsb3d8eyJFbXB0eU1hcGkiOnRydWUsIlYiOiIwLjAuMDAwMCI
>>> s
>>> IlAiOiJXaW4zMiIsIkFOIjoiTWFpbCIsIldUIjoyfQ%3D%3D%7C0%7C%7C%7C&sdata=N
>>> S
>>> FhEupwMkgqUAhpRgFbKnyFl%2FsY%2B4UJMhVQHhqOaNM%3D&reserved=0
>>>
>>> Can you have a look at this error please?
>>>
>>> Best regards,
>>> Mathieu
>>>
>>> --
>>> Mathieu Dubois-Briand, Bootlin
>>> Embedded Linux and Kernel engineering https://boot/
>>> lin.com%2F&data=05%7C02%7Ctony.rex%40ericsson.com%7Cde1ae2aceeda470b3
>>> 8
>>> 1608dddefdd730%7C92e84cebfbfd47abbe52080c6b87953f%7C0%7C0%7C638911904
>>> 0
>>> 05263327%7CUnknown%7CTWFpbGZsb3d8eyJFbXB0eU1hcGkiOnRydWUsIlYiOiIwLjAu
>>> M
>>> DAwMCIsIlAiOiJXaW4zMiIsIkFOIjoiTWFpbCIsIldUIjoyfQ%3D%3D%7C0%7C%7C%7C&
>>> s
>>> data=oyi7YWsXDVHn7QB4fDdG1sBp7gQsfGWcJTWtX1CjAw4%3D&reserved=0
>>>
>>>
>>>
>>>
>>>
>>> -=-=-=-=-=-=-=-=-=-=-=-
>>> Links: You receive all messages sent to this group.
>>> View/Reply Online (#222149):
>>> https://lis/
>>> ts.openembedded.org%2Fg%2Fopenembedded-core%2Fmessage%2F222149&data=0
>>> 5%7C02%7Ctony.rex%40ericsson.com%7C2de957deabb44811987008dddfd2db2e%7
>>> C92e84cebfbfd47abbe52080c6b87953f%7C0%7C0%7C638912818889885720%7CUnkn
>>> own%7CTWFpbGZsb3d8eyJFbXB0eU1hcGkiOnRydWUsIlYiOiIwLjAuMDAwMCIsIlAiOiJ
>>> XaW4zMiIsIkFOIjoiTWFpbCIsIldUIjoyfQ%3D%3D%7C0%7C%7C%7C&sdata=gDT0CZ9r
>>> SmJYVsVWQN2eMCrJvTBVS%2FHmPu20h6Olzuw%3D&reserved=0
>>> Mute This Topic:
>>> https://lis/
>>> ts.openembedded.org%2Fmt%2F114706197%2F6084445&data=05%7C02%7Ctony.re
>>> x%40ericsson.com%7C2de957deabb44811987008dddfd2db2e%7C92e84cebfbfd47a
>>> bbe52080c6b87953f%7C0%7C0%7C638912818889899266%7CUnknown%7CTWFpbGZsb3
>>> d8eyJFbXB0eU1hcGkiOnRydWUsIlYiOiIwLjAuMDAwMCIsIlAiOiJXaW4zMiIsIkFOIjo
>>> iTWFpbCIsIldUIjoyfQ%3D%3D%7C0%7C%7C%7C&sdata=1%2BbrJUes0Q0HERH1%2F22%
>>> 2FpeuvSRqJiHS9MKZe8gtIzJE%3D&reserved=0
>>> Group Owner: openembedded-core+owner@lists.openembedded.org
>>> Unsubscribe:
>>> https://lis/
>>> ts.openembedded.org%2Fg%2Fopenembedded-core%2Funsub&data=05%7C02%7Cto
>>> ny.rex%40ericsson.com%7C2de957deabb44811987008dddfd2db2e%7C92e84cebfb
>>> fd47abbe52080c6b87953f%7C0%7C0%7C638912818889915666%7CUnknown%7CTWFpb
>>> GZsb3d8eyJFbXB0eU1hcGkiOnRydWUsIlYiOiIwLjAuMDAwMCIsIlAiOiJXaW4zMiIsIk
>>> FOIjoiTWFpbCIsIldUIjoyfQ%3D%3D%7C0%7C%7C%7C&sdata=tv1%2FYgsOKZRkU0KM8
>>> Cyqr3P%2FKjZsOBwcvPmz7ztR0f0%3D&reserved=0 [skandigraun@gmail.com]
>>> -=-=-=-=-=-=-=-=-=-=-=-
>>>

Re: [OE-core] [PATCH 0/1] base.bbclass: Fix the recipe parsing race issue

[Gyorgy Sarvari]

On 8/20/25 17:44, Gyorgy Sarvari via lists.openembedded.org wrote:
> On 8/20/25 13:53, Tony Rex wrote:
>> It was introduced by this commit:
>> https://git.yoctoproject.org/poky/commit/?id=ebfa1700f41b3411aec040144605166c35b8dd14
>>
>> Maybe "not allowed" is a strong wording, but that was how I interpreted it. Sorry, my bad if I misinterpreted it.
> In my understanding (which could be wrong), this is more to avoid
> working on a repository, where the exact work-revision can't be
> determined. But in your example, it can be determined, if your patch is
> not applied.
>
> Some extra ramblings, which may or may not be unadulterated garbage from
> me: Your initial commit message mentioned an expansion error. I looked
> into that a bit, and is it possible that the repository that is causing
> this issue has some access rate limit? The only thing that can fail
> sometimes in that code path seems to be "git ls-remote".
>
> Now, when I noticed this, I compiled a git version for myself that
> randomly throws an error with "git ls-remote" command, and when I used
> it with bitbake, I got this:
>
> ERROR: ExpansionError during parsing
> /home/meee/stuff/ptest-images/poky/meta/recipes-core/ell/mytest_get.bb
> bb.data_smart.ExpansionError: Failure expanding variable
> fetcher_hashes_dummyfunc[vardepvalue], expression was
> <etc etc etc>
>
> Is this the same error you got?

Sorry for the spam, one thing I forgot - the error on my end also
contains details about the command that failed, pointing to git. If it's
not the same as you get, do you have a copy of the full bitbake output
you get when you see this error?

>> Tony
>>
>> -----Original Message-----
>> From: Gyorgy Sarvari <skandigraun@gmail.com>
>> Sent: Wednesday, 20 August 2025 12:18
>> To: Tony Rex <tony.rex@ericsson.com>; Alexander Kanavin <alex.kanavin@gmail.com>
>> Cc: Mathieu Dubois-Briand <mathieu.dubois-briand@bootlin.com>; openembedded-core@lists.openembedded.org
>> Subject: Re: [OE-core] [PATCH 0/1] base.bbclass: Fix the recipe parsing race issue
>>
>> [You don't often get email from skandigraun@gmail.com. Learn why this is important at https://aka.ms/LearnAboutSenderIdentification ]
>>
>> On 8/19/25 23:10, Tony Rex via lists.openembedded.org wrote:
>>> Hi Alex and Mathieu!
>>>
>>> Ok, so the patch fixes a bug and then the floating tag/branch function is in place again.
>>> The floating tag/branch function is broken due to the bug and the patch fixes it.
>>>
>>> The mail from Mathieu below might not fully reflect the issue here, since it seems to create a recipe with the help of devtool.
>>>
>>> To verify:
>>> 1. add layers/oe-core/meta-selftest to conf/bblayers.conf 2. run
>>> bitbake gitrepotest -c fetch
>>> result:
>>> NOTE: Tasks Summary: Attempted 2 tasks of which 0 didn't need to be rerun and all succeeded.
>>> 3. run bitbake gitrepotest -c cleanall 4. edit
>>> ../layers/oe-core/meta-selftest/recipes-test/gitrepotest/gitrepotest.b
>>> b
>>> add tag=2.12 in SRC_URI
>>> SRC_URI =
>>> "git://git.yoctoproject.org/git/matchbox-panel-2;branch=master;tag=2.1
>>> 2;protocol=https \ and comment out SRCREV #SRCREV = "f82ca3f42510fb3ef10f598b393eb373a2c34ca7"
>>> 5. run bitbake gitrepotest -c fetch
>>> result:
>>> NOTE: Tasks Summary: Attempted 2 tasks of which 0 didn't need to be rerun and all succeeded.
>>> The fetch goes through although this is not allowed according to the floating tag/branch error statement if not d.getVar("__BBSRCREV_SEEN"): in git.py.
>> Could you point to a commit/email/etc about this "no floating tags allowed" statement? AFAIK tags without specifying revision is allowed and supported. Not recommended, due to mandatory network access, and because git tags are not immutable, but I'm surprised to hear that they are not supposed to work (which of course can also mean that I just missed a memo)
>>
>>> 6. run bitbake gitrepotest -c cleanall 7. apply patch
>>> 0001-base.bbclass-Fix-the-recipe-parsing-race-issue.patch
>>> 8. run bitbake gitrepotest -c fetch
>>> result:
>>> ERROR: gitrepotest-1.0-r0 do_fetch: Bitbake Fetcher Error:
>>> FetchError("Recipe uses a floating tag/branch '2.12' for repo
>>> 'git.yoctoproject.org/git/matchbox-panel-2' without a fixed SRCREV yet
>>> doesn't call bb.fetch2.get_srcrev() (use SRCPV in PV for OE).", None)
>>> ERROR: Logfile of failure stored in:
>>> .../build/tmp-glibc/work/cortexa53-wrs-linux/gitrepotest/1.0/temp/log.
>>> do_fetch.1254999
>>> ERROR: Task (.../layers/oe-core/meta-selftest/recipes-test/gitrepotest/gitrepotest.bb:do_fetch) failed with exit code '1'
>>> NOTE: Tasks Summary: Attempted 2 tasks of which 0 didn't need to be rerun and 1 failed.
>>>
>>> Error is the expected result when only having a tag set.
>>>
>>> Tony
>>>
>>> -----Original Message-----
>>> From: Alexander Kanavin <alex.kanavin@gmail.com>
>>> Sent: Tuesday, 19 August 2025 10:53
>>> To: Tony Rex <tony.rex@ericsson.com>
>>> Cc: Mathieu Dubois-Briand <mathieu.dubois-briand@bootlin.com>;
>>> openembedded-core@lists.openembedded.org
>>> Subject: Re: [OE-core] [PATCH 0/1] base.bbclass: Fix the recipe
>>> parsing race issue
>>>
>>> [You don't often get email from alex.kanavin@gmail.com. Learn why this
>>> is important at https://aka.ms/LearnAboutSenderIdentification ]
>>>
>>> You need to make things a little bit easier for maintainers and try to save their time.
>>>
>>> I would suggest that you provide clear steps to observe the issue.
>>> Attach the recipe file that would demonstrate the issue, and provide clear steps to trigger it. What is confusing right now is that you say here's a recipe that builds ok, but then you 'apply the patch' and then there's some kind of failure. So which patch needs to be applied?
>>>
>>> Alex
>>>
>>> On Fri, 15 Aug 2025 at 17:29, Tony Rex via lists.openembedded.org <tony.rex=ericsson.com@lists.openembedded.org> wrote:
>>>> Hi Mathieu!
>>>>
>>>> I think that this is how it is intended to work, bitbake does not allow floating tag/branch anymore.
>>>> Please correct me if I'm wrong @Richard Purdie (who implemented this
>>>> for security reasons I think.)
>>>>
>>>> The patch that I have makes sure that the parsing and setting of the
>>>> variables in the recipe are done before executing the do_fetch command, and as a side effect the floating tag/branch protection is working as intended again.
>>>>
>>>> How does the recipe look like?
>>>> devtool add kernel-module-hello-world https://git/.
>>>> yoctoproject.org%2Fgit%2Fkernel-module-hello-world&data=05%7C02%7Cton
>>>> y
>>>> .rex%40ericsson.com%7Cde1ae2aceeda470b381608dddefdd730%7C92e84cebfbfd
>>>> 4
>>>> 7abbe52080c6b87953f%7C0%7C0%7C638911904005151487%7CUnknown%7CTWFpbGZs
>>>> b
>>>> 3d8eyJFbXB0eU1hcGkiOnRydWUsIlYiOiIwLjAuMDAwMCIsIlAiOiJXaW4zMiIsIkFOIj
>>>> o
>>>> iTWFpbCIsIldUIjoyfQ%3D%3D%7C0%7C%7C%7C&sdata=jyCicRQT%2FLtb1YxiFHKgw5
>>>> B
>>>> nNktN4t9IciiyxtsEwO4%3D&reserved=0
>>>> The one I get looks like this:
>>>>
>>>> cat
>>>> build/workspace/recipes/kernel-module-hello-world/kernel-module-hello
>>>> -
>>>> world_git.bb
>>>> # Recipe created by recipetool
>>>> # This is the basis of a recipe and may need further editing in order to be fully functional.
>>>> # (Feel free to remove these comments when editing.)
>>>>
>>>> # WARNING: the following LICENSE and LIC_FILES_CHKSUM values are best
>>>> guesses - it is # your responsibility to verify that the values are complete and correct.
>>>> #
>>>> # The following license files were not able to be identified and are
>>>> # represented as "Unknown" below, you will need to check them yourself:
>>>> #   LICENSE
>>>> LICENSE = "Unknown"
>>>> LIC_FILES_CHKSUM = "file://LICENSE;md5=7b785be6433a2e4af6a641cc7f0330d1"
>>>>
>>>> SRC_URI = "git://git.yoctoproject.org/git/kernel-module-hello-world;protocol=https;branch=master"
>>>>
>>>> # Modify these as desired
>>>> PV = "1.0+git"
>>>> SRCREV = "b66f0f2bd559a1df81fc1bda240eba07a2ecdd2b"
>>>>
>>>> S = "${WORKDIR}/git"
>>>>
>>>> inherit module
>>>>
>>>> EXTRA_OEMAKE:append:task-install = " -C ${STAGING_KERNEL_DIR} M=${S}"
>>>> EXTRA_OEMAKE += "KERNEL_DIR=${STAGING_KERNEL_DIR}"
>>>>
>>>> And it contains SRCREV = "b66f0f2bd559a1df81fc1bda240eba07a2ecdd2b"
>>>>
>>>> Tony
>>>>
>>>> -----Original Message-----
>>>> From: Mathieu Dubois-Briand <mathieu.dubois-briand@bootlin.com>
>>>> Sent: Friday, 15 August 2025 11:26
>>>> To: Tony Rex <tony.rex@ericsson.com>;
>>>> openembedded-core@lists.openembedded.org
>>>> Subject: Re: [OE-core] [PATCH 0/1] base.bbclass: Fix the recipe
>>>> parsing race issue
>>>>
>>>> [You don't often get email from mathieu.dubois-briand@bootlin.com.
>>>> Learn why this is important at
>>>> https://aka.ms/LearnAboutSenderIdentification ]
>>>>
>>>> On Thu Aug 14, 2025 at 7:23 PM CEST, Tony Rex via lists.openembedded.org wrote:
>>>>> It seems that the race condition happens when doing do_fetch on
>>>>> really small repos and it can be difficult to reproduce so below the
>>>>> other case with floating tag/branch that is easier to verify.
>>>>>
>>>>> To verify the floating tag/branch issue have s simple recipe
>>>>> hello_1.0.bb and set the SRC_URI to SRC_URI = "git://<repository>;protocol=https;user=${USER};branch=master;tag=v1.0"
>>>>>
>>>>> run command bitbake hello everything builds.
>>>>>
>>>>> Apply the patch
>>>>>
>>>>> run command bitbake hello
>>>>> ERROR: hello-1.0-r1 do_fetch: Bitbake Fetcher Error:
>>>>> FetchError("Recipe uses a floating tag/branch 'v1.0' for repo
>>>>> '<repository>' without a fixed SRCREV yet doesn't call
>>>>> bb.fetch2.get_srcrev() (use SRCPV in PV for OE).", None)
>>>>> ERROR: Logfile of failure stored in:
>>>>> hello/1.0/temp/log.do_fetch.3289932
>>>>> ERROR: Task (hello/hello_1.0.bb:do_fetch) failed with exit code '1'
>>>>>
>>>>> Tony Rex (1):
>>>>>   base.bbclass: Fix the recipe parsing race issue
>>>>>
>>>>>  meta/classes-global/base.bbclass | 2 +-
>>>>>  1 file changed, 1 insertion(+), 1 deletion(-)
>>>> Hi Tony,
>>>>
>>>> Thanks for your patch.
>>>>
>>>> It looks like this is breaking some builds with the following error:
>>>>
>>>> oeqa.utils.subprocesstweak.OETestCalledProcessError: Command '. /srv/pokybuild/yocto-worker/qemux86-64/build/build/tmp/work/qemux86_64-poky-linux/core-image-sato/1.0/testsdkext/environment-setup-core2-64-poky-linux > /dev/null; devtool add kernel-module-hello-world https://git.yoctoproject.org/git/kernel-module-hello-world;' returned non-zero exit status 1.
>>>> ...
>>>> ERROR: Bitbake Fetcher Error: FetchError("Recipe uses a floating
>>>> tag/branch 'master' for repo
>>>> 'git.yoctoproject.org/git/kernel-module-hello-world' without a fixed
>>>> SRCREV yet doesn't call bb.fetch2.get_srcrev() (use SRCPV in PV for
>>>> OE).", None)
>>>>
>>>> https://auto/
>>>> builder.yoctoproject.org%2Fvalkyrie%2F%23%2Fbuilders%2F68%2Fbuilds%2F
>>>> 2
>>>> 284&data=05%7C02%7Ctony.rex%40ericsson.com%7Cde1ae2aceeda470b381608dd
>>>> d
>>>> efdd730%7C92e84cebfbfd47abbe52080c6b87953f%7C0%7C0%7C6389119040052090
>>>> 0
>>>> 3%7CUnknown%7CTWFpbGZsb3d8eyJFbXB0eU1hcGkiOnRydWUsIlYiOiIwLjAuMDAwMCI
>>>> s
>>>> IlAiOiJXaW4zMiIsIkFOIjoiTWFpbCIsIldUIjoyfQ%3D%3D%7C0%7C%7C%7C&sdata=m
>>>> P
>>>> UXMdb%2FYOyGIm0rnncaWgcqtwkUIh8364IP3eYDPe4%3D&reserved=0
>>>> https://auto/
>>>> builder.yoctoproject.org%2Fvalkyrie%2F%23%2Fbuilders%2F30%2Fbuilds%2F
>>>> 2
>>>> 199&data=05%7C02%7Ctony.rex%40ericsson.com%7Cde1ae2aceeda470b381608dd
>>>> d
>>>> efdd730%7C92e84cebfbfd47abbe52080c6b87953f%7C0%7C0%7C6389119040052274
>>>> 3
>>>> 2%7CUnknown%7CTWFpbGZsb3d8eyJFbXB0eU1hcGkiOnRydWUsIlYiOiIwLjAuMDAwMCI
>>>> s
>>>> IlAiOiJXaW4zMiIsIkFOIjoiTWFpbCIsIldUIjoyfQ%3D%3D%7C0%7C%7C%7C&sdata=R
>>>> 0
>>>> HFameGl2LmIVMT9ml%2BDMnj%2FMbG6SW0%2F3yNZHQu2js%3D&reserved=0
>>>> https://auto/
>>>> builder.yoctoproject.org%2Fvalkyrie%2F%23%2Fbuilders%2F16%2Fbuilds%2F
>>>> 2
>>>> 247&data=05%7C02%7Ctony.rex%40ericsson.com%7Cde1ae2aceeda470b381608dd
>>>> d
>>>> efdd730%7C92e84cebfbfd47abbe52080c6b87953f%7C0%7C0%7C6389119040052453
>>>> 8
>>>> 7%7CUnknown%7CTWFpbGZsb3d8eyJFbXB0eU1hcGkiOnRydWUsIlYiOiIwLjAuMDAwMCI
>>>> s
>>>> IlAiOiJXaW4zMiIsIkFOIjoiTWFpbCIsIldUIjoyfQ%3D%3D%7C0%7C%7C%7C&sdata=N
>>>> S
>>>> FhEupwMkgqUAhpRgFbKnyFl%2FsY%2B4UJMhVQHhqOaNM%3D&reserved=0
>>>>
>>>> Can you have a look at this error please?
>>>>
>>>> Best regards,
>>>> Mathieu
>>>>
>>>> --
>>>> Mathieu Dubois-Briand, Bootlin
>>>> Embedded Linux and Kernel engineering https://boot/
>>>> lin.com%2F&data=05%7C02%7Ctony.rex%40ericsson.com%7Cde1ae2aceeda470b3
>>>> 8
>>>> 1608dddefdd730%7C92e84cebfbfd47abbe52080c6b87953f%7C0%7C0%7C638911904
>>>> 0
>>>> 05263327%7CUnknown%7CTWFpbGZsb3d8eyJFbXB0eU1hcGkiOnRydWUsIlYiOiIwLjAu
>>>> M
>>>> DAwMCIsIlAiOiJXaW4zMiIsIkFOIjoiTWFpbCIsIldUIjoyfQ%3D%3D%7C0%7C%7C%7C&
>>>> s
>>>> data=oyi7YWsXDVHn7QB4fDdG1sBp7gQsfGWcJTWtX1CjAw4%3D&reserved=0
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>
> -=-=-=-=-=-=-=-=-=-=-=-
> Links: You receive all messages sent to this group.
> View/Reply Online (#222190): https://lists.openembedded.org/g/openembedded-core/message/222190
> Mute This Topic: https://lists.openembedded.org/mt/114706197/6084445
> Group Owner: openembedded-core+owner@lists.openembedded.org
> Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [skandigraun@gmail.com]
> -=-=-=-=-=-=-=-=-=-=-=-
>

Re: [OE-core] [PATCH 0/1] base.bbclass: Fix the recipe parsing race issue

[Tony Rex]

Yes, I get the same or similar error if the patch is not applied.

ERROR: ExpansionError during parsing .../generic-if.bb################################   | ETA:  0:00:00
bb.data_smart.ExpansionError: Failure expanding variable fetcher_hashes_dummyfunc[vardepvalue], expression was ${@bb.fetch.get_hashvalue(d)} which triggered exception FetchError: Fetcher failure: Unable to resolve '${TAGNAME}-1.0' in upstream git repository in git ls-remote output for gerrit...
The variable dependency chain for the failure is: fetcher_hashes_dummyfunc[vardepvalue]

ERROR: Parsing halted due to errors, see error messages above

How does your mytest_get.bb look like? Since I have had problem with reproducing the error on in a common setup maybe it can be done with your bb file.

Tony

-----Original Message-----
From: Gyorgy Sarvari <skandigraun@gmail.com>
Sent: Wednesday, 20 August 2025 18:12
To: Tony Rex <tony.rex@ericsson.com>; Alexander Kanavin <alex.kanavin@gmail.com>
Cc: Mathieu Dubois-Briand <mathieu.dubois-briand@bootlin.com>; openembedded-core@lists.openembedded.org
Subject: Re: [OE-core] [PATCH 0/1] base.bbclass: Fix the recipe parsing race issue

[You don't often get email from skandigraun@gmail.com. Learn why this is important at https://aka.ms/LearnAboutSenderIdentification ]

On 8/20/25 17:44, Gyorgy Sarvari via lists.openembedded.org wrote:
> On 8/20/25 13:53, Tony Rex wrote:
>> It was introduced by this commit:
>> https://git/
>> .yoctoproject.org%2Fpoky%2Fcommit%2F%3Fid%3Debfa1700f41b3411aec040144
>> 605166c35b8dd14&data=05%7C02%7Ctony.rex%40ericsson.com%7C7b5c099d1700
>> 43080cca08dde004478d%7C92e84cebfbfd47abbe52080c6b87953f%7C0%7C0%7C638
>> 913031194171335%7CUnknown%7CTWFpbGZsb3d8eyJFbXB0eU1hcGkiOnRydWUsIlYiO
>> iIwLjAuMDAwMCIsIlAiOiJXaW4zMiIsIkFOIjoiTWFpbCIsIldUIjoyfQ%3D%3D%7C0%7
>> C%7C%7C&sdata=iYvLGK1DwJk78tEvmL%2FPOL2olj7iNL87eXqdsEraxLw%3D&reserv
>> ed=0
>>
>> Maybe "not allowed" is a strong wording, but that was how I interpreted it. Sorry, my bad if I misinterpreted it.
> In my understanding (which could be wrong), this is more to avoid
> working on a repository, where the exact work-revision can't be
> determined. But in your example, it can be determined, if your patch
> is not applied.
>
> Some extra ramblings, which may or may not be unadulterated garbage
> from
> me: Your initial commit message mentioned an expansion error. I looked
> into that a bit, and is it possible that the repository that is
> causing this issue has some access rate limit? The only thing that can
> fail sometimes in that code path seems to be "git ls-remote".
>
> Now, when I noticed this, I compiled a git version for myself that
> randomly throws an error with "git ls-remote" command, and when I used
> it with bitbake, I got this:
>
> ERROR: ExpansionError during parsing
> /home/meee/stuff/ptest-images/poky/meta/recipes-core/ell/mytest_get.bb
> bb.data_smart.ExpansionError: Failure expanding variable
> fetcher_hashes_dummyfunc[vardepvalue], expression was <etc etc etc>
>
> Is this the same error you got?

Sorry for the spam, one thing I forgot - the error on my end also contains details about the command that failed, pointing to git. If it's not the same as you get, do you have a copy of the full bitbake output you get when you see this error?

>> Tony
>>
>> -----Original Message-----
>> From: Gyorgy Sarvari <skandigraun@gmail.com>
>> Sent: Wednesday, 20 August 2025 12:18
>> To: Tony Rex <tony.rex@ericsson.com>; Alexander Kanavin
>> <alex.kanavin@gmail.com>
>> Cc: Mathieu Dubois-Briand <mathieu.dubois-briand@bootlin.com>;
>> openembedded-core@lists.openembedded.org
>> Subject: Re: [OE-core] [PATCH 0/1] base.bbclass: Fix the recipe
>> parsing race issue
>>
>> [You don't often get email from skandigraun@gmail.com. Learn why this
>> is important at https://aka.ms/LearnAboutSenderIdentification ]
>>
>> On 8/19/25 23:10, Tony Rex via lists.openembedded.org wrote:
>>> Hi Alex and Mathieu!
>>>
>>> Ok, so the patch fixes a bug and then the floating tag/branch function is in place again.
>>> The floating tag/branch function is broken due to the bug and the patch fixes it.
>>>
>>> The mail from Mathieu below might not fully reflect the issue here, since it seems to create a recipe with the help of devtool.
>>>
>>> To verify:
>>> 1. add layers/oe-core/meta-selftest to conf/bblayers.conf 2. run
>>> bitbake gitrepotest -c fetch
>>> result:
>>> NOTE: Tasks Summary: Attempted 2 tasks of which 0 didn't need to be rerun and all succeeded.
>>> 3. run bitbake gitrepotest -c cleanall 4. edit
>>> ../layers/oe-core/meta-selftest/recipes-test/gitrepotest/gitrepotest
>>> .b
>>> b
>>> add tag=2.12 in SRC_URI
>>> SRC_URI =
>>> "git://git.yoctoproject.org/git/matchbox-panel-2;branch=master;tag=2
>>> .1 2;protocol=https \ and comment out SRCREV #SRCREV =
>>> "f82ca3f42510fb3ef10f598b393eb373a2c34ca7"
>>> 5. run bitbake gitrepotest -c fetch
>>> result:
>>> NOTE: Tasks Summary: Attempted 2 tasks of which 0 didn't need to be rerun and all succeeded.
>>> The fetch goes through although this is not allowed according to the floating tag/branch error statement if not d.getVar("__BBSRCREV_SEEN"): in git.py.
>> Could you point to a commit/email/etc about this "no floating tags
>> allowed" statement? AFAIK tags without specifying revision is allowed
>> and supported. Not recommended, due to mandatory network access, and
>> because git tags are not immutable, but I'm surprised to hear that
>> they are not supposed to work (which of course can also mean that I
>> just missed a memo)
>>
>>> 6. run bitbake gitrepotest -c cleanall 7. apply patch
>>> 0001-base.bbclass-Fix-the-recipe-parsing-race-issue.patch
>>> 8. run bitbake gitrepotest -c fetch
>>> result:
>>> ERROR: gitrepotest-1.0-r0 do_fetch: Bitbake Fetcher Error:
>>> FetchError("Recipe uses a floating tag/branch '2.12' for repo
>>> 'git.yoctoproject.org/git/matchbox-panel-2' without a fixed SRCREV
>>> yet doesn't call bb.fetch2.get_srcrev() (use SRCPV in PV for OE).",
>>> None)
>>> ERROR: Logfile of failure stored in:
>>> .../build/tmp-glibc/work/cortexa53-wrs-linux/gitrepotest/1.0/temp/log.
>>> do_fetch.1254999
>>> ERROR: Task (.../layers/oe-core/meta-selftest/recipes-test/gitrepotest/gitrepotest.bb:do_fetch) failed with exit code '1'
>>> NOTE: Tasks Summary: Attempted 2 tasks of which 0 didn't need to be rerun and 1 failed.
>>>
>>> Error is the expected result when only having a tag set.
>>>
>>> Tony
>>>
>>> -----Original Message-----
>>> From: Alexander Kanavin <alex.kanavin@gmail.com>
>>> Sent: Tuesday, 19 August 2025 10:53
>>> To: Tony Rex <tony.rex@ericsson.com>
>>> Cc: Mathieu Dubois-Briand <mathieu.dubois-briand@bootlin.com>;
>>> openembedded-core@lists.openembedded.org
>>> Subject: Re: [OE-core] [PATCH 0/1] base.bbclass: Fix the recipe
>>> parsing race issue
>>>
>>> [You don't often get email from alex.kanavin@gmail.com. Learn why
>>> this is important at https://aka.ms/LearnAboutSenderIdentification ]
>>>
>>> You need to make things a little bit easier for maintainers and try to save their time.
>>>
>>> I would suggest that you provide clear steps to observe the issue.
>>> Attach the recipe file that would demonstrate the issue, and provide clear steps to trigger it. What is confusing right now is that you say here's a recipe that builds ok, but then you 'apply the patch' and then there's some kind of failure. So which patch needs to be applied?
>>>
>>> Alex
>>>
>>> On Fri, 15 Aug 2025 at 17:29, Tony Rex via lists.openembedded.org <tony.rex=ericsson.com@lists.openembedded.org> wrote:
>>>> Hi Mathieu!
>>>>
>>>> I think that this is how it is intended to work, bitbake does not allow floating tag/branch anymore.
>>>> Please correct me if I'm wrong @Richard Purdie (who implemented
>>>> this for security reasons I think.)
>>>>
>>>> The patch that I have makes sure that the parsing and setting of
>>>> the variables in the recipe are done before executing the do_fetch command, and as a side effect the floating tag/branch protection is working as intended again.
>>>>
>>>> How does the recipe look like?
>>>> devtool add kernel-module-hello-world https://git/.
>>>> yoctoproject.org%2Fgit%2Fkernel-module-hello-world&data=05%7C02%7Ct
>>>> on
>>>> y
>>>> .rex%40ericsson.com%7Cde1ae2aceeda470b381608dddefdd730%7C92e84cebfb
>>>> fd
>>>> 4
>>>> 7abbe52080c6b87953f%7C0%7C0%7C638911904005151487%7CUnknown%7CTWFpbG
>>>> Zs
>>>> b
>>>> 3d8eyJFbXB0eU1hcGkiOnRydWUsIlYiOiIwLjAuMDAwMCIsIlAiOiJXaW4zMiIsIkFO
>>>> Ij
>>>> o
>>>> iTWFpbCIsIldUIjoyfQ%3D%3D%7C0%7C%7C%7C&sdata=jyCicRQT%2FLtb1YxiFHKg
>>>> w5
>>>> B
>>>> nNktN4t9IciiyxtsEwO4%3D&reserved=0
>>>> The one I get looks like this:
>>>>
>>>> cat
>>>> build/workspace/recipes/kernel-module-hello-world/kernel-module-hel
>>>> lo
>>>> -
>>>> world_git.bb
>>>> # Recipe created by recipetool
>>>> # This is the basis of a recipe and may need further editing in order to be fully functional.
>>>> # (Feel free to remove these comments when editing.)
>>>>
>>>> # WARNING: the following LICENSE and LIC_FILES_CHKSUM values are
>>>> best guesses - it is # your responsibility to verify that the values are complete and correct.
>>>> #
>>>> # The following license files were not able to be identified and
>>>> are # represented as "Unknown" below, you will need to check them yourself:
>>>> #   LICENSE
>>>> LICENSE = "Unknown"
>>>> LIC_FILES_CHKSUM = "file://LICENSE;md5=7b785be6433a2e4af6a641cc7f0330d1"
>>>>
>>>> SRC_URI = "git://git.yoctoproject.org/git/kernel-module-hello-world;protocol=https;branch=master"
>>>>
>>>> # Modify these as desired
>>>> PV = "1.0+git"
>>>> SRCREV = "b66f0f2bd559a1df81fc1bda240eba07a2ecdd2b"
>>>>
>>>> S = "${WORKDIR}/git"
>>>>
>>>> inherit module
>>>>
>>>> EXTRA_OEMAKE:append:task-install = " -C ${STAGING_KERNEL_DIR} M=${S}"
>>>> EXTRA_OEMAKE += "KERNEL_DIR=${STAGING_KERNEL_DIR}"
>>>>
>>>> And it contains SRCREV = "b66f0f2bd559a1df81fc1bda240eba07a2ecdd2b"
>>>>
>>>> Tony
>>>>
>>>> -----Original Message-----
>>>> From: Mathieu Dubois-Briand <mathieu.dubois-briand@bootlin.com>
>>>> Sent: Friday, 15 August 2025 11:26
>>>> To: Tony Rex <tony.rex@ericsson.com>;
>>>> openembedded-core@lists.openembedded.org
>>>> Subject: Re: [OE-core] [PATCH 0/1] base.bbclass: Fix the recipe
>>>> parsing race issue
>>>>
>>>> [You don't often get email from mathieu.dubois-briand@bootlin.com.
>>>> Learn why this is important at
>>>> https://aka.ms/LearnAboutSenderIdentification ]
>>>>
>>>> On Thu Aug 14, 2025 at 7:23 PM CEST, Tony Rex via lists.openembedded.org wrote:
>>>>> It seems that the race condition happens when doing do_fetch on
>>>>> really small repos and it can be difficult to reproduce so below
>>>>> the other case with floating tag/branch that is easier to verify.
>>>>>
>>>>> To verify the floating tag/branch issue have s simple recipe
>>>>> hello_1.0.bb and set the SRC_URI to SRC_URI = "git://<repository>;protocol=https;user=${USER};branch=master;tag=v1.0"
>>>>>
>>>>> run command bitbake hello everything builds.
>>>>>
>>>>> Apply the patch
>>>>>
>>>>> run command bitbake hello
>>>>> ERROR: hello-1.0-r1 do_fetch: Bitbake Fetcher Error:
>>>>> FetchError("Recipe uses a floating tag/branch 'v1.0' for repo
>>>>> '<repository>' without a fixed SRCREV yet doesn't call
>>>>> bb.fetch2.get_srcrev() (use SRCPV in PV for OE).", None)
>>>>> ERROR: Logfile of failure stored in:
>>>>> hello/1.0/temp/log.do_fetch.3289932
>>>>> ERROR: Task (hello/hello_1.0.bb:do_fetch) failed with exit code '1'
>>>>>
>>>>> Tony Rex (1):
>>>>>   base.bbclass: Fix the recipe parsing race issue
>>>>>
>>>>>  meta/classes-global/base.bbclass | 2 +-
>>>>>  1 file changed, 1 insertion(+), 1 deletion(-)
>>>> Hi Tony,
>>>>
>>>> Thanks for your patch.
>>>>
>>>> It looks like this is breaking some builds with the following error:
>>>>
>>>> oeqa.utils.subprocesstweak.OETestCalledProcessError: Command '. /srv/pokybuild/yocto-worker/qemux86-64/build/build/tmp/work/qemux86_64-poky-linux/core-image-sato/1.0/testsdkext/environment-setup-core2-64-poky-linux > /dev/null; devtool add kernel-module-hello-world https://git.yoctoproject.org/git/kernel-module-hello-world;' returned non-zero exit status 1.
>>>> ...
>>>> ERROR: Bitbake Fetcher Error: FetchError("Recipe uses a floating
>>>> tag/branch 'master' for repo
>>>> 'git.yoctoproject.org/git/kernel-module-hello-world' without a
>>>> fixed SRCREV yet doesn't call bb.fetch2.get_srcrev() (use SRCPV in
>>>> PV for OE).", None)
>>>>
>>>> https://auto/
>>>> builder.yoctoproject.org%2Fvalkyrie%2F%23%2Fbuilders%2F68%2Fbuilds%
>>>> 2F
>>>> 2
>>>> 284&data=05%7C02%7Ctony.rex%40ericsson.com%7Cde1ae2aceeda470b381608
>>>> dd
>>>> d
>>>> efdd730%7C92e84cebfbfd47abbe52080c6b87953f%7C0%7C0%7C63891190400520
>>>> 90
>>>> 0
>>>> 3%7CUnknown%7CTWFpbGZsb3d8eyJFbXB0eU1hcGkiOnRydWUsIlYiOiIwLjAuMDAwM
>>>> CI
>>>> s
>>>> IlAiOiJXaW4zMiIsIkFOIjoiTWFpbCIsIldUIjoyfQ%3D%3D%7C0%7C%7C%7C&sdata
>>>> =m
>>>> P
>>>> UXMdb%2FYOyGIm0rnncaWgcqtwkUIh8364IP3eYDPe4%3D&reserved=0
>>>> https://auto/
>>>> builder.yoctoproject.org%2Fvalkyrie%2F%23%2Fbuilders%2F30%2Fbuilds%
>>>> 2F
>>>> 2
>>>> 199&data=05%7C02%7Ctony.rex%40ericsson.com%7Cde1ae2aceeda470b381608
>>>> dd
>>>> d
>>>> efdd730%7C92e84cebfbfd47abbe52080c6b87953f%7C0%7C0%7C63891190400522
>>>> 74
>>>> 3
>>>> 2%7CUnknown%7CTWFpbGZsb3d8eyJFbXB0eU1hcGkiOnRydWUsIlYiOiIwLjAuMDAwM
>>>> CI
>>>> s
>>>> IlAiOiJXaW4zMiIsIkFOIjoiTWFpbCIsIldUIjoyfQ%3D%3D%7C0%7C%7C%7C&sdata
>>>> =R
>>>> 0
>>>> HFameGl2LmIVMT9ml%2BDMnj%2FMbG6SW0%2F3yNZHQu2js%3D&reserved=0
>>>> https://auto/
>>>> builder.yoctoproject.org%2Fvalkyrie%2F%23%2Fbuilders%2F16%2Fbuilds%
>>>> 2F
>>>> 2
>>>> 247&data=05%7C02%7Ctony.rex%40ericsson.com%7Cde1ae2aceeda470b381608
>>>> dd
>>>> d
>>>> efdd730%7C92e84cebfbfd47abbe52080c6b87953f%7C0%7C0%7C63891190400524
>>>> 53
>>>> 8
>>>> 7%7CUnknown%7CTWFpbGZsb3d8eyJFbXB0eU1hcGkiOnRydWUsIlYiOiIwLjAuMDAwM
>>>> CI
>>>> s
>>>> IlAiOiJXaW4zMiIsIkFOIjoiTWFpbCIsIldUIjoyfQ%3D%3D%7C0%7C%7C%7C&sdata
>>>> =N
>>>> S
>>>> FhEupwMkgqUAhpRgFbKnyFl%2FsY%2B4UJMhVQHhqOaNM%3D&reserved=0
>>>>
>>>> Can you have a look at this error please?
>>>>
>>>> Best regards,
>>>> Mathieu
>>>>
>>>> --
>>>> Mathieu Dubois-Briand, Bootlin
>>>> Embedded Linux and Kernel engineering https://boot/
>>>> lin.com%2F&data=05%7C02%7Ctony.rex%40ericsson.com%7Cde1ae2aceeda470
>>>> b3
>>>> 8
>>>> 1608dddefdd730%7C92e84cebfbfd47abbe52080c6b87953f%7C0%7C0%7C6389119
>>>> 04
>>>> 0
>>>> 05263327%7CUnknown%7CTWFpbGZsb3d8eyJFbXB0eU1hcGkiOnRydWUsIlYiOiIwLj
>>>> Au
>>>> M
>>>> DAwMCIsIlAiOiJXaW4zMiIsIkFOIjoiTWFpbCIsIldUIjoyfQ%3D%3D%7C0%7C%7C%7
>>>> C&
>>>> s
>>>> data=oyi7YWsXDVHn7QB4fDdG1sBp7gQsfGWcJTWtX1CjAw4%3D&reserved=0
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>
> 
>

Re: [OE-core] [PATCH 0/1] base.bbclass: Fix the recipe parsing race issue

[Gyorgy Sarvari]

On 8/21/25 09:10, Tony Rex wrote:
> Yes, I get the same or similar error if the patch is not applied.
>
> ERROR: ExpansionError during parsing .../generic-if.bb################################   | ETA:  0:00:00
> bb.data_smart.ExpansionError: Failure expanding variable fetcher_hashes_dummyfunc[vardepvalue], expression was ${@bb.fetch.get_hashvalue(d)} which triggered exception FetchError: Fetcher failure: Unable to resolve '${TAGNAME}-1.0' in upstream git repository in git ls-remote output for gerrit...
> The variable dependency chain for the failure is: fetcher_hashes_dummyfunc[vardepvalue]
>
> ERROR: Parsing halted due to errors, see error messages above
>
> How does your mytest_get.bb look like? Since I have had problem with reproducing the error on in a common setup maybe it can be done with your bb file.
>
> Tony

The failure I reproduced is not recipe dependent. As some diagnostic
test, try this script a few times with your failing git repo (change the
repo placeholder):

for i in `seq 50`; do
  git clone $GIT_REPO_URL target_folder
  git -C ./target_folder ls-remote
  if [ $? -ne 0 ]; then
    echo ls-remote failed, iteration: $i
    break
  fi
  rm -rf ./target_folder
done

If I haven't made a typo, the script should clone your repo, do an
ls-remote inside it, delete the folder, and do the same again.

If your problem is the same that I reproduced, then I would expect the
above script to fail sometimes - if it does, then the problem is with
the infrastructure (network, remote git server, etc... something is flaky).

Just for the record: the script generates a small amount of successive
requests which some network admins might consider flooding.



> -----Original Message-----
> From: Gyorgy Sarvari <skandigraun@gmail.com>
> Sent: Wednesday, 20 August 2025 18:12
> To: Tony Rex <tony.rex@ericsson.com>; Alexander Kanavin <alex.kanavin@gmail.com>
> Cc: Mathieu Dubois-Briand <mathieu.dubois-briand@bootlin.com>; openembedded-core@lists.openembedded.org
> Subject: Re: [OE-core] [PATCH 0/1] base.bbclass: Fix the recipe parsing race issue
>
> [You don't often get email from skandigraun@gmail.com. Learn why this is important at https://aka.ms/LearnAboutSenderIdentification ]
>
> On 8/20/25 17:44, Gyorgy Sarvari via lists.openembedded.org wrote:
>> On 8/20/25 13:53, Tony Rex wrote:
>>> It was introduced by this commit:
>>> https://git/
>>> .yoctoproject.org%2Fpoky%2Fcommit%2F%3Fid%3Debfa1700f41b3411aec040144
>>> 605166c35b8dd14&data=05%7C02%7Ctony.rex%40ericsson.com%7C7b5c099d1700
>>> 43080cca08dde004478d%7C92e84cebfbfd47abbe52080c6b87953f%7C0%7C0%7C638
>>> 913031194171335%7CUnknown%7CTWFpbGZsb3d8eyJFbXB0eU1hcGkiOnRydWUsIlYiO
>>> iIwLjAuMDAwMCIsIlAiOiJXaW4zMiIsIkFOIjoiTWFpbCIsIldUIjoyfQ%3D%3D%7C0%7
>>> C%7C%7C&sdata=iYvLGK1DwJk78tEvmL%2FPOL2olj7iNL87eXqdsEraxLw%3D&reserv
>>> ed=0
>>>
>>> Maybe "not allowed" is a strong wording, but that was how I interpreted it. Sorry, my bad if I misinterpreted it.
>> In my understanding (which could be wrong), this is more to avoid
>> working on a repository, where the exact work-revision can't be
>> determined. But in your example, it can be determined, if your patch
>> is not applied.
>>
>> Some extra ramblings, which may or may not be unadulterated garbage
>> from
>> me: Your initial commit message mentioned an expansion error. I looked
>> into that a bit, and is it possible that the repository that is
>> causing this issue has some access rate limit? The only thing that can
>> fail sometimes in that code path seems to be "git ls-remote".
>>
>> Now, when I noticed this, I compiled a git version for myself that
>> randomly throws an error with "git ls-remote" command, and when I used
>> it with bitbake, I got this:
>>
>> ERROR: ExpansionError during parsing
>> /home/meee/stuff/ptest-images/poky/meta/recipes-core/ell/mytest_get.bb
>> bb.data_smart.ExpansionError: Failure expanding variable
>> fetcher_hashes_dummyfunc[vardepvalue], expression was <etc etc etc>
>>
>> Is this the same error you got?
> Sorry for the spam, one thing I forgot - the error on my end also contains details about the command that failed, pointing to git. If it's not the same as you get, do you have a copy of the full bitbake output you get when you see this error?
>
>>> Tony
>>>
>>> -----Original Message-----
>>> From: Gyorgy Sarvari <skandigraun@gmail.com>
>>> Sent: Wednesday, 20 August 2025 12:18
>>> To: Tony Rex <tony.rex@ericsson.com>; Alexander Kanavin
>>> <alex.kanavin@gmail.com>
>>> Cc: Mathieu Dubois-Briand <mathieu.dubois-briand@bootlin.com>;
>>> openembedded-core@lists.openembedded.org
>>> Subject: Re: [OE-core] [PATCH 0/1] base.bbclass: Fix the recipe
>>> parsing race issue
>>>
>>> [You don't often get email from skandigraun@gmail.com. Learn why this
>>> is important at https://aka.ms/LearnAboutSenderIdentification ]
>>>
>>> On 8/19/25 23:10, Tony Rex via lists.openembedded.org wrote:
>>>> Hi Alex and Mathieu!
>>>>
>>>> Ok, so the patch fixes a bug and then the floating tag/branch function is in place again.
>>>> The floating tag/branch function is broken due to the bug and the patch fixes it.
>>>>
>>>> The mail from Mathieu below might not fully reflect the issue here, since it seems to create a recipe with the help of devtool.
>>>>
>>>> To verify:
>>>> 1. add layers/oe-core/meta-selftest to conf/bblayers.conf 2. run
>>>> bitbake gitrepotest -c fetch
>>>> result:
>>>> NOTE: Tasks Summary: Attempted 2 tasks of which 0 didn't need to be rerun and all succeeded.
>>>> 3. run bitbake gitrepotest -c cleanall 4. edit
>>>> ../layers/oe-core/meta-selftest/recipes-test/gitrepotest/gitrepotest
>>>> .b
>>>> b
>>>> add tag=2.12 in SRC_URI
>>>> SRC_URI =
>>>> "git://git.yoctoproject.org/git/matchbox-panel-2;branch=master;tag=2
>>>> .1 2;protocol=https \ and comment out SRCREV #SRCREV =
>>>> "f82ca3f42510fb3ef10f598b393eb373a2c34ca7"
>>>> 5. run bitbake gitrepotest -c fetch
>>>> result:
>>>> NOTE: Tasks Summary: Attempted 2 tasks of which 0 didn't need to be rerun and all succeeded.
>>>> The fetch goes through although this is not allowed according to the floating tag/branch error statement if not d.getVar("__BBSRCREV_SEEN"): in git.py.
>>> Could you point to a commit/email/etc about this "no floating tags
>>> allowed" statement? AFAIK tags without specifying revision is allowed
>>> and supported. Not recommended, due to mandatory network access, and
>>> because git tags are not immutable, but I'm surprised to hear that
>>> they are not supposed to work (which of course can also mean that I
>>> just missed a memo)
>>>
>>>> 6. run bitbake gitrepotest -c cleanall 7. apply patch
>>>> 0001-base.bbclass-Fix-the-recipe-parsing-race-issue.patch
>>>> 8. run bitbake gitrepotest -c fetch
>>>> result:
>>>> ERROR: gitrepotest-1.0-r0 do_fetch: Bitbake Fetcher Error:
>>>> FetchError("Recipe uses a floating tag/branch '2.12' for repo
>>>> 'git.yoctoproject.org/git/matchbox-panel-2' without a fixed SRCREV
>>>> yet doesn't call bb.fetch2.get_srcrev() (use SRCPV in PV for OE).",
>>>> None)
>>>> ERROR: Logfile of failure stored in:
>>>> .../build/tmp-glibc/work/cortexa53-wrs-linux/gitrepotest/1.0/temp/log.
>>>> do_fetch.1254999
>>>> ERROR: Task (.../layers/oe-core/meta-selftest/recipes-test/gitrepotest/gitrepotest.bb:do_fetch) failed with exit code '1'
>>>> NOTE: Tasks Summary: Attempted 2 tasks of which 0 didn't need to be rerun and 1 failed.
>>>>
>>>> Error is the expected result when only having a tag set.
>>>>
>>>> Tony
>>>>
>>>> -----Original Message-----
>>>> From: Alexander Kanavin <alex.kanavin@gmail.com>
>>>> Sent: Tuesday, 19 August 2025 10:53
>>>> To: Tony Rex <tony.rex@ericsson.com>
>>>> Cc: Mathieu Dubois-Briand <mathieu.dubois-briand@bootlin.com>;
>>>> openembedded-core@lists.openembedded.org
>>>> Subject: Re: [OE-core] [PATCH 0/1] base.bbclass: Fix the recipe
>>>> parsing race issue
>>>>
>>>> [You don't often get email from alex.kanavin@gmail.com. Learn why
>>>> this is important at https://aka.ms/LearnAboutSenderIdentification ]
>>>>
>>>> You need to make things a little bit easier for maintainers and try to save their time.
>>>>
>>>> I would suggest that you provide clear steps to observe the issue.
>>>> Attach the recipe file that would demonstrate the issue, and provide clear steps to trigger it. What is confusing right now is that you say here's a recipe that builds ok, but then you 'apply the patch' and then there's some kind of failure. So which patch needs to be applied?
>>>>
>>>> Alex
>>>>
>>>> On Fri, 15 Aug 2025 at 17:29, Tony Rex via lists.openembedded.org <tony.rex=ericsson.com@lists.openembedded.org> wrote:
>>>>> Hi Mathieu!
>>>>>
>>>>> I think that this is how it is intended to work, bitbake does not allow floating tag/branch anymore.
>>>>> Please correct me if I'm wrong @Richard Purdie (who implemented
>>>>> this for security reasons I think.)
>>>>>
>>>>> The patch that I have makes sure that the parsing and setting of
>>>>> the variables in the recipe are done before executing the do_fetch command, and as a side effect the floating tag/branch protection is working as intended again.
>>>>>
>>>>> How does the recipe look like?
>>>>> devtool add kernel-module-hello-world https://git/.
>>>>> yoctoproject.org%2Fgit%2Fkernel-module-hello-world&data=05%7C02%7Ct
>>>>> on
>>>>> y
>>>>> .rex%40ericsson.com%7Cde1ae2aceeda470b381608dddefdd730%7C92e84cebfb
>>>>> fd
>>>>> 4
>>>>> 7abbe52080c6b87953f%7C0%7C0%7C638911904005151487%7CUnknown%7CTWFpbG
>>>>> Zs
>>>>> b
>>>>> 3d8eyJFbXB0eU1hcGkiOnRydWUsIlYiOiIwLjAuMDAwMCIsIlAiOiJXaW4zMiIsIkFO
>>>>> Ij
>>>>> o
>>>>> iTWFpbCIsIldUIjoyfQ%3D%3D%7C0%7C%7C%7C&sdata=jyCicRQT%2FLtb1YxiFHKg
>>>>> w5
>>>>> B
>>>>> nNktN4t9IciiyxtsEwO4%3D&reserved=0
>>>>> The one I get looks like this:
>>>>>
>>>>> cat
>>>>> build/workspace/recipes/kernel-module-hello-world/kernel-module-hel
>>>>> lo
>>>>> -
>>>>> world_git.bb
>>>>> # Recipe created by recipetool
>>>>> # This is the basis of a recipe and may need further editing in order to be fully functional.
>>>>> # (Feel free to remove these comments when editing.)
>>>>>
>>>>> # WARNING: the following LICENSE and LIC_FILES_CHKSUM values are
>>>>> best guesses - it is # your responsibility to verify that the values are complete and correct.
>>>>> #
>>>>> # The following license files were not able to be identified and
>>>>> are # represented as "Unknown" below, you will need to check them yourself:
>>>>> #   LICENSE
>>>>> LICENSE = "Unknown"
>>>>> LIC_FILES_CHKSUM = "file://LICENSE;md5=7b785be6433a2e4af6a641cc7f0330d1"
>>>>>
>>>>> SRC_URI = "git://git.yoctoproject.org/git/kernel-module-hello-world;protocol=https;branch=master"
>>>>>
>>>>> # Modify these as desired
>>>>> PV = "1.0+git"
>>>>> SRCREV = "b66f0f2bd559a1df81fc1bda240eba07a2ecdd2b"
>>>>>
>>>>> S = "${WORKDIR}/git"
>>>>>
>>>>> inherit module
>>>>>
>>>>> EXTRA_OEMAKE:append:task-install = " -C ${STAGING_KERNEL_DIR} M=${S}"
>>>>> EXTRA_OEMAKE += "KERNEL_DIR=${STAGING_KERNEL_DIR}"
>>>>>
>>>>> And it contains SRCREV = "b66f0f2bd559a1df81fc1bda240eba07a2ecdd2b"
>>>>>
>>>>> Tony
>>>>>
>>>>> -----Original Message-----
>>>>> From: Mathieu Dubois-Briand <mathieu.dubois-briand@bootlin.com>
>>>>> Sent: Friday, 15 August 2025 11:26
>>>>> To: Tony Rex <tony.rex@ericsson.com>;
>>>>> openembedded-core@lists.openembedded.org
>>>>> Subject: Re: [OE-core] [PATCH 0/1] base.bbclass: Fix the recipe
>>>>> parsing race issue
>>>>>
>>>>> [You don't often get email from mathieu.dubois-briand@bootlin.com.
>>>>> Learn why this is important at
>>>>> https://aka.ms/LearnAboutSenderIdentification ]
>>>>>
>>>>> On Thu Aug 14, 2025 at 7:23 PM CEST, Tony Rex via lists.openembedded.org wrote:
>>>>>> It seems that the race condition happens when doing do_fetch on
>>>>>> really small repos and it can be difficult to reproduce so below
>>>>>> the other case with floating tag/branch that is easier to verify.
>>>>>>
>>>>>> To verify the floating tag/branch issue have s simple recipe
>>>>>> hello_1.0.bb and set the SRC_URI to SRC_URI = "git://<repository>;protocol=https;user=${USER};branch=master;tag=v1.0"
>>>>>>
>>>>>> run command bitbake hello everything builds.
>>>>>>
>>>>>> Apply the patch
>>>>>>
>>>>>> run command bitbake hello
>>>>>> ERROR: hello-1.0-r1 do_fetch: Bitbake Fetcher Error:
>>>>>> FetchError("Recipe uses a floating tag/branch 'v1.0' for repo
>>>>>> '<repository>' without a fixed SRCREV yet doesn't call
>>>>>> bb.fetch2.get_srcrev() (use SRCPV in PV for OE).", None)
>>>>>> ERROR: Logfile of failure stored in:
>>>>>> hello/1.0/temp/log.do_fetch.3289932
>>>>>> ERROR: Task (hello/hello_1.0.bb:do_fetch) failed with exit code '1'
>>>>>>
>>>>>> Tony Rex (1):
>>>>>>   base.bbclass: Fix the recipe parsing race issue
>>>>>>
>>>>>>  meta/classes-global/base.bbclass | 2 +-
>>>>>>  1 file changed, 1 insertion(+), 1 deletion(-)
>>>>> Hi Tony,
>>>>>
>>>>> Thanks for your patch.
>>>>>
>>>>> It looks like this is breaking some builds with the following error:
>>>>>
>>>>> oeqa.utils.subprocesstweak.OETestCalledProcessError: Command '. /srv/pokybuild/yocto-worker/qemux86-64/build/build/tmp/work/qemux86_64-poky-linux/core-image-sato/1.0/testsdkext/environment-setup-core2-64-poky-linux > /dev/null; devtool add kernel-module-hello-world https://git.yoctoproject.org/git/kernel-module-hello-world;' returned non-zero exit status 1.
>>>>> ...
>>>>> ERROR: Bitbake Fetcher Error: FetchError("Recipe uses a floating
>>>>> tag/branch 'master' for repo
>>>>> 'git.yoctoproject.org/git/kernel-module-hello-world' without a
>>>>> fixed SRCREV yet doesn't call bb.fetch2.get_srcrev() (use SRCPV in
>>>>> PV for OE).", None)
>>>>>
>>>>> https://auto/
>>>>> builder.yoctoproject.org%2Fvalkyrie%2F%23%2Fbuilders%2F68%2Fbuilds%
>>>>> 2F
>>>>> 2
>>>>> 284&data=05%7C02%7Ctony.rex%40ericsson.com%7Cde1ae2aceeda470b381608
>>>>> dd
>>>>> d
>>>>> efdd730%7C92e84cebfbfd47abbe52080c6b87953f%7C0%7C0%7C63891190400520
>>>>> 90
>>>>> 0
>>>>> 3%7CUnknown%7CTWFpbGZsb3d8eyJFbXB0eU1hcGkiOnRydWUsIlYiOiIwLjAuMDAwM
>>>>> CI
>>>>> s
>>>>> IlAiOiJXaW4zMiIsIkFOIjoiTWFpbCIsIldUIjoyfQ%3D%3D%7C0%7C%7C%7C&sdata
>>>>> =m
>>>>> P
>>>>> UXMdb%2FYOyGIm0rnncaWgcqtwkUIh8364IP3eYDPe4%3D&reserved=0
>>>>> https://auto/
>>>>> builder.yoctoproject.org%2Fvalkyrie%2F%23%2Fbuilders%2F30%2Fbuilds%
>>>>> 2F
>>>>> 2
>>>>> 199&data=05%7C02%7Ctony.rex%40ericsson.com%7Cde1ae2aceeda470b381608
>>>>> dd
>>>>> d
>>>>> efdd730%7C92e84cebfbfd47abbe52080c6b87953f%7C0%7C0%7C63891190400522
>>>>> 74
>>>>> 3
>>>>> 2%7CUnknown%7CTWFpbGZsb3d8eyJFbXB0eU1hcGkiOnRydWUsIlYiOiIwLjAuMDAwM
>>>>> CI
>>>>> s
>>>>> IlAiOiJXaW4zMiIsIkFOIjoiTWFpbCIsIldUIjoyfQ%3D%3D%7C0%7C%7C%7C&sdata
>>>>> =R
>>>>> 0
>>>>> HFameGl2LmIVMT9ml%2BDMnj%2FMbG6SW0%2F3yNZHQu2js%3D&reserved=0
>>>>> https://auto/
>>>>> builder.yoctoproject.org%2Fvalkyrie%2F%23%2Fbuilders%2F16%2Fbuilds%
>>>>> 2F
>>>>> 2
>>>>> 247&data=05%7C02%7Ctony.rex%40ericsson.com%7Cde1ae2aceeda470b381608
>>>>> dd
>>>>> d
>>>>> efdd730%7C92e84cebfbfd47abbe52080c6b87953f%7C0%7C0%7C63891190400524
>>>>> 53
>>>>> 8
>>>>> 7%7CUnknown%7CTWFpbGZsb3d8eyJFbXB0eU1hcGkiOnRydWUsIlYiOiIwLjAuMDAwM
>>>>> CI
>>>>> s
>>>>> IlAiOiJXaW4zMiIsIkFOIjoiTWFpbCIsIldUIjoyfQ%3D%3D%7C0%7C%7C%7C&sdata
>>>>> =N
>>>>> S
>>>>> FhEupwMkgqUAhpRgFbKnyFl%2FsY%2B4UJMhVQHhqOaNM%3D&reserved=0
>>>>>
>>>>> Can you have a look at this error please?
>>>>>
>>>>> Best regards,
>>>>> Mathieu
>>>>>
>>>>> --
>>>>> Mathieu Dubois-Briand, Bootlin
>>>>> Embedded Linux and Kernel engineering https://boot/
>>>>> lin.com%2F&data=05%7C02%7Ctony.rex%40ericsson.com%7Cde1ae2aceeda470
>>>>> b3
>>>>> 8
>>>>> 1608dddefdd730%7C92e84cebfbfd47abbe52080c6b87953f%7C0%7C0%7C6389119
>>>>> 04
>>>>> 0
>>>>> 05263327%7CUnknown%7CTWFpbGZsb3d8eyJFbXB0eU1hcGkiOnRydWUsIlYiOiIwLj
>>>>> Au
>>>>> M
>>>>> DAwMCIsIlAiOiJXaW4zMiIsIkFOIjoiTWFpbCIsIldUIjoyfQ%3D%3D%7C0%7C%7C%7
>>>>> C&
>>>>> s
>>>>> data=oyi7YWsXDVHn7QB4fDdG1sBp7gQsfGWcJTWtX1CjAw4%3D&reserved=0
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>> -=-=-=-=-=-=-=-=-=-=-=-
>> Links: You receive all messages sent to this group.
>> View/Reply Online (#222190):
>> https://list/
>> s.openembedded.org%2Fg%2Fopenembedded-core%2Fmessage%2F222190&data=05%
>> 7C02%7Ctony.rex%40ericsson.com%7C7b5c099d170043080cca08dde004478d%7C92
>> e84cebfbfd47abbe52080c6b87953f%7C0%7C0%7C638913031194226137%7CUnknown%
>> 7CTWFpbGZsb3d8eyJFbXB0eU1hcGkiOnRydWUsIlYiOiIwLjAuMDAwMCIsIlAiOiJXaW4z
>> MiIsIkFOIjoiTWFpbCIsIldUIjoyfQ%3D%3D%7C0%7C%7C%7C&sdata=WbB87ExjlYOSmq
>> I0%2BEee6LbpupPfWxzk0vNYlbCUP5g%3D&reserved=0
>> Mute This Topic:
>> https://list/
>> s.openembedded.org%2Fmt%2F114706197%2F6084445&data=05%7C02%7Ctony.rex%
>> 40ericsson.com%7C7b5c099d170043080cca08dde004478d%7C92e84cebfbfd47abbe
>> 52080c6b87953f%7C0%7C0%7C638913031194241729%7CUnknown%7CTWFpbGZsb3d8ey
>> JFbXB0eU1hcGkiOnRydWUsIlYiOiIwLjAuMDAwMCIsIlAiOiJXaW4zMiIsIkFOIjoiTWFp
>> bCIsIldUIjoyfQ%3D%3D%7C0%7C%7C%7C&sdata=bHfsli%2Fba3g8lov%2Bxmfovaal5C
>> uBGuFnZVXs8AjHGIY%3D&reserved=0 Group Owner:
>> openembedded-core+owner@lists.openembedded.org
>> Unsubscribe:
>> https://list/
>> s.openembedded.org%2Fg%2Fopenembedded-core%2Funsub&data=05%7C02%7Ctony
>> .rex%40ericsson.com%7C7b5c099d170043080cca08dde004478d%7C92e84cebfbfd4
>> 7abbe52080c6b87953f%7C0%7C0%7C638913031194543958%7CUnknown%7CTWFpbGZsb
>> 3d8eyJFbXB0eU1hcGkiOnRydWUsIlYiOiIwLjAuMDAwMCIsIlAiOiJXaW4zMiIsIkFOIjo
>> iTWFpbCIsIldUIjoyfQ%3D%3D%7C0%7C%7C%7C&sdata=gy4aai6oflVendGBp8nbYAL8K
>> CGj77ObOMy2rYDnGN0%3D&reserved=0 [skandigraun@gmail.com]
>> -=-=-=-=-=-=-=-=-=-=-=-
>>

Re: [OE-core] [PATCH 0/1] base.bbclass: Fix the recipe parsing race issue

[Tony Rex]

I'm not sure how to run your (Gyorgy) code inside a recipe to test it, but in a bash script it works, of course.

The thing that I'm looking at now is  a bit confusing, when commenting out the fetcher_hashes_dummyfunc function from the base.bbclass it
seems to work properly.

I don't know what the real purpose of this function is, am I missing something here, like a certain setup or a corner case?
It seems to work both on whinlatter and scarthgap. Can someone please help out and clarify.

git diff classes-global/base.bbclass
diff --git a/meta/classes-global/base.bbclass b/meta/classes-global/base.bbclass
index 6c08870571..83f82b2a4d 100644
--- a/meta/classes-global/base.bbclass
+++ b/meta/classes-global/base.bbclass
@@ -128,16 +128,16 @@ def setup_hosttools_dir(dest, toolsvar, d, fatal=True):

 # We can't use vardepvalue against do_fetch directly since that would overwrite
 # the other task dependencies so we use an indirect function.
-python fetcher_hashes_dummyfunc() {
-    return
-}
-fetcher_hashes_dummyfunc[vardepvalue] = "${@bb.fetch.get_hashvalue(d)}"
+#python fetcher_hashes_dummyfunc() {
+#    return
+#}
+#fetcher_hashes_dummyfunc[vardepvalue] = "${@bb.fetch.get_hashvalue(d)}"

 addtask fetch
 do_fetch[dirs] = "${DL_DIR}"
 do_fetch[file-checksums] = "${@bb.fetch.get_checksum_file_list(d)}"
 do_fetch[file-checksums] += " ${@get_lic_checksum_file_list(d)}"
-do_fetch[prefuncs] += "fetcher_hashes_dummyfunc"
+#do_fetch[prefuncs] += "fetcher_hashes_dummyfunc"
 do_fetch[network] = "1"
 python base_do_fetch() {

Tony

-----Original Message-----
From: Gyorgy Sarvari <skandigraun@gmail.com>
Sent: Thursday, 21 August 2025 09:30
To: Tony Rex <tony.rex@ericsson.com>; Alexander Kanavin <alex.kanavin@gmail.com>
Cc: Mathieu Dubois-Briand <mathieu.dubois-briand@bootlin.com>; openembedded-core@lists.openembedded.org
Subject: Re: [OE-core] [PATCH 0/1] base.bbclass: Fix the recipe parsing race issue

On 8/21/25 09:10, Tony Rex wrote:
> Yes, I get the same or similar error if the patch is not applied.
>
> ERROR: ExpansionError during parsing .../generic-if.bb################################   | ETA:  0:00:00
> bb.data_smart.ExpansionError: Failure expanding variable fetcher_hashes_dummyfunc[vardepvalue], expression was ${@bb.fetch.get_hashvalue(d)} which triggered exception FetchError: Fetcher failure: Unable to resolve '${TAGNAME}-1.0' in upstream git repository in git ls-remote output for gerrit...
> The variable dependency chain for the failure is:
> fetcher_hashes_dummyfunc[vardepvalue]
>
> ERROR: Parsing halted due to errors, see error messages above
>
> How does your mytest_get.bb look like? Since I have had problem with reproducing the error on in a common setup maybe it can be done with your bb file.
>
> Tony

The failure I reproduced is not recipe dependent. As some diagnostic test, try this script a few times with your failing git repo (change the repo placeholder):

for i in `seq 50`; do
  git clone $GIT_REPO_URL target_folder
  git -C ./target_folder ls-remote
  if [ $? -ne 0 ]; then
    echo ls-remote failed, iteration: $i
    break
  fi
  rm -rf ./target_folder
done

If I haven't made a typo, the script should clone your repo, do an ls-remote inside it, delete the folder, and do the same again.

If your problem is the same that I reproduced, then I would expect the above script to fail sometimes - if it does, then the problem is with the infrastructure (network, remote git server, etc... something is flaky).

Just for the record: the script generates a small amount of successive requests which some network admins might consider flooding.



> -----Original Message-----
> From: Gyorgy Sarvari <skandigraun@gmail.com>
> Sent: Wednesday, 20 August 2025 18:12
> To: Tony Rex <tony.rex@ericsson.com>; Alexander Kanavin
> <alex.kanavin@gmail.com>
> Cc: Mathieu Dubois-Briand <mathieu.dubois-briand@bootlin.com>;
> openembedded-core@lists.openembedded.org
> Subject: Re: [OE-core] [PATCH 0/1] base.bbclass: Fix the recipe
> parsing race issue
>
> [You don't often get email from skandigraun@gmail.com. Learn why this
> is important at https://aka.ms/LearnAboutSenderIdentification ]
>
> On 8/20/25 17:44, Gyorgy Sarvari via lists.openembedded.org wrote:
>> On 8/20/25 13:53, Tony Rex wrote:
>>> It was introduced by this commit:
>>> https://git/
>>> .yoctoproject.org%2Fpoky%2Fcommit%2F%3Fid%3Debfa1700f41b3411aec04014
>>> 4
>>> 605166c35b8dd14&data=05%7C02%7Ctony.rex%40ericsson.com%7C7b5c099d170
>>> 0
>>> 43080cca08dde004478d%7C92e84cebfbfd47abbe52080c6b87953f%7C0%7C0%7C63
>>> 8
>>> 913031194171335%7CUnknown%7CTWFpbGZsb3d8eyJFbXB0eU1hcGkiOnRydWUsIlYi
>>> O
>>> iIwLjAuMDAwMCIsIlAiOiJXaW4zMiIsIkFOIjoiTWFpbCIsIldUIjoyfQ%3D%3D%7C0%
>>> 7
>>> C%7C%7C&sdata=iYvLGK1DwJk78tEvmL%2FPOL2olj7iNL87eXqdsEraxLw%3D&reser
>>> v
>>> ed=0
>>>
>>> Maybe "not allowed" is a strong wording, but that was how I interpreted it. Sorry, my bad if I misinterpreted it.
>> In my understanding (which could be wrong), this is more to avoid
>> working on a repository, where the exact work-revision can't be
>> determined. But in your example, it can be determined, if your patch
>> is not applied.
>>
>> Some extra ramblings, which may or may not be unadulterated garbage
>> from
>> me: Your initial commit message mentioned an expansion error. I
>> looked into that a bit, and is it possible that the repository that
>> is causing this issue has some access rate limit? The only thing that
>> can fail sometimes in that code path seems to be "git ls-remote".
>>
>> Now, when I noticed this, I compiled a git version for myself that
>> randomly throws an error with "git ls-remote" command, and when I
>> used it with bitbake, I got this:
>>
>> ERROR: ExpansionError during parsing
>> /home/meee/stuff/ptest-images/poky/meta/recipes-core/ell/mytest_get.b
>> b
>> bb.data_smart.ExpansionError: Failure expanding variable
>> fetcher_hashes_dummyfunc[vardepvalue], expression was <etc etc etc>
>>
>> Is this the same error you got?
> Sorry for the spam, one thing I forgot - the error on my end also contains details about the command that failed, pointing to git. If it's not the same as you get, do you have a copy of the full bitbake output you get when you see this error?
>
>>> Tony
>>>
>>> -----Original Message-----
>>> From: Gyorgy Sarvari <skandigraun@gmail.com>
>>> Sent: Wednesday, 20 August 2025 12:18
>>> To: Tony Rex <tony.rex@ericsson.com>; Alexander Kanavin
>>> <alex.kanavin@gmail.com>
>>> Cc: Mathieu Dubois-Briand <mathieu.dubois-briand@bootlin.com>;
>>> openembedded-core@lists.openembedded.org
>>> Subject: Re: [OE-core] [PATCH 0/1] base.bbclass: Fix the recipe
>>> parsing race issue
>>>
>>> [You don't often get email from skandigraun@gmail.com. Learn why
>>> this is important at https://aka.ms/LearnAboutSenderIdentification ]
>>>
>>> On 8/19/25 23:10, Tony Rex via lists.openembedded.org wrote:
>>>> Hi Alex and Mathieu!
>>>>
>>>> Ok, so the patch fixes a bug and then the floating tag/branch function is in place again.
>>>> The floating tag/branch function is broken due to the bug and the patch fixes it.
>>>>
>>>> The mail from Mathieu below might not fully reflect the issue here, since it seems to create a recipe with the help of devtool.
>>>>
>>>> To verify:
>>>> 1. add layers/oe-core/meta-selftest to conf/bblayers.conf 2. run
>>>> bitbake gitrepotest -c fetch
>>>> result:
>>>> NOTE: Tasks Summary: Attempted 2 tasks of which 0 didn't need to be rerun and all succeeded.
>>>> 3. run bitbake gitrepotest -c cleanall 4. edit
>>>> ../layers/oe-core/meta-selftest/recipes-test/gitrepotest/gitrepotes
>>>> t
>>>> .b
>>>> b
>>>> add tag=2.12 in SRC_URI
>>>> SRC_URI =
>>>> "git://git.yoctoproject.org/git/matchbox-panel-2;branch=master;tag=
>>>> 2
>>>> .1 2;protocol=https \ and comment out SRCREV #SRCREV =
>>>> "f82ca3f42510fb3ef10f598b393eb373a2c34ca7"
>>>> 5. run bitbake gitrepotest -c fetch
>>>> result:
>>>> NOTE: Tasks Summary: Attempted 2 tasks of which 0 didn't need to be rerun and all succeeded.
>>>> The fetch goes through although this is not allowed according to the floating tag/branch error statement if not d.getVar("__BBSRCREV_SEEN"): in git.py.
>>> Could you point to a commit/email/etc about this "no floating tags
>>> allowed" statement? AFAIK tags without specifying revision is
>>> allowed and supported. Not recommended, due to mandatory network
>>> access, and because git tags are not immutable, but I'm surprised to
>>> hear that they are not supposed to work (which of course can also
>>> mean that I just missed a memo)
>>>
>>>> 6. run bitbake gitrepotest -c cleanall 7. apply patch
>>>> 0001-base.bbclass-Fix-the-recipe-parsing-race-issue.patch
>>>> 8. run bitbake gitrepotest -c fetch
>>>> result:
>>>> ERROR: gitrepotest-1.0-r0 do_fetch: Bitbake Fetcher Error:
>>>> FetchError("Recipe uses a floating tag/branch '2.12' for repo
>>>> 'git.yoctoproject.org/git/matchbox-panel-2' without a fixed SRCREV
>>>> yet doesn't call bb.fetch2.get_srcrev() (use SRCPV in PV for OE).",
>>>> None)
>>>> ERROR: Logfile of failure stored in:
>>>> .../build/tmp-glibc/work/cortexa53-wrs-linux/gitrepotest/1.0/temp/log.
>>>> do_fetch.1254999
>>>> ERROR: Task (.../layers/oe-core/meta-selftest/recipes-test/gitrepotest/gitrepotest.bb:do_fetch) failed with exit code '1'
>>>> NOTE: Tasks Summary: Attempted 2 tasks of which 0 didn't need to be rerun and 1 failed.
>>>>
>>>> Error is the expected result when only having a tag set.
>>>>
>>>> Tony
>>>>
>>>> -----Original Message-----
>>>> From: Alexander Kanavin <alex.kanavin@gmail.com>
>>>> Sent: Tuesday, 19 August 2025 10:53
>>>> To: Tony Rex <tony.rex@ericsson.com>
>>>> Cc: Mathieu Dubois-Briand <mathieu.dubois-briand@bootlin.com>;
>>>> openembedded-core@lists.openembedded.org
>>>> Subject: Re: [OE-core] [PATCH 0/1] base.bbclass: Fix the recipe
>>>> parsing race issue
>>>>
>>>> [You don't often get email from alex.kanavin@gmail.com. Learn why
>>>> this is important at https://aka.ms/LearnAboutSenderIdentification
>>>> ]
>>>>
>>>> You need to make things a little bit easier for maintainers and try to save their time.
>>>>
>>>> I would suggest that you provide clear steps to observe the issue.
>>>> Attach the recipe file that would demonstrate the issue, and provide clear steps to trigger it. What is confusing right now is that you say here's a recipe that builds ok, but then you 'apply the patch' and then there's some kind of failure. So which patch needs to be applied?
>>>>
>>>> Alex
>>>>
>>>> On Fri, 15 Aug 2025 at 17:29, Tony Rex via lists.openembedded.org <tony.rex=ericsson.com@lists.openembedded.org> wrote:
>>>>> Hi Mathieu!
>>>>>
>>>>> I think that this is how it is intended to work, bitbake does not allow floating tag/branch anymore.
>>>>> Please correct me if I'm wrong @Richard Purdie (who implemented
>>>>> this for security reasons I think.)
>>>>>
>>>>> The patch that I have makes sure that the parsing and setting of
>>>>> the variables in the recipe are done before executing the do_fetch command, and as a side effect the floating tag/branch protection is working as intended again.
>>>>>
>>>>> How does the recipe look like?
>>>>> devtool add kernel-module-hello-world https://git/.
>>>>> yoctoproject.org%2Fgit%2Fkernel-module-hello-world&data=05%7C02%7C
>>>>> t
>>>>> on
>>>>> y
>>>>> .rex%40ericsson.com%7Cde1ae2aceeda470b381608dddefdd730%7C92e84cebf
>>>>> b
>>>>> fd
>>>>> 4
>>>>> 7abbe52080c6b87953f%7C0%7C0%7C638911904005151487%7CUnknown%7CTWFpb
>>>>> G
>>>>> Zs
>>>>> b
>>>>> 3d8eyJFbXB0eU1hcGkiOnRydWUsIlYiOiIwLjAuMDAwMCIsIlAiOiJXaW4zMiIsIkF
>>>>> O
>>>>> Ij
>>>>> o
>>>>> iTWFpbCIsIldUIjoyfQ%3D%3D%7C0%7C%7C%7C&sdata=jyCicRQT%2FLtb1YxiFHK
>>>>> g
>>>>> w5
>>>>> B
>>>>> nNktN4t9IciiyxtsEwO4%3D&reserved=0
>>>>> The one I get looks like this:
>>>>>
>>>>> cat
>>>>> build/workspace/recipes/kernel-module-hello-world/kernel-module-he
>>>>> l
>>>>> lo
>>>>> -
>>>>> world_git.bb
>>>>> # Recipe created by recipetool
>>>>> # This is the basis of a recipe and may need further editing in order to be fully functional.
>>>>> # (Feel free to remove these comments when editing.)
>>>>>
>>>>> # WARNING: the following LICENSE and LIC_FILES_CHKSUM values are
>>>>> best guesses - it is # your responsibility to verify that the values are complete and correct.
>>>>> #
>>>>> # The following license files were not able to be identified and
>>>>> are # represented as "Unknown" below, you will need to check them yourself:
>>>>> #   LICENSE
>>>>> LICENSE = "Unknown"
>>>>> LIC_FILES_CHKSUM = "file://LICENSE;md5=7b785be6433a2e4af6a641cc7f0330d1"
>>>>>
>>>>> SRC_URI = "git://git.yoctoproject.org/git/kernel-module-hello-world;protocol=https;branch=master"
>>>>>
>>>>> # Modify these as desired
>>>>> PV = "1.0+git"
>>>>> SRCREV = "b66f0f2bd559a1df81fc1bda240eba07a2ecdd2b"
>>>>>
>>>>> S = "${WORKDIR}/git"
>>>>>
>>>>> inherit module
>>>>>
>>>>> EXTRA_OEMAKE:append:task-install = " -C ${STAGING_KERNEL_DIR} M=${S}"
>>>>> EXTRA_OEMAKE += "KERNEL_DIR=${STAGING_KERNEL_DIR}"
>>>>>
>>>>> And it contains SRCREV = "b66f0f2bd559a1df81fc1bda240eba07a2ecdd2b"
>>>>>
>>>>> Tony
>>>>>
>>>>> -----Original Message-----
>>>>> From: Mathieu Dubois-Briand <mathieu.dubois-briand@bootlin.com>
>>>>> Sent: Friday, 15 August 2025 11:26
>>>>> To: Tony Rex <tony.rex@ericsson.com>;
>>>>> openembedded-core@lists.openembedded.org
>>>>> Subject: Re: [OE-core] [PATCH 0/1] base.bbclass: Fix the recipe
>>>>> parsing race issue
>>>>>
>>>>> [You don't often get email from mathieu.dubois-briand@bootlin.com.
>>>>> Learn why this is important at
>>>>> https://aka.ms/LearnAboutSenderIdentification ]
>>>>>
>>>>> On Thu Aug 14, 2025 at 7:23 PM CEST, Tony Rex via lists.openembedded.org wrote:
>>>>>> It seems that the race condition happens when doing do_fetch on
>>>>>> really small repos and it can be difficult to reproduce so below
>>>>>> the other case with floating tag/branch that is easier to verify.
>>>>>>
>>>>>> To verify the floating tag/branch issue have s simple recipe
>>>>>> hello_1.0.bb and set the SRC_URI to SRC_URI = "git://<repository>;protocol=https;user=${USER};branch=master;tag=v1.0"
>>>>>>
>>>>>> run command bitbake hello everything builds.
>>>>>>
>>>>>> Apply the patch
>>>>>>
>>>>>> run command bitbake hello
>>>>>> ERROR: hello-1.0-r1 do_fetch: Bitbake Fetcher Error:
>>>>>> FetchError("Recipe uses a floating tag/branch 'v1.0' for repo
>>>>>> '<repository>' without a fixed SRCREV yet doesn't call
>>>>>> bb.fetch2.get_srcrev() (use SRCPV in PV for OE).", None)
>>>>>> ERROR: Logfile of failure stored in:
>>>>>> hello/1.0/temp/log.do_fetch.3289932
>>>>>> ERROR: Task (hello/hello_1.0.bb:do_fetch) failed with exit code '1'
>>>>>>
>>>>>> Tony Rex (1):
>>>>>>   base.bbclass: Fix the recipe parsing race issue
>>>>>>
>>>>>>  meta/classes-global/base.bbclass | 2 +-
>>>>>>  1 file changed, 1 insertion(+), 1 deletion(-)
>>>>> Hi Tony,
>>>>>
>>>>> Thanks for your patch.
>>>>>
>>>>> It looks like this is breaking some builds with the following error:
>>>>>
>>>>> oeqa.utils.subprocesstweak.OETestCalledProcessError: Command '. /srv/pokybuild/yocto-worker/qemux86-64/build/build/tmp/work/qemux86_64-poky-linux/core-image-sato/1.0/testsdkext/environment-setup-core2-64-poky-linux > /dev/null; devtool add kernel-module-hello-world https://git.yoctoproject.org/git/kernel-module-hello-world;' returned non-zero exit status 1.
>>>>> ...
>>>>> ERROR: Bitbake Fetcher Error: FetchError("Recipe uses a floating
>>>>> tag/branch 'master' for repo
>>>>> 'git.yoctoproject.org/git/kernel-module-hello-world' without a
>>>>> fixed SRCREV yet doesn't call bb.fetch2.get_srcrev() (use SRCPV in
>>>>> PV for OE).", None)
>>>>>
>>>>> https://auto/
>>>>> builder.yoctoproject.org%2Fvalkyrie%2F%23%2Fbuilders%2F68%2Fbuilds
>>>>> %
>>>>> 2F
>>>>> 2
>>>>> 284&data=05%7C02%7Ctony.rex%40ericsson.com%7Cde1ae2aceeda470b38160
>>>>> 8
>>>>> dd
>>>>> d
>>>>> efdd730%7C92e84cebfbfd47abbe52080c6b87953f%7C0%7C0%7C6389119040052
>>>>> 0
>>>>> 90
>>>>> 0
>>>>> 3%7CUnknown%7CTWFpbGZsb3d8eyJFbXB0eU1hcGkiOnRydWUsIlYiOiIwLjAuMDAw
>>>>> M
>>>>> CI
>>>>> s
>>>>> IlAiOiJXaW4zMiIsIkFOIjoiTWFpbCIsIldUIjoyfQ%3D%3D%7C0%7C%7C%7C&sdat
>>>>> a
>>>>> =m
>>>>> P
>>>>> UXMdb%2FYOyGIm0rnncaWgcqtwkUIh8364IP3eYDPe4%3D&reserved=0
>>>>> https://auto/
>>>>> builder.yoctoproject.org%2Fvalkyrie%2F%23%2Fbuilders%2F30%2Fbuilds
>>>>> %
>>>>> 2F
>>>>> 2
>>>>> 199&data=05%7C02%7Ctony.rex%40ericsson.com%7Cde1ae2aceeda470b38160
>>>>> 8
>>>>> dd
>>>>> d
>>>>> efdd730%7C92e84cebfbfd47abbe52080c6b87953f%7C0%7C0%7C6389119040052
>>>>> 2
>>>>> 74
>>>>> 3
>>>>> 2%7CUnknown%7CTWFpbGZsb3d8eyJFbXB0eU1hcGkiOnRydWUsIlYiOiIwLjAuMDAw
>>>>> M
>>>>> CI
>>>>> s
>>>>> IlAiOiJXaW4zMiIsIkFOIjoiTWFpbCIsIldUIjoyfQ%3D%3D%7C0%7C%7C%7C&sdat
>>>>> a
>>>>> =R
>>>>> 0
>>>>> HFameGl2LmIVMT9ml%2BDMnj%2FMbG6SW0%2F3yNZHQu2js%3D&reserved=0
>>>>> https://auto/
>>>>> builder.yoctoproject.org%2Fvalkyrie%2F%23%2Fbuilders%2F16%2Fbuilds
>>>>> %
>>>>> 2F
>>>>> 2
>>>>> 247&data=05%7C02%7Ctony.rex%40ericsson.com%7Cde1ae2aceeda470b38160
>>>>> 8
>>>>> dd
>>>>> d
>>>>> efdd730%7C92e84cebfbfd47abbe52080c6b87953f%7C0%7C0%7C6389119040052
>>>>> 4
>>>>> 53
>>>>> 8
>>>>> 7%7CUnknown%7CTWFpbGZsb3d8eyJFbXB0eU1hcGkiOnRydWUsIlYiOiIwLjAuMDAw
>>>>> M
>>>>> CI
>>>>> s
>>>>> IlAiOiJXaW4zMiIsIkFOIjoiTWFpbCIsIldUIjoyfQ%3D%3D%7C0%7C%7C%7C&sdat
>>>>> a
>>>>> =N
>>>>> S
>>>>> FhEupwMkgqUAhpRgFbKnyFl%2FsY%2B4UJMhVQHhqOaNM%3D&reserved=0
>>>>>
>>>>> Can you have a look at this error please?
>>>>>
>>>>> Best regards,
>>>>> Mathieu
>>>>>
>>>>> --
>>>>> Mathieu Dubois-Briand, Bootlin
>>>>> Embedded Linux and Kernel engineering https://boot/
>>>>> lin.com%2F&data=05%7C02%7Ctony.rex%40ericsson.com%7Cde1ae2aceeda47
>>>>> 0
>>>>> b3
>>>>> 8
>>>>> 1608dddefdd730%7C92e84cebfbfd47abbe52080c6b87953f%7C0%7C0%7C638911
>>>>> 9
>>>>> 04
>>>>> 0
>>>>> 05263327%7CUnknown%7CTWFpbGZsb3d8eyJFbXB0eU1hcGkiOnRydWUsIlYiOiIwL
>>>>> j
>>>>> Au
>>>>> M
>>>>> DAwMCIsIlAiOiJXaW4zMiIsIkFOIjoiTWFpbCIsIldUIjoyfQ%3D%3D%7C0%7C%7C%
>>>>> 7
>>>>> C&
>>>>> s
>>>>> data=oyi7YWsXDVHn7QB4fDdG1sBp7gQsfGWcJTWtX1CjAw4%3D&reserved=0
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>> 
>>

Re: [OE-core] [PATCH 0/1] base.bbclass: Fix the recipe parsing race issue

[Tony Rex]

So to me it looks like the fetcher_hashes_dummyfunc makes it possible to catch any faulty or unwanted setups in the parsing stage.

It seems hard to reproduce and expose this error on a common target, but I can reproduce it easily in the local setup.
So I share my observations instead and hopefully you can make some sense out them.

The error that I'm seeing is that when fetcher_hashes_dummyfunc[vardepvalue] = "${@bb.fetch.get_hashvalue(d)}" is executed the PV is not set, nor is the PN.
So it seems that if the git ls-remote check happens to quick and early all variables are not set properly.
The repo in this case is small (140K) and I believe that the access to the git server is fast (don't know of it matters) the git ls-remote check does not contain the
right information if for example, the tag in the SRC_URI contains the PV value.

When doing printouts from get_hashvalue the revs variable is empty and PN = defaultpkgname and PV = 1.0 has default values so the variables doesn't seem to be
properly set yet and gives this error:

ERROR: ExpansionError during parsing .../generic-if.bb################################   | ETA:  0:00:00
bb.data_smart.ExpansionError: Failure expanding variable fetcher_hashes_dummyfunc[vardepvalue], expression was ${@bb.fetch.get_hashvalue(d)} which triggered exception FetchError: Fetcher failure: Unable to resolve '${TAGNAME}-1.0' in upstream git repository in git ls-remote output for gerrit...
The variable dependency chain for the failure is:
fetcher_hashes_dummyfunc[vardepvalue]

ERROR: Parsing halted due to errors, see error messages above

The patch that is provided fixes this issue, but there might be other and/or maybe better solutions to this matter.
Another workaround for this problem that I have tested and that also seems to get around this issue is to set the SRC_URI with := or ?=

This is why I consider it to be a race condition during parsing and the git ls-remote check.

Tony

-----Original Message-----
From: Tony Rex
Sent: Thursday, 21 August 2025 17:48
To: Gyorgy Sarvari <skandigraun@gmail.com>; Alexander Kanavin <alex.kanavin@gmail.com>
Cc: Mathieu Dubois-Briand <mathieu.dubois-briand@bootlin.com>; openembedded-core@lists.openembedded.org
Subject: RE: [OE-core] [PATCH 0/1] base.bbclass: Fix the recipe parsing race issue

I'm not sure how to run your (Gyorgy) code inside a recipe to test it, but in a bash script it works, of course.

The thing that I'm looking at now is  a bit confusing, when commenting out the fetcher_hashes_dummyfunc function from the base.bbclass it seems to work properly.

I don't know what the real purpose of this function is, am I missing something here, like a certain setup or a corner case?
It seems to work both on whinlatter and scarthgap. Can someone please help out and clarify.

git diff classes-global/base.bbclass
diff --git a/meta/classes-global/base.bbclass b/meta/classes-global/base.bbclass
index 6c08870571..83f82b2a4d 100644
--- a/meta/classes-global/base.bbclass
+++ b/meta/classes-global/base.bbclass
@@ -128,16 +128,16 @@ def setup_hosttools_dir(dest, toolsvar, d, fatal=True):

 # We can't use vardepvalue against do_fetch directly since that would overwrite  # the other task dependencies so we use an indirect function.
-python fetcher_hashes_dummyfunc() {
-    return
-}
-fetcher_hashes_dummyfunc[vardepvalue] = "${@bb.fetch.get_hashvalue(d)}"
+#python fetcher_hashes_dummyfunc() {
+#    return
+#}
+#fetcher_hashes_dummyfunc[vardepvalue] = "${@bb.fetch.get_hashvalue(d)}"

 addtask fetch
 do_fetch[dirs] = "${DL_DIR}"
 do_fetch[file-checksums] = "${@bb.fetch.get_checksum_file_list(d)}"
 do_fetch[file-checksums] += " ${@get_lic_checksum_file_list(d)}"
-do_fetch[prefuncs] += "fetcher_hashes_dummyfunc"
+#do_fetch[prefuncs] += "fetcher_hashes_dummyfunc"
 do_fetch[network] = "1"
 python base_do_fetch() {

Tony

-----Original Message-----
From: Gyorgy Sarvari <skandigraun@gmail.com>
Sent: Thursday, 21 August 2025 09:30
To: Tony Rex <tony.rex@ericsson.com>; Alexander Kanavin <alex.kanavin@gmail.com>
Cc: Mathieu Dubois-Briand <mathieu.dubois-briand@bootlin.com>; openembedded-core@lists.openembedded.org
Subject: Re: [OE-core] [PATCH 0/1] base.bbclass: Fix the recipe parsing race issue

On 8/21/25 09:10, Tony Rex wrote:
> Yes, I get the same or similar error if the patch is not applied.
>
> ERROR: ExpansionError during parsing .../generic-if.bb################################   | ETA:  0:00:00
> bb.data_smart.ExpansionError: Failure expanding variable fetcher_hashes_dummyfunc[vardepvalue], expression was ${@bb.fetch.get_hashvalue(d)} which triggered exception FetchError: Fetcher failure: Unable to resolve '${TAGNAME}-1.0' in upstream git repository in git ls-remote output for gerrit...
> The variable dependency chain for the failure is:
> fetcher_hashes_dummyfunc[vardepvalue]
>
> ERROR: Parsing halted due to errors, see error messages above
>
> How does your mytest_get.bb look like? Since I have had problem with reproducing the error on in a common setup maybe it can be done with your bb file.
>
> Tony

The failure I reproduced is not recipe dependent. As some diagnostic test, try this script a few times with your failing git repo (change the repo placeholder):

for i in `seq 50`; do
  git clone $GIT_REPO_URL target_folder
  git -C ./target_folder ls-remote
  if [ $? -ne 0 ]; then
    echo ls-remote failed, iteration: $i
    break
  fi
  rm -rf ./target_folder
done

If I haven't made a typo, the script should clone your repo, do an ls-remote inside it, delete the folder, and do the same again.

If your problem is the same that I reproduced, then I would expect the above script to fail sometimes - if it does, then the problem is with the infrastructure (network, remote git server, etc... something is flaky).

Just for the record: the script generates a small amount of successive requests which some network admins might consider flooding.



> -----Original Message-----
> From: Gyorgy Sarvari <skandigraun@gmail.com>
> Sent: Wednesday, 20 August 2025 18:12
> To: Tony Rex <tony.rex@ericsson.com>; Alexander Kanavin
> <alex.kanavin@gmail.com>
> Cc: Mathieu Dubois-Briand <mathieu.dubois-briand@bootlin.com>;
> openembedded-core@lists.openembedded.org
> Subject: Re: [OE-core] [PATCH 0/1] base.bbclass: Fix the recipe
> parsing race issue
>
> [You don't often get email from skandigraun@gmail.com. Learn why this
> is important at https://aka.ms/LearnAboutSenderIdentification ]
>
> On 8/20/25 17:44, Gyorgy Sarvari via lists.openembedded.org wrote:
>> On 8/20/25 13:53, Tony Rex wrote:
>>> It was introduced by this commit:
>>> https://git/
>>> .yoctoproject.org%2Fpoky%2Fcommit%2F%3Fid%3Debfa1700f41b3411aec04014
>>> 4
>>> 605166c35b8dd14&data=05%7C02%7Ctony.rex%40ericsson.com%7C7b5c099d170
>>> 0
>>> 43080cca08dde004478d%7C92e84cebfbfd47abbe52080c6b87953f%7C0%7C0%7C63
>>> 8
>>> 913031194171335%7CUnknown%7CTWFpbGZsb3d8eyJFbXB0eU1hcGkiOnRydWUsIlYi
>>> O
>>> iIwLjAuMDAwMCIsIlAiOiJXaW4zMiIsIkFOIjoiTWFpbCIsIldUIjoyfQ%3D%3D%7C0%
>>> 7
>>> C%7C%7C&sdata=iYvLGK1DwJk78tEvmL%2FPOL2olj7iNL87eXqdsEraxLw%3D&reser
>>> v
>>> ed=0
>>>
>>> Maybe "not allowed" is a strong wording, but that was how I interpreted it. Sorry, my bad if I misinterpreted it.
>> In my understanding (which could be wrong), this is more to avoid
>> working on a repository, where the exact work-revision can't be
>> determined. But in your example, it can be determined, if your patch
>> is not applied.
>>
>> Some extra ramblings, which may or may not be unadulterated garbage
>> from
>> me: Your initial commit message mentioned an expansion error. I
>> looked into that a bit, and is it possible that the repository that
>> is causing this issue has some access rate limit? The only thing that
>> can fail sometimes in that code path seems to be "git ls-remote".
>>
>> Now, when I noticed this, I compiled a git version for myself that
>> randomly throws an error with "git ls-remote" command, and when I
>> used it with bitbake, I got this:
>>
>> ERROR: ExpansionError during parsing
>> /home/meee/stuff/ptest-images/poky/meta/recipes-core/ell/mytest_get.b
>> b
>> bb.data_smart.ExpansionError: Failure expanding variable
>> fetcher_hashes_dummyfunc[vardepvalue], expression was <etc etc etc>
>>
>> Is this the same error you got?
> Sorry for the spam, one thing I forgot - the error on my end also contains details about the command that failed, pointing to git. If it's not the same as you get, do you have a copy of the full bitbake output you get when you see this error?
>
>>> Tony
>>>
>>> -----Original Message-----
>>> From: Gyorgy Sarvari <skandigraun@gmail.com>
>>> Sent: Wednesday, 20 August 2025 12:18
>>> To: Tony Rex <tony.rex@ericsson.com>; Alexander Kanavin
>>> <alex.kanavin@gmail.com>
>>> Cc: Mathieu Dubois-Briand <mathieu.dubois-briand@bootlin.com>;
>>> openembedded-core@lists.openembedded.org
>>> Subject: Re: [OE-core] [PATCH 0/1] base.bbclass: Fix the recipe
>>> parsing race issue
>>>
>>> [You don't often get email from skandigraun@gmail.com. Learn why
>>> this is important at https://aka.ms/LearnAboutSenderIdentification ]
>>>
>>> On 8/19/25 23:10, Tony Rex via lists.openembedded.org wrote:
>>>> Hi Alex and Mathieu!
>>>>
>>>> Ok, so the patch fixes a bug and then the floating tag/branch function is in place again.
>>>> The floating tag/branch function is broken due to the bug and the patch fixes it.
>>>>
>>>> The mail from Mathieu below might not fully reflect the issue here, since it seems to create a recipe with the help of devtool.
>>>>
>>>> To verify:
>>>> 1. add layers/oe-core/meta-selftest to conf/bblayers.conf 2. run
>>>> bitbake gitrepotest -c fetch
>>>> result:
>>>> NOTE: Tasks Summary: Attempted 2 tasks of which 0 didn't need to be rerun and all succeeded.
>>>> 3. run bitbake gitrepotest -c cleanall 4. edit
>>>> ../layers/oe-core/meta-selftest/recipes-test/gitrepotest/gitrepotes
>>>> t
>>>> .b
>>>> b
>>>> add tag=2.12 in SRC_URI
>>>> SRC_URI =
>>>> "git://git.yoctoproject.org/git/matchbox-panel-2;branch=master;tag=
>>>> 2
>>>> .1 2;protocol=https \ and comment out SRCREV #SRCREV =
>>>> "f82ca3f42510fb3ef10f598b393eb373a2c34ca7"
>>>> 5. run bitbake gitrepotest -c fetch
>>>> result:
>>>> NOTE: Tasks Summary: Attempted 2 tasks of which 0 didn't need to be rerun and all succeeded.
>>>> The fetch goes through although this is not allowed according to the floating tag/branch error statement if not d.getVar("__BBSRCREV_SEEN"): in git.py.
>>> Could you point to a commit/email/etc about this "no floating tags
>>> allowed" statement? AFAIK tags without specifying revision is
>>> allowed and supported. Not recommended, due to mandatory network
>>> access, and because git tags are not immutable, but I'm surprised to
>>> hear that they are not supposed to work (which of course can also
>>> mean that I just missed a memo)
>>>
>>>> 6. run bitbake gitrepotest -c cleanall 7. apply patch
>>>> 0001-base.bbclass-Fix-the-recipe-parsing-race-issue.patch
>>>> 8. run bitbake gitrepotest -c fetch
>>>> result:
>>>> ERROR: gitrepotest-1.0-r0 do_fetch: Bitbake Fetcher Error:
>>>> FetchError("Recipe uses a floating tag/branch '2.12' for repo
>>>> 'git.yoctoproject.org/git/matchbox-panel-2' without a fixed SRCREV
>>>> yet doesn't call bb.fetch2.get_srcrev() (use SRCPV in PV for OE).",
>>>> None)
>>>> ERROR: Logfile of failure stored in:
>>>> .../build/tmp-glibc/work/cortexa53-wrs-linux/gitrepotest/1.0/temp/log.
>>>> do_fetch.1254999
>>>> ERROR: Task (.../layers/oe-core/meta-selftest/recipes-test/gitrepotest/gitrepotest.bb:do_fetch) failed with exit code '1'
>>>> NOTE: Tasks Summary: Attempted 2 tasks of which 0 didn't need to be rerun and 1 failed.
>>>>
>>>> Error is the expected result when only having a tag set.
>>>>
>>>> Tony
>>>>
>>>> -----Original Message-----
>>>> From: Alexander Kanavin <alex.kanavin@gmail.com>
>>>> Sent: Tuesday, 19 August 2025 10:53
>>>> To: Tony Rex <tony.rex@ericsson.com>
>>>> Cc: Mathieu Dubois-Briand <mathieu.dubois-briand@bootlin.com>;
>>>> openembedded-core@lists.openembedded.org
>>>> Subject: Re: [OE-core] [PATCH 0/1] base.bbclass: Fix the recipe
>>>> parsing race issue
>>>>
>>>> [You don't often get email from alex.kanavin@gmail.com. Learn why
>>>> this is important at https://aka.ms/LearnAboutSenderIdentification
>>>> ]
>>>>
>>>> You need to make things a little bit easier for maintainers and try to save their time.
>>>>
>>>> I would suggest that you provide clear steps to observe the issue.
>>>> Attach the recipe file that would demonstrate the issue, and provide clear steps to trigger it. What is confusing right now is that you say here's a recipe that builds ok, but then you 'apply the patch' and then there's some kind of failure. So which patch needs to be applied?
>>>>
>>>> Alex
>>>>
>>>> On Fri, 15 Aug 2025 at 17:29, Tony Rex via lists.openembedded.org <tony.rex=ericsson.com@lists.openembedded.org> wrote:
>>>>> Hi Mathieu!
>>>>>
>>>>> I think that this is how it is intended to work, bitbake does not allow floating tag/branch anymore.
>>>>> Please correct me if I'm wrong @Richard Purdie (who implemented
>>>>> this for security reasons I think.)
>>>>>
>>>>> The patch that I have makes sure that the parsing and setting of
>>>>> the variables in the recipe are done before executing the do_fetch command, and as a side effect the floating tag/branch protection is working as intended again.
>>>>>
>>>>> How does the recipe look like?
>>>>> devtool add kernel-module-hello-world https://git/.
>>>>> yoctoproject.org%2Fgit%2Fkernel-module-hello-world&data=05%7C02%7C
>>>>> t
>>>>> on
>>>>> y
>>>>> .rex%40ericsson.com%7Cde1ae2aceeda470b381608dddefdd730%7C92e84cebf
>>>>> b
>>>>> fd
>>>>> 4
>>>>> 7abbe52080c6b87953f%7C0%7C0%7C638911904005151487%7CUnknown%7CTWFpb
>>>>> G
>>>>> Zs
>>>>> b
>>>>> 3d8eyJFbXB0eU1hcGkiOnRydWUsIlYiOiIwLjAuMDAwMCIsIlAiOiJXaW4zMiIsIkF
>>>>> O
>>>>> Ij
>>>>> o
>>>>> iTWFpbCIsIldUIjoyfQ%3D%3D%7C0%7C%7C%7C&sdata=jyCicRQT%2FLtb1YxiFHK
>>>>> g
>>>>> w5
>>>>> B
>>>>> nNktN4t9IciiyxtsEwO4%3D&reserved=0
>>>>> The one I get looks like this:
>>>>>
>>>>> cat
>>>>> build/workspace/recipes/kernel-module-hello-world/kernel-module-he
>>>>> l
>>>>> lo
>>>>> -
>>>>> world_git.bb
>>>>> # Recipe created by recipetool
>>>>> # This is the basis of a recipe and may need further editing in order to be fully functional.
>>>>> # (Feel free to remove these comments when editing.)
>>>>>
>>>>> # WARNING: the following LICENSE and LIC_FILES_CHKSUM values are
>>>>> best guesses - it is # your responsibility to verify that the values are complete and correct.
>>>>> #
>>>>> # The following license files were not able to be identified and
>>>>> are # represented as "Unknown" below, you will need to check them yourself:
>>>>> #   LICENSE
>>>>> LICENSE = "Unknown"
>>>>> LIC_FILES_CHKSUM = "file://LICENSE;md5=7b785be6433a2e4af6a641cc7f0330d1"
>>>>>
>>>>> SRC_URI = "git://git.yoctoproject.org/git/kernel-module-hello-world;protocol=https;branch=master"
>>>>>
>>>>> # Modify these as desired
>>>>> PV = "1.0+git"
>>>>> SRCREV = "b66f0f2bd559a1df81fc1bda240eba07a2ecdd2b"
>>>>>
>>>>> S = "${WORKDIR}/git"
>>>>>
>>>>> inherit module
>>>>>
>>>>> EXTRA_OEMAKE:append:task-install = " -C ${STAGING_KERNEL_DIR} M=${S}"
>>>>> EXTRA_OEMAKE += "KERNEL_DIR=${STAGING_KERNEL_DIR}"
>>>>>
>>>>> And it contains SRCREV = "b66f0f2bd559a1df81fc1bda240eba07a2ecdd2b"
>>>>>
>>>>> Tony
>>>>>
>>>>> -----Original Message-----
>>>>> From: Mathieu Dubois-Briand <mathieu.dubois-briand@bootlin.com>
>>>>> Sent: Friday, 15 August 2025 11:26
>>>>> To: Tony Rex <tony.rex@ericsson.com>;
>>>>> openembedded-core@lists.openembedded.org
>>>>> Subject: Re: [OE-core] [PATCH 0/1] base.bbclass: Fix the recipe
>>>>> parsing race issue
>>>>>
>>>>> [You don't often get email from mathieu.dubois-briand@bootlin.com.
>>>>> Learn why this is important at
>>>>> https://aka.ms/LearnAboutSenderIdentification ]
>>>>>
>>>>> On Thu Aug 14, 2025 at 7:23 PM CEST, Tony Rex via lists.openembedded.org wrote:
>>>>>> It seems that the race condition happens when doing do_fetch on
>>>>>> really small repos and it can be difficult to reproduce so below
>>>>>> the other case with floating tag/branch that is easier to verify.
>>>>>>
>>>>>> To verify the floating tag/branch issue have s simple recipe
>>>>>> hello_1.0.bb and set the SRC_URI to SRC_URI = "git://<repository>;protocol=https;user=${USER};branch=master;tag=v1.0"
>>>>>>
>>>>>> run command bitbake hello everything builds.
>>>>>>
>>>>>> Apply the patch
>>>>>>
>>>>>> run command bitbake hello
>>>>>> ERROR: hello-1.0-r1 do_fetch: Bitbake Fetcher Error:
>>>>>> FetchError("Recipe uses a floating tag/branch 'v1.0' for repo
>>>>>> '<repository>' without a fixed SRCREV yet doesn't call
>>>>>> bb.fetch2.get_srcrev() (use SRCPV in PV for OE).", None)
>>>>>> ERROR: Logfile of failure stored in:
>>>>>> hello/1.0/temp/log.do_fetch.3289932
>>>>>> ERROR: Task (hello/hello_1.0.bb:do_fetch) failed with exit code '1'
>>>>>>
>>>>>> Tony Rex (1):
>>>>>>   base.bbclass: Fix the recipe parsing race issue
>>>>>>
>>>>>>  meta/classes-global/base.bbclass | 2 +-
>>>>>>  1 file changed, 1 insertion(+), 1 deletion(-)
>>>>> Hi Tony,
>>>>>
>>>>> Thanks for your patch.
>>>>>
>>>>> It looks like this is breaking some builds with the following error:
>>>>>
>>>>> oeqa.utils.subprocesstweak.OETestCalledProcessError: Command '. /srv/pokybuild/yocto-worker/qemux86-64/build/build/tmp/work/qemux86_64-poky-linux/core-image-sato/1.0/testsdkext/environment-setup-core2-64-poky-linux > /dev/null; devtool add kernel-module-hello-world https://git.yoctoproject.org/git/kernel-module-hello-world;' returned non-zero exit status 1.
>>>>> ...
>>>>> ERROR: Bitbake Fetcher Error: FetchError("Recipe uses a floating
>>>>> tag/branch 'master' for repo
>>>>> 'git.yoctoproject.org/git/kernel-module-hello-world' without a
>>>>> fixed SRCREV yet doesn't call bb.fetch2.get_srcrev() (use SRCPV in
>>>>> PV for OE).", None)
>>>>>
>>>>> https://auto/
>>>>> builder.yoctoproject.org%2Fvalkyrie%2F%23%2Fbuilders%2F68%2Fbuilds
>>>>> %
>>>>> 2F
>>>>> 2
>>>>> 284&data=05%7C02%7Ctony.rex%40ericsson.com%7Cde1ae2aceeda470b38160
>>>>> 8
>>>>> dd
>>>>> d
>>>>> efdd730%7C92e84cebfbfd47abbe52080c6b87953f%7C0%7C0%7C6389119040052
>>>>> 0
>>>>> 90
>>>>> 0
>>>>> 3%7CUnknown%7CTWFpbGZsb3d8eyJFbXB0eU1hcGkiOnRydWUsIlYiOiIwLjAuMDAw
>>>>> M
>>>>> CI
>>>>> s
>>>>> IlAiOiJXaW4zMiIsIkFOIjoiTWFpbCIsIldUIjoyfQ%3D%3D%7C0%7C%7C%7C&sdat
>>>>> a
>>>>> =m
>>>>> P
>>>>> UXMdb%2FYOyGIm0rnncaWgcqtwkUIh8364IP3eYDPe4%3D&reserved=0
>>>>> https://auto/
>>>>> builder.yoctoproject.org%2Fvalkyrie%2F%23%2Fbuilders%2F30%2Fbuilds
>>>>> %
>>>>> 2F
>>>>> 2
>>>>> 199&data=05%7C02%7Ctony.rex%40ericsson.com%7Cde1ae2aceeda470b38160
>>>>> 8
>>>>> dd
>>>>> d
>>>>> efdd730%7C92e84cebfbfd47abbe52080c6b87953f%7C0%7C0%7C6389119040052
>>>>> 2
>>>>> 74
>>>>> 3
>>>>> 2%7CUnknown%7CTWFpbGZsb3d8eyJFbXB0eU1hcGkiOnRydWUsIlYiOiIwLjAuMDAw
>>>>> M
>>>>> CI
>>>>> s
>>>>> IlAiOiJXaW4zMiIsIkFOIjoiTWFpbCIsIldUIjoyfQ%3D%3D%7C0%7C%7C%7C&sdat
>>>>> a
>>>>> =R
>>>>> 0
>>>>> HFameGl2LmIVMT9ml%2BDMnj%2FMbG6SW0%2F3yNZHQu2js%3D&reserved=0
>>>>> https://auto/
>>>>> builder.yoctoproject.org%2Fvalkyrie%2F%23%2Fbuilders%2F16%2Fbuilds
>>>>> %
>>>>> 2F
>>>>> 2
>>>>> 247&data=05%7C02%7Ctony.rex%40ericsson.com%7Cde1ae2aceeda470b38160
>>>>> 8
>>>>> dd
>>>>> d
>>>>> efdd730%7C92e84cebfbfd47abbe52080c6b87953f%7C0%7C0%7C6389119040052
>>>>> 4
>>>>> 53
>>>>> 8
>>>>> 7%7CUnknown%7CTWFpbGZsb3d8eyJFbXB0eU1hcGkiOnRydWUsIlYiOiIwLjAuMDAw
>>>>> M
>>>>> CI
>>>>> s
>>>>> IlAiOiJXaW4zMiIsIkFOIjoiTWFpbCIsIldUIjoyfQ%3D%3D%7C0%7C%7C%7C&sdat
>>>>> a
>>>>> =N
>>>>> S
>>>>> FhEupwMkgqUAhpRgFbKnyFl%2FsY%2B4UJMhVQHhqOaNM%3D&reserved=0
>>>>>
>>>>> Can you have a look at this error please?
>>>>>
>>>>> Best regards,
>>>>> Mathieu
>>>>>
>>>>> --
>>>>> Mathieu Dubois-Briand, Bootlin
>>>>> Embedded Linux and Kernel engineering https://boot/
>>>>> lin.com%2F&data=05%7C02%7Ctony.rex%40ericsson.com%7Cde1ae2aceeda47
>>>>> 0
>>>>> b3
>>>>> 8
>>>>> 1608dddefdd730%7C92e84cebfbfd47abbe52080c6b87953f%7C0%7C0%7C638911
>>>>> 9
>>>>> 04
>>>>> 0
>>>>> 05263327%7CUnknown%7CTWFpbGZsb3d8eyJFbXB0eU1hcGkiOnRydWUsIlYiOiIwL
>>>>> j
>>>>> Au
>>>>> M
>>>>> DAwMCIsIlAiOiJXaW4zMiIsIkFOIjoiTWFpbCIsIldUIjoyfQ%3D%3D%7C0%7C%7C%
>>>>> 7
>>>>> C&
>>>>> s
>>>>> data=oyi7YWsXDVHn7QB4fDdG1sBp7gQsfGWcJTWtX1CjAw4%3D&reserved=0
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>> 
>>

Re: [OE-core] [PATCH 0/1] base.bbclass: Fix the recipe parsing race issue

[Gyorgy Sarvari]

On 8/22/25 13:26, Tony Rex wrote:
> So to me it looks like the fetcher_hashes_dummyfunc makes it possible to catch any faulty or unwanted setups in the parsing stage.
>
> It seems hard to reproduce and expose this error on a common target, but I can reproduce it easily in the local setup.
> So I share my observations instead and hopefully you can make some sense out them.
>
> The error that I'm seeing is that when fetcher_hashes_dummyfunc[vardepvalue] = "${@bb.fetch.get_hashvalue(d)}" is executed the PV is not set, nor is the PN.
> So it seems that if the git ls-remote check happens to quick and early all variables are not set properly.
> The repo in this case is small (140K) and I believe that the access to the git server is fast (don't know of it matters) the git ls-remote check does not contain the
> right information if for example, the tag in the SRC_URI contains the PV value.
>
> When doing printouts from get_hashvalue the revs variable is empty and PN = defaultpkgname and PV = 1.0 has default values so the variables doesn't seem to be
> properly set yet and gives this error:
>
> ERROR: ExpansionError during parsing .../generic-if.bb################################   | ETA:  0:00:00
> bb.data_smart.ExpansionError: Failure expanding variable fetcher_hashes_dummyfunc[vardepvalue], expression was ${@bb.fetch.get_hashvalue(d)} which triggered exception FetchError: Fetcher failure: Unable to resolve '${TAGNAME}-1.0' in upstream git repository in git ls-remote output for gerrit...
> The variable dependency chain for the failure is:
> fetcher_hashes_dummyfunc[vardepvalue]
>
> ERROR: Parsing halted due to errors, see error messages above
>
> The patch that is provided fixes this issue, but there might be other and/or maybe better solutions to this matter.
> Another workaround for this problem that I have tested and that also seems to get around this issue is to set the SRC_URI with := or ?=
>
> This is why I consider it to be a race condition during parsing and the git ls-remote check.

Could you please share the _full_ error output, and the recipe that is
causing it? Feel free to change confidential things to placeholders, but
the more parts would be available, the better it would be.


>
> Tony
>
> -----Original Message-----
> From: Tony Rex
> Sent: Thursday, 21 August 2025 17:48
> To: Gyorgy Sarvari <skandigraun@gmail.com>; Alexander Kanavin <alex.kanavin@gmail.com>
> Cc: Mathieu Dubois-Briand <mathieu.dubois-briand@bootlin.com>; openembedded-core@lists.openembedded.org
> Subject: RE: [OE-core] [PATCH 0/1] base.bbclass: Fix the recipe parsing race issue
>
> I'm not sure how to run your (Gyorgy) code inside a recipe to test it, but in a bash script it works, of course.
>
> The thing that I'm looking at now is  a bit confusing, when commenting out the fetcher_hashes_dummyfunc function from the base.bbclass it seems to work properly.
>
> I don't know what the real purpose of this function is, am I missing something here, like a certain setup or a corner case?
> It seems to work both on whinlatter and scarthgap. Can someone please help out and clarify.
>
> git diff classes-global/base.bbclass
> diff --git a/meta/classes-global/base.bbclass b/meta/classes-global/base.bbclass
> index 6c08870571..83f82b2a4d 100644
> --- a/meta/classes-global/base.bbclass
> +++ b/meta/classes-global/base.bbclass
> @@ -128,16 +128,16 @@ def setup_hosttools_dir(dest, toolsvar, d, fatal=True):
>
>  # We can't use vardepvalue against do_fetch directly since that would overwrite  # the other task dependencies so we use an indirect function.
> -python fetcher_hashes_dummyfunc() {
> -    return
> -}
> -fetcher_hashes_dummyfunc[vardepvalue] = "${@bb.fetch.get_hashvalue(d)}"
> +#python fetcher_hashes_dummyfunc() {
> +#    return
> +#}
> +#fetcher_hashes_dummyfunc[vardepvalue] = "${@bb.fetch.get_hashvalue(d)}"
>
>  addtask fetch
>  do_fetch[dirs] = "${DL_DIR}"
>  do_fetch[file-checksums] = "${@bb.fetch.get_checksum_file_list(d)}"
>  do_fetch[file-checksums] += " ${@get_lic_checksum_file_list(d)}"
> -do_fetch[prefuncs] += "fetcher_hashes_dummyfunc"
> +#do_fetch[prefuncs] += "fetcher_hashes_dummyfunc"
>  do_fetch[network] = "1"
>  python base_do_fetch() {
>
> Tony
>
> -----Original Message-----
> From: Gyorgy Sarvari <skandigraun@gmail.com>
> Sent: Thursday, 21 August 2025 09:30
> To: Tony Rex <tony.rex@ericsson.com>; Alexander Kanavin <alex.kanavin@gmail.com>
> Cc: Mathieu Dubois-Briand <mathieu.dubois-briand@bootlin.com>; openembedded-core@lists.openembedded.org
> Subject: Re: [OE-core] [PATCH 0/1] base.bbclass: Fix the recipe parsing race issue
>
> On 8/21/25 09:10, Tony Rex wrote:
>> Yes, I get the same or similar error if the patch is not applied.
>>
>> ERROR: ExpansionError during parsing .../generic-if.bb################################   | ETA:  0:00:00
>> bb.data_smart.ExpansionError: Failure expanding variable fetcher_hashes_dummyfunc[vardepvalue], expression was ${@bb.fetch.get_hashvalue(d)} which triggered exception FetchError: Fetcher failure: Unable to resolve '${TAGNAME}-1.0' in upstream git repository in git ls-remote output for gerrit...
>> The variable dependency chain for the failure is:
>> fetcher_hashes_dummyfunc[vardepvalue]
>>
>> ERROR: Parsing halted due to errors, see error messages above
>>
>> How does your mytest_get.bb look like? Since I have had problem with reproducing the error on in a common setup maybe it can be done with your bb file.
>>
>> Tony
> The failure I reproduced is not recipe dependent. As some diagnostic test, try this script a few times with your failing git repo (change the repo placeholder):
>
> for i in `seq 50`; do
>   git clone $GIT_REPO_URL target_folder
>   git -C ./target_folder ls-remote
>   if [ $? -ne 0 ]; then
>     echo ls-remote failed, iteration: $i
>     break
>   fi
>   rm -rf ./target_folder
> done
>
> If I haven't made a typo, the script should clone your repo, do an ls-remote inside it, delete the folder, and do the same again.
>
> If your problem is the same that I reproduced, then I would expect the above script to fail sometimes - if it does, then the problem is with the infrastructure (network, remote git server, etc... something is flaky).
>
> Just for the record: the script generates a small amount of successive requests which some network admins might consider flooding.
>
>
>
>> -----Original Message-----
>> From: Gyorgy Sarvari <skandigraun@gmail.com>
>> Sent: Wednesday, 20 August 2025 18:12
>> To: Tony Rex <tony.rex@ericsson.com>; Alexander Kanavin
>> <alex.kanavin@gmail.com>
>> Cc: Mathieu Dubois-Briand <mathieu.dubois-briand@bootlin.com>;
>> openembedded-core@lists.openembedded.org
>> Subject: Re: [OE-core] [PATCH 0/1] base.bbclass: Fix the recipe
>> parsing race issue
>>
>> [You don't often get email from skandigraun@gmail.com. Learn why this
>> is important at https://aka.ms/LearnAboutSenderIdentification ]
>>
>> On 8/20/25 17:44, Gyorgy Sarvari via lists.openembedded.org wrote:
>>> On 8/20/25 13:53, Tony Rex wrote:
>>>> It was introduced by this commit:
>>>> https://git/
>>>> .yoctoproject.org%2Fpoky%2Fcommit%2F%3Fid%3Debfa1700f41b3411aec04014
>>>> 4
>>>> 605166c35b8dd14&data=05%7C02%7Ctony.rex%40ericsson.com%7C7b5c099d170
>>>> 0
>>>> 43080cca08dde004478d%7C92e84cebfbfd47abbe52080c6b87953f%7C0%7C0%7C63
>>>> 8
>>>> 913031194171335%7CUnknown%7CTWFpbGZsb3d8eyJFbXB0eU1hcGkiOnRydWUsIlYi
>>>> O
>>>> iIwLjAuMDAwMCIsIlAiOiJXaW4zMiIsIkFOIjoiTWFpbCIsIldUIjoyfQ%3D%3D%7C0%
>>>> 7
>>>> C%7C%7C&sdata=iYvLGK1DwJk78tEvmL%2FPOL2olj7iNL87eXqdsEraxLw%3D&reser
>>>> v
>>>> ed=0
>>>>
>>>> Maybe "not allowed" is a strong wording, but that was how I interpreted it. Sorry, my bad if I misinterpreted it.
>>> In my understanding (which could be wrong), this is more to avoid
>>> working on a repository, where the exact work-revision can't be
>>> determined. But in your example, it can be determined, if your patch
>>> is not applied.
>>>
>>> Some extra ramblings, which may or may not be unadulterated garbage
>>> from
>>> me: Your initial commit message mentioned an expansion error. I
>>> looked into that a bit, and is it possible that the repository that
>>> is causing this issue has some access rate limit? The only thing that
>>> can fail sometimes in that code path seems to be "git ls-remote".
>>>
>>> Now, when I noticed this, I compiled a git version for myself that
>>> randomly throws an error with "git ls-remote" command, and when I
>>> used it with bitbake, I got this:
>>>
>>> ERROR: ExpansionError during parsing
>>> /home/meee/stuff/ptest-images/poky/meta/recipes-core/ell/mytest_get.b
>>> b
>>> bb.data_smart.ExpansionError: Failure expanding variable
>>> fetcher_hashes_dummyfunc[vardepvalue], expression was <etc etc etc>
>>>
>>> Is this the same error you got?
>> Sorry for the spam, one thing I forgot - the error on my end also contains details about the command that failed, pointing to git. If it's not the same as you get, do you have a copy of the full bitbake output you get when you see this error?
>>
>>>> Tony
>>>>
>>>> -----Original Message-----
>>>> From: Gyorgy Sarvari <skandigraun@gmail.com>
>>>> Sent: Wednesday, 20 August 2025 12:18
>>>> To: Tony Rex <tony.rex@ericsson.com>; Alexander Kanavin
>>>> <alex.kanavin@gmail.com>
>>>> Cc: Mathieu Dubois-Briand <mathieu.dubois-briand@bootlin.com>;
>>>> openembedded-core@lists.openembedded.org
>>>> Subject: Re: [OE-core] [PATCH 0/1] base.bbclass: Fix the recipe
>>>> parsing race issue
>>>>
>>>> [You don't often get email from skandigraun@gmail.com. Learn why
>>>> this is important at https://aka.ms/LearnAboutSenderIdentification ]
>>>>
>>>> On 8/19/25 23:10, Tony Rex via lists.openembedded.org wrote:
>>>>> Hi Alex and Mathieu!
>>>>>
>>>>> Ok, so the patch fixes a bug and then the floating tag/branch function is in place again.
>>>>> The floating tag/branch function is broken due to the bug and the patch fixes it.
>>>>>
>>>>> The mail from Mathieu below might not fully reflect the issue here, since it seems to create a recipe with the help of devtool.
>>>>>
>>>>> To verify:
>>>>> 1. add layers/oe-core/meta-selftest to conf/bblayers.conf 2. run
>>>>> bitbake gitrepotest -c fetch
>>>>> result:
>>>>> NOTE: Tasks Summary: Attempted 2 tasks of which 0 didn't need to be rerun and all succeeded.
>>>>> 3. run bitbake gitrepotest -c cleanall 4. edit
>>>>> ../layers/oe-core/meta-selftest/recipes-test/gitrepotest/gitrepotes
>>>>> t
>>>>> .b
>>>>> b
>>>>> add tag=2.12 in SRC_URI
>>>>> SRC_URI =
>>>>> "git://git.yoctoproject.org/git/matchbox-panel-2;branch=master;tag=
>>>>> 2
>>>>> .1 2;protocol=https \ and comment out SRCREV #SRCREV =
>>>>> "f82ca3f42510fb3ef10f598b393eb373a2c34ca7"
>>>>> 5. run bitbake gitrepotest -c fetch
>>>>> result:
>>>>> NOTE: Tasks Summary: Attempted 2 tasks of which 0 didn't need to be rerun and all succeeded.
>>>>> The fetch goes through although this is not allowed according to the floating tag/branch error statement if not d.getVar("__BBSRCREV_SEEN"): in git.py.
>>>> Could you point to a commit/email/etc about this "no floating tags
>>>> allowed" statement? AFAIK tags without specifying revision is
>>>> allowed and supported. Not recommended, due to mandatory network
>>>> access, and because git tags are not immutable, but I'm surprised to
>>>> hear that they are not supposed to work (which of course can also
>>>> mean that I just missed a memo)
>>>>
>>>>> 6. run bitbake gitrepotest -c cleanall 7. apply patch
>>>>> 0001-base.bbclass-Fix-the-recipe-parsing-race-issue.patch
>>>>> 8. run bitbake gitrepotest -c fetch
>>>>> result:
>>>>> ERROR: gitrepotest-1.0-r0 do_fetch: Bitbake Fetcher Error:
>>>>> FetchError("Recipe uses a floating tag/branch '2.12' for repo
>>>>> 'git.yoctoproject.org/git/matchbox-panel-2' without a fixed SRCREV
>>>>> yet doesn't call bb.fetch2.get_srcrev() (use SRCPV in PV for OE).",
>>>>> None)
>>>>> ERROR: Logfile of failure stored in:
>>>>> .../build/tmp-glibc/work/cortexa53-wrs-linux/gitrepotest/1.0/temp/log.
>>>>> do_fetch.1254999
>>>>> ERROR: Task (.../layers/oe-core/meta-selftest/recipes-test/gitrepotest/gitrepotest.bb:do_fetch) failed with exit code '1'
>>>>> NOTE: Tasks Summary: Attempted 2 tasks of which 0 didn't need to be rerun and 1 failed.
>>>>>
>>>>> Error is the expected result when only having a tag set.
>>>>>
>>>>> Tony
>>>>>
>>>>> -----Original Message-----
>>>>> From: Alexander Kanavin <alex.kanavin@gmail.com>
>>>>> Sent: Tuesday, 19 August 2025 10:53
>>>>> To: Tony Rex <tony.rex@ericsson.com>
>>>>> Cc: Mathieu Dubois-Briand <mathieu.dubois-briand@bootlin.com>;
>>>>> openembedded-core@lists.openembedded.org
>>>>> Subject: Re: [OE-core] [PATCH 0/1] base.bbclass: Fix the recipe
>>>>> parsing race issue
>>>>>
>>>>> [You don't often get email from alex.kanavin@gmail.com. Learn why
>>>>> this is important at https://aka.ms/LearnAboutSenderIdentification
>>>>> ]
>>>>>
>>>>> You need to make things a little bit easier for maintainers and try to save their time.
>>>>>
>>>>> I would suggest that you provide clear steps to observe the issue.
>>>>> Attach the recipe file that would demonstrate the issue, and provide clear steps to trigger it. What is confusing right now is that you say here's a recipe that builds ok, but then you 'apply the patch' and then there's some kind of failure. So which patch needs to be applied?
>>>>>
>>>>> Alex
>>>>>
>>>>> On Fri, 15 Aug 2025 at 17:29, Tony Rex via lists.openembedded.org <tony.rex=ericsson.com@lists.openembedded.org> wrote:
>>>>>> Hi Mathieu!
>>>>>>
>>>>>> I think that this is how it is intended to work, bitbake does not allow floating tag/branch anymore.
>>>>>> Please correct me if I'm wrong @Richard Purdie (who implemented
>>>>>> this for security reasons I think.)
>>>>>>
>>>>>> The patch that I have makes sure that the parsing and setting of
>>>>>> the variables in the recipe are done before executing the do_fetch command, and as a side effect the floating tag/branch protection is working as intended again.
>>>>>>
>>>>>> How does the recipe look like?
>>>>>> devtool add kernel-module-hello-world https://git/.
>>>>>> yoctoproject.org%2Fgit%2Fkernel-module-hello-world&data=05%7C02%7C
>>>>>> t
>>>>>> on
>>>>>> y
>>>>>> .rex%40ericsson.com%7Cde1ae2aceeda470b381608dddefdd730%7C92e84cebf
>>>>>> b
>>>>>> fd
>>>>>> 4
>>>>>> 7abbe52080c6b87953f%7C0%7C0%7C638911904005151487%7CUnknown%7CTWFpb
>>>>>> G
>>>>>> Zs
>>>>>> b
>>>>>> 3d8eyJFbXB0eU1hcGkiOnRydWUsIlYiOiIwLjAuMDAwMCIsIlAiOiJXaW4zMiIsIkF
>>>>>> O
>>>>>> Ij
>>>>>> o
>>>>>> iTWFpbCIsIldUIjoyfQ%3D%3D%7C0%7C%7C%7C&sdata=jyCicRQT%2FLtb1YxiFHK
>>>>>> g
>>>>>> w5
>>>>>> B
>>>>>> nNktN4t9IciiyxtsEwO4%3D&reserved=0
>>>>>> The one I get looks like this:
>>>>>>
>>>>>> cat
>>>>>> build/workspace/recipes/kernel-module-hello-world/kernel-module-he
>>>>>> l
>>>>>> lo
>>>>>> -
>>>>>> world_git.bb
>>>>>> # Recipe created by recipetool
>>>>>> # This is the basis of a recipe and may need further editing in order to be fully functional.
>>>>>> # (Feel free to remove these comments when editing.)
>>>>>>
>>>>>> # WARNING: the following LICENSE and LIC_FILES_CHKSUM values are
>>>>>> best guesses - it is # your responsibility to verify that the values are complete and correct.
>>>>>> #
>>>>>> # The following license files were not able to be identified and
>>>>>> are # represented as "Unknown" below, you will need to check them yourself:
>>>>>> #   LICENSE
>>>>>> LICENSE = "Unknown"
>>>>>> LIC_FILES_CHKSUM = "file://LICENSE;md5=7b785be6433a2e4af6a641cc7f0330d1"
>>>>>>
>>>>>> SRC_URI = "git://git.yoctoproject.org/git/kernel-module-hello-world;protocol=https;branch=master"
>>>>>>
>>>>>> # Modify these as desired
>>>>>> PV = "1.0+git"
>>>>>> SRCREV = "b66f0f2bd559a1df81fc1bda240eba07a2ecdd2b"
>>>>>>
>>>>>> S = "${WORKDIR}/git"
>>>>>>
>>>>>> inherit module
>>>>>>
>>>>>> EXTRA_OEMAKE:append:task-install = " -C ${STAGING_KERNEL_DIR} M=${S}"
>>>>>> EXTRA_OEMAKE += "KERNEL_DIR=${STAGING_KERNEL_DIR}"
>>>>>>
>>>>>> And it contains SRCREV = "b66f0f2bd559a1df81fc1bda240eba07a2ecdd2b"
>>>>>>
>>>>>> Tony
>>>>>>
>>>>>> -----Original Message-----
>>>>>> From: Mathieu Dubois-Briand <mathieu.dubois-briand@bootlin.com>
>>>>>> Sent: Friday, 15 August 2025 11:26
>>>>>> To: Tony Rex <tony.rex@ericsson.com>;
>>>>>> openembedded-core@lists.openembedded.org
>>>>>> Subject: Re: [OE-core] [PATCH 0/1] base.bbclass: Fix the recipe
>>>>>> parsing race issue
>>>>>>
>>>>>> [You don't often get email from mathieu.dubois-briand@bootlin.com.
>>>>>> Learn why this is important at
>>>>>> https://aka.ms/LearnAboutSenderIdentification ]
>>>>>>
>>>>>> On Thu Aug 14, 2025 at 7:23 PM CEST, Tony Rex via lists.openembedded.org wrote:
>>>>>>> It seems that the race condition happens when doing do_fetch on
>>>>>>> really small repos and it can be difficult to reproduce so below
>>>>>>> the other case with floating tag/branch that is easier to verify.
>>>>>>>
>>>>>>> To verify the floating tag/branch issue have s simple recipe
>>>>>>> hello_1.0.bb and set the SRC_URI to SRC_URI = "git://<repository>;protocol=https;user=${USER};branch=master;tag=v1.0"
>>>>>>>
>>>>>>> run command bitbake hello everything builds.
>>>>>>>
>>>>>>> Apply the patch
>>>>>>>
>>>>>>> run command bitbake hello
>>>>>>> ERROR: hello-1.0-r1 do_fetch: Bitbake Fetcher Error:
>>>>>>> FetchError("Recipe uses a floating tag/branch 'v1.0' for repo
>>>>>>> '<repository>' without a fixed SRCREV yet doesn't call
>>>>>>> bb.fetch2.get_srcrev() (use SRCPV in PV for OE).", None)
>>>>>>> ERROR: Logfile of failure stored in:
>>>>>>> hello/1.0/temp/log.do_fetch.3289932
>>>>>>> ERROR: Task (hello/hello_1.0.bb:do_fetch) failed with exit code '1'
>>>>>>>
>>>>>>> Tony Rex (1):
>>>>>>>   base.bbclass: Fix the recipe parsing race issue
>>>>>>>
>>>>>>>  meta/classes-global/base.bbclass | 2 +-
>>>>>>>  1 file changed, 1 insertion(+), 1 deletion(-)
>>>>>> Hi Tony,
>>>>>>
>>>>>> Thanks for your patch.
>>>>>>
>>>>>> It looks like this is breaking some builds with the following error:
>>>>>>
>>>>>> oeqa.utils.subprocesstweak.OETestCalledProcessError: Command '. /srv/pokybuild/yocto-worker/qemux86-64/build/build/tmp/work/qemux86_64-poky-linux/core-image-sato/1.0/testsdkext/environment-setup-core2-64-poky-linux > /dev/null; devtool add kernel-module-hello-world https://git.yoctoproject.org/git/kernel-module-hello-world;' returned non-zero exit status 1.
>>>>>> ...
>>>>>> ERROR: Bitbake Fetcher Error: FetchError("Recipe uses a floating
>>>>>> tag/branch 'master' for repo
>>>>>> 'git.yoctoproject.org/git/kernel-module-hello-world' without a
>>>>>> fixed SRCREV yet doesn't call bb.fetch2.get_srcrev() (use SRCPV in
>>>>>> PV for OE).", None)
>>>>>>
>>>>>> https://auto/
>>>>>> builder.yoctoproject.org%2Fvalkyrie%2F%23%2Fbuilders%2F68%2Fbuilds
>>>>>> %
>>>>>> 2F
>>>>>> 2
>>>>>> 284&data=05%7C02%7Ctony.rex%40ericsson.com%7Cde1ae2aceeda470b38160
>>>>>> 8
>>>>>> dd
>>>>>> d
>>>>>> efdd730%7C92e84cebfbfd47abbe52080c6b87953f%7C0%7C0%7C6389119040052
>>>>>> 0
>>>>>> 90
>>>>>> 0
>>>>>> 3%7CUnknown%7CTWFpbGZsb3d8eyJFbXB0eU1hcGkiOnRydWUsIlYiOiIwLjAuMDAw
>>>>>> M
>>>>>> CI
>>>>>> s
>>>>>> IlAiOiJXaW4zMiIsIkFOIjoiTWFpbCIsIldUIjoyfQ%3D%3D%7C0%7C%7C%7C&sdat
>>>>>> a
>>>>>> =m
>>>>>> P
>>>>>> UXMdb%2FYOyGIm0rnncaWgcqtwkUIh8364IP3eYDPe4%3D&reserved=0
>>>>>> https://auto/
>>>>>> builder.yoctoproject.org%2Fvalkyrie%2F%23%2Fbuilders%2F30%2Fbuilds
>>>>>> %
>>>>>> 2F
>>>>>> 2
>>>>>> 199&data=05%7C02%7Ctony.rex%40ericsson.com%7Cde1ae2aceeda470b38160
>>>>>> 8
>>>>>> dd
>>>>>> d
>>>>>> efdd730%7C92e84cebfbfd47abbe52080c6b87953f%7C0%7C0%7C6389119040052
>>>>>> 2
>>>>>> 74
>>>>>> 3
>>>>>> 2%7CUnknown%7CTWFpbGZsb3d8eyJFbXB0eU1hcGkiOnRydWUsIlYiOiIwLjAuMDAw
>>>>>> M
>>>>>> CI
>>>>>> s
>>>>>> IlAiOiJXaW4zMiIsIkFOIjoiTWFpbCIsIldUIjoyfQ%3D%3D%7C0%7C%7C%7C&sdat
>>>>>> a
>>>>>> =R
>>>>>> 0
>>>>>> HFameGl2LmIVMT9ml%2BDMnj%2FMbG6SW0%2F3yNZHQu2js%3D&reserved=0
>>>>>> https://auto/
>>>>>> builder.yoctoproject.org%2Fvalkyrie%2F%23%2Fbuilders%2F16%2Fbuilds
>>>>>> %
>>>>>> 2F
>>>>>> 2
>>>>>> 247&data=05%7C02%7Ctony.rex%40ericsson.com%7Cde1ae2aceeda470b38160
>>>>>> 8
>>>>>> dd
>>>>>> d
>>>>>> efdd730%7C92e84cebfbfd47abbe52080c6b87953f%7C0%7C0%7C6389119040052
>>>>>> 4
>>>>>> 53
>>>>>> 8
>>>>>> 7%7CUnknown%7CTWFpbGZsb3d8eyJFbXB0eU1hcGkiOnRydWUsIlYiOiIwLjAuMDAw
>>>>>> M
>>>>>> CI
>>>>>> s
>>>>>> IlAiOiJXaW4zMiIsIkFOIjoiTWFpbCIsIldUIjoyfQ%3D%3D%7C0%7C%7C%7C&sdat
>>>>>> a
>>>>>> =N
>>>>>> S
>>>>>> FhEupwMkgqUAhpRgFbKnyFl%2FsY%2B4UJMhVQHhqOaNM%3D&reserved=0
>>>>>>
>>>>>> Can you have a look at this error please?
>>>>>>
>>>>>> Best regards,
>>>>>> Mathieu
>>>>>>
>>>>>> --
>>>>>> Mathieu Dubois-Briand, Bootlin
>>>>>> Embedded Linux and Kernel engineering https://boot/
>>>>>> lin.com%2F&data=05%7C02%7Ctony.rex%40ericsson.com%7Cde1ae2aceeda47
>>>>>> 0
>>>>>> b3
>>>>>> 8
>>>>>> 1608dddefdd730%7C92e84cebfbfd47abbe52080c6b87953f%7C0%7C0%7C638911
>>>>>> 9
>>>>>> 04
>>>>>> 0
>>>>>> 05263327%7CUnknown%7CTWFpbGZsb3d8eyJFbXB0eU1hcGkiOnRydWUsIlYiOiIwL
>>>>>> j
>>>>>> Au
>>>>>> M
>>>>>> DAwMCIsIlAiOiJXaW4zMiIsIkFOIjoiTWFpbCIsIldUIjoyfQ%3D%3D%7C0%7C%7C%
>>>>>> 7
>>>>>> C&
>>>>>> s
>>>>>> data=oyi7YWsXDVHn7QB4fDdG1sBp7gQsfGWcJTWtX1CjAw4%3D&reserved=0
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>> -=-=-=-=-=-=-=-=-=-=-=-
>>> Links: You receive all messages sent to this group.
>>> View/Reply Online (#222190):
>>> https://list/
>>> s.openembedded.org%2Fg%2Fopenembedded-core%2Fmessage%2F222190&data=05
>>> %
>>> 7C02%7Ctony.rex%40ericsson.com%7C7b5c099d170043080cca08dde004478d%7C9
>>> 2
>>> e84cebfbfd47abbe52080c6b87953f%7C0%7C0%7C638913031194226137%7CUnknown
>>> %
>>> 7CTWFpbGZsb3d8eyJFbXB0eU1hcGkiOnRydWUsIlYiOiIwLjAuMDAwMCIsIlAiOiJXaW4
>>> z
>>> MiIsIkFOIjoiTWFpbCIsIldUIjoyfQ%3D%3D%7C0%7C%7C%7C&sdata=WbB87ExjlYOSm
>>> q
>>> I0%2BEee6LbpupPfWxzk0vNYlbCUP5g%3D&reserved=0
>>> Mute This Topic:
>>> https://list/
>>> s.openembedded.org%2Fmt%2F114706197%2F6084445&data=05%7C02%7Ctony.rex
>>> %
>>> 40ericsson.com%7C7b5c099d170043080cca08dde004478d%7C92e84cebfbfd47abb
>>> e
>>> 52080c6b87953f%7C0%7C0%7C638913031194241729%7CUnknown%7CTWFpbGZsb3d8e
>>> y
>>> JFbXB0eU1hcGkiOnRydWUsIlYiOiIwLjAuMDAwMCIsIlAiOiJXaW4zMiIsIkFOIjoiTWF
>>> p
>>> bCIsIldUIjoyfQ%3D%3D%7C0%7C%7C%7C&sdata=bHfsli%2Fba3g8lov%2Bxmfovaal5
>>> C
>>> uBGuFnZVXs8AjHGIY%3D&reserved=0 Group Owner:
>>> openembedded-core+owner@lists.openembedded.org
>>> Unsubscribe:
>>> https://list/
>>> s.openembedded.org%2Fg%2Fopenembedded-core%2Funsub&data=05%7C02%7Cton
>>> y
>>> .rex%40ericsson.com%7C7b5c099d170043080cca08dde004478d%7C92e84cebfbfd
>>> 4
>>> 7abbe52080c6b87953f%7C0%7C0%7C638913031194543958%7CUnknown%7CTWFpbGZs
>>> b
>>> 3d8eyJFbXB0eU1hcGkiOnRydWUsIlYiOiIwLjAuMDAwMCIsIlAiOiJXaW4zMiIsIkFOIj
>>> o
>>> iTWFpbCIsIldUIjoyfQ%3D%3D%7C0%7C%7C%7C&sdata=gy4aai6oflVendGBp8nbYAL8
>>> K
>>> CGj77ObOMy2rYDnGN0%3D&reserved=0 [skandigraun@gmail.com]
>>> -=-=-=-=-=-=-=-=-=-=-=-
>>>

Re: [OE-core] [PATCH 0/1] base.bbclass: Fix the recipe parsing race issue

[Tony Rex]

It is the full error output that is in the mail thread, there is nothing more I think.

But can you please try to reproduce the error by doing the changes in meta-selftest layer described below and evaluate the outcome.
I think the key is the +BBCLASSEXTEND="nativesdk" line.

git diff
diff --git a/meta-selftest/conf/layer.conf b/meta-selftest/conf/layer.conf
index 48ca8464f9..3d83072fb0 100644
--- a/meta-selftest/conf/layer.conf
+++ b/meta-selftest/conf/layer.conf
@@ -12,3 +12,6 @@ BBFILE_PRIORITY_selftest = "5"
 addpylib ${LAYERDIR}/lib oeqa

 LAYERSERIES_COMPAT_selftest = "scarthgap"
+
+TAGNAME:pn-gitrepotest="2"
+PV:pn-gitrepotest="12"
diff --git a/meta-selftest/recipes-test/gitrepotest/gitrepotest.bb b/meta-selftest/recipes-test/gitrepotest/gitrepotest.bb
index f1b6c55833..10943584df 100644
--- a/meta-selftest/recipes-test/gitrepotest/gitrepotest.bb
+++ b/meta-selftest/recipes-test/gitrepotest/gitrepotest.bb
@@ -7,10 +7,12 @@ INHIBIT_DEFAULT_DEPS = "1"

 PATCHTOOL="git"

-SRC_URI = "git://git.yoctoproject.org/git/matchbox-panel-2;branch=master;protocol=https \
+TAG="${TAGNAME}.${PV}"
+SRC_URI = "git://git.yoctoproject.org/git/matchbox-panel-2;branch=master;protocol=https;tag=${TAG} \
            file://0001-testpatch.patch \
           "

-SRCREV = "f82ca3f42510fb3ef10f598b393eb373a2c34ca7"
+#SRCREV = "f82ca3f42510fb3ef10f598b393eb373a2c34ca7"

 S = "${WORKDIR}/git"
+BBCLASSEXTEND="nativesdk"


You should get something like this:
ERROR: ExpansionError during parsing .../oe-core/meta-selftest/recipes-test/gitrepotest/gitrepotest.bb   | ETA:  0:00:35
bb.data_smart.ExpansionError: Failure expanding variable fetcher_hashes_dummyfunc[vardepvalue], expression was ${@bb.fetch.get_hashvalue(d)} which triggered exception FetchError: Fetcher failure: Unable to resolve '${TAGNAME}.1.0' in upstream git repository in git ls-remote output for git.yoctoproject.org/git/matchbox-panel-2
The variable dependency chain for the failure is: fetcher_hashes_dummyfunc[vardepvalue]

ERROR: Parsing halted due to errors, see error messages above


Tony

-----Original Message-----
From: Gyorgy Sarvari <skandigraun@gmail.com>
Sent: Friday, 22 August 2025 16:10
To: Tony Rex <tony.rex@ericsson.com>; Alexander Kanavin <alex.kanavin@gmail.com>
Cc: Mathieu Dubois-Briand <mathieu.dubois-briand@bootlin.com>; openembedded-core@lists.openembedded.org
Subject: Re: [OE-core] [PATCH 0/1] base.bbclass: Fix the recipe parsing race issue

On 8/22/25 13:26, Tony Rex wrote:
> So to me it looks like the fetcher_hashes_dummyfunc makes it possible to catch any faulty or unwanted setups in the parsing stage.
>
> It seems hard to reproduce and expose this error on a common target, but I can reproduce it easily in the local setup.
> So I share my observations instead and hopefully you can make some sense out them.
>
> The error that I'm seeing is that when fetcher_hashes_dummyfunc[vardepvalue] = "${@bb.fetch.get_hashvalue(d)}" is executed the PV is not set, nor is the PN.
> So it seems that if the git ls-remote check happens to quick and early all variables are not set properly.
> The repo in this case is small (140K) and I believe that the access to
> the git server is fast (don't know of it matters) the git ls-remote check does not contain the right information if for example, the tag in the SRC_URI contains the PV value.
>
> When doing printouts from get_hashvalue the revs variable is empty and
> PN = defaultpkgname and PV = 1.0 has default values so the variables doesn't seem to be properly set yet and gives this error:
>
> ERROR: ExpansionError during parsing .../generic-if.bb################################   | ETA:  0:00:00
> bb.data_smart.ExpansionError: Failure expanding variable fetcher_hashes_dummyfunc[vardepvalue], expression was ${@bb.fetch.get_hashvalue(d)} which triggered exception FetchError: Fetcher failure: Unable to resolve '${TAGNAME}-1.0' in upstream git repository in git ls-remote output for gerrit...
> The variable dependency chain for the failure is:
> fetcher_hashes_dummyfunc[vardepvalue]
>
> ERROR: Parsing halted due to errors, see error messages above
>
> The patch that is provided fixes this issue, but there might be other and/or maybe better solutions to this matter.
> Another workaround for this problem that I have tested and that also
> seems to get around this issue is to set the SRC_URI with := or ?=
>
> This is why I consider it to be a race condition during parsing and the git ls-remote check.

Could you please share the _full_ error output, and the recipe that is causing it? Feel free to change confidential things to placeholders, but the more parts would be available, the better it would be.


>
> Tony
>
> -----Original Message-----
> From: Tony Rex
> Sent: Thursday, 21 August 2025 17:48
> To: Gyorgy Sarvari <skandigraun@gmail.com>; Alexander Kanavin
> <alex.kanavin@gmail.com>
> Cc: Mathieu Dubois-Briand <mathieu.dubois-briand@bootlin.com>;
> openembedded-core@lists.openembedded.org
> Subject: RE: [OE-core] [PATCH 0/1] base.bbclass: Fix the recipe
> parsing race issue
>
> I'm not sure how to run your (Gyorgy) code inside a recipe to test it, but in a bash script it works, of course.
>
> The thing that I'm looking at now is  a bit confusing, when commenting out the fetcher_hashes_dummyfunc function from the base.bbclass it seems to work properly.
>
> I don't know what the real purpose of this function is, am I missing something here, like a certain setup or a corner case?
> It seems to work both on whinlatter and scarthgap. Can someone please help out and clarify.
>
> git diff classes-global/base.bbclass
> diff --git a/meta/classes-global/base.bbclass
> b/meta/classes-global/base.bbclass
> index 6c08870571..83f82b2a4d 100644
> --- a/meta/classes-global/base.bbclass
> +++ b/meta/classes-global/base.bbclass
> @@ -128,16 +128,16 @@ def setup_hosttools_dir(dest, toolsvar, d, fatal=True):
>
>  # We can't use vardepvalue against do_fetch directly since that would overwrite  # the other task dependencies so we use an indirect function.
> -python fetcher_hashes_dummyfunc() {
> -    return
> -}
> -fetcher_hashes_dummyfunc[vardepvalue] = "${@bb.fetch.get_hashvalue(d)}"
> +#python fetcher_hashes_dummyfunc() {
> +#    return
> +#}
> +#fetcher_hashes_dummyfunc[vardepvalue] = "${@bb.fetch.get_hashvalue(d)}"
>
>  addtask fetch
>  do_fetch[dirs] = "${DL_DIR}"
>  do_fetch[file-checksums] = "${@bb.fetch.get_checksum_file_list(d)}"
>  do_fetch[file-checksums] += " ${@get_lic_checksum_file_list(d)}"
> -do_fetch[prefuncs] += "fetcher_hashes_dummyfunc"
> +#do_fetch[prefuncs] += "fetcher_hashes_dummyfunc"
>  do_fetch[network] = "1"
>  python base_do_fetch() {
>
> Tony
>
> -----Original Message-----
> From: Gyorgy Sarvari <skandigraun@gmail.com>
> Sent: Thursday, 21 August 2025 09:30
> To: Tony Rex <tony.rex@ericsson.com>; Alexander Kanavin
> <alex.kanavin@gmail.com>
> Cc: Mathieu Dubois-Briand <mathieu.dubois-briand@bootlin.com>;
> openembedded-core@lists.openembedded.org
> Subject: Re: [OE-core] [PATCH 0/1] base.bbclass: Fix the recipe
> parsing race issue
>
> On 8/21/25 09:10, Tony Rex wrote:
>> Yes, I get the same or similar error if the patch is not applied.
>>
>> ERROR: ExpansionError during parsing .../generic-if.bb################################   | ETA:  0:00:00
>> bb.data_smart.ExpansionError: Failure expanding variable fetcher_hashes_dummyfunc[vardepvalue], expression was ${@bb.fetch.get_hashvalue(d)} which triggered exception FetchError: Fetcher failure: Unable to resolve '${TAGNAME}-1.0' in upstream git repository in git ls-remote output for gerrit...
>> The variable dependency chain for the failure is:
>> fetcher_hashes_dummyfunc[vardepvalue]
>>
>> ERROR: Parsing halted due to errors, see error messages above
>>
>> How does your mytest_get.bb look like? Since I have had problem with reproducing the error on in a common setup maybe it can be done with your bb file.
>>
>> Tony
> The failure I reproduced is not recipe dependent. As some diagnostic test, try this script a few times with your failing git repo (change the repo placeholder):
>
> for i in `seq 50`; do
>   git clone $GIT_REPO_URL target_folder
>   git -C ./target_folder ls-remote
>   if [ $? -ne 0 ]; then
>     echo ls-remote failed, iteration: $i
>     break
>   fi
>   rm -rf ./target_folder
> done
>
> If I haven't made a typo, the script should clone your repo, do an ls-remote inside it, delete the folder, and do the same again.
>
> If your problem is the same that I reproduced, then I would expect the above script to fail sometimes - if it does, then the problem is with the infrastructure (network, remote git server, etc... something is flaky).
>
> Just for the record: the script generates a small amount of successive requests which some network admins might consider flooding.
>
>
>
>> -----Original Message-----
>> From: Gyorgy Sarvari <skandigraun@gmail.com>
>> Sent: Wednesday, 20 August 2025 18:12
>> To: Tony Rex <tony.rex@ericsson.com>; Alexander Kanavin
>> <alex.kanavin@gmail.com>
>> Cc: Mathieu Dubois-Briand <mathieu.dubois-briand@bootlin.com>;
>> openembedded-core@lists.openembedded.org
>> Subject: Re: [OE-core] [PATCH 0/1] base.bbclass: Fix the recipe
>> parsing race issue
>>
>> [You don't often get email from skandigraun@gmail.com. Learn why this
>> is important at https://aka.ms/LearnAboutSenderIdentification ]
>>
>> On 8/20/25 17:44, Gyorgy Sarvari via lists.openembedded.org wrote:
>>> On 8/20/25 13:53, Tony Rex wrote:
>>>> It was introduced by this commit:
>>>> https://git/
>>>> .yoctoproject.org%2Fpoky%2Fcommit%2F%3Fid%3Debfa1700f41b3411aec0401
>>>> 4
>>>> 4
>>>> 605166c35b8dd14&data=05%7C02%7Ctony.rex%40ericsson.com%7C7b5c099d17
>>>> 0
>>>> 0
>>>> 43080cca08dde004478d%7C92e84cebfbfd47abbe52080c6b87953f%7C0%7C0%7C6
>>>> 3
>>>> 8
>>>> 913031194171335%7CUnknown%7CTWFpbGZsb3d8eyJFbXB0eU1hcGkiOnRydWUsIlY
>>>> i
>>>> O
>>>> iIwLjAuMDAwMCIsIlAiOiJXaW4zMiIsIkFOIjoiTWFpbCIsIldUIjoyfQ%3D%3D%7C0
>>>> %
>>>> 7
>>>> C%7C%7C&sdata=iYvLGK1DwJk78tEvmL%2FPOL2olj7iNL87eXqdsEraxLw%3D&rese
>>>> r
>>>> v
>>>> ed=0
>>>>
>>>> Maybe "not allowed" is a strong wording, but that was how I interpreted it. Sorry, my bad if I misinterpreted it.
>>> In my understanding (which could be wrong), this is more to avoid
>>> working on a repository, where the exact work-revision can't be
>>> determined. But in your example, it can be determined, if your patch
>>> is not applied.
>>>
>>> Some extra ramblings, which may or may not be unadulterated garbage
>>> from
>>> me: Your initial commit message mentioned an expansion error. I
>>> looked into that a bit, and is it possible that the repository that
>>> is causing this issue has some access rate limit? The only thing
>>> that can fail sometimes in that code path seems to be "git ls-remote".
>>>
>>> Now, when I noticed this, I compiled a git version for myself that
>>> randomly throws an error with "git ls-remote" command, and when I
>>> used it with bitbake, I got this:
>>>
>>> ERROR: ExpansionError during parsing
>>> /home/meee/stuff/ptest-images/poky/meta/recipes-core/ell/mytest_get.
>>> b
>>> b
>>> bb.data_smart.ExpansionError: Failure expanding variable
>>> fetcher_hashes_dummyfunc[vardepvalue], expression was <etc etc etc>
>>>
>>> Is this the same error you got?
>> Sorry for the spam, one thing I forgot - the error on my end also contains details about the command that failed, pointing to git. If it's not the same as you get, do you have a copy of the full bitbake output you get when you see this error?
>>
>>>> Tony
>>>>
>>>> -----Original Message-----
>>>> From: Gyorgy Sarvari <skandigraun@gmail.com>
>>>> Sent: Wednesday, 20 August 2025 12:18
>>>> To: Tony Rex <tony.rex@ericsson.com>; Alexander Kanavin
>>>> <alex.kanavin@gmail.com>
>>>> Cc: Mathieu Dubois-Briand <mathieu.dubois-briand@bootlin.com>;
>>>> openembedded-core@lists.openembedded.org
>>>> Subject: Re: [OE-core] [PATCH 0/1] base.bbclass: Fix the recipe
>>>> parsing race issue
>>>>
>>>> [You don't often get email from skandigraun@gmail.com. Learn why
>>>> this is important at https://aka.ms/LearnAboutSenderIdentification
>>>> ]
>>>>
>>>> On 8/19/25 23:10, Tony Rex via lists.openembedded.org wrote:
>>>>> Hi Alex and Mathieu!
>>>>>
>>>>> Ok, so the patch fixes a bug and then the floating tag/branch function is in place again.
>>>>> The floating tag/branch function is broken due to the bug and the patch fixes it.
>>>>>
>>>>> The mail from Mathieu below might not fully reflect the issue here, since it seems to create a recipe with the help of devtool.
>>>>>
>>>>> To verify:
>>>>> 1. add layers/oe-core/meta-selftest to conf/bblayers.conf 2. run
>>>>> bitbake gitrepotest -c fetch
>>>>> result:
>>>>> NOTE: Tasks Summary: Attempted 2 tasks of which 0 didn't need to be rerun and all succeeded.
>>>>> 3. run bitbake gitrepotest -c cleanall 4. edit
>>>>> ../layers/oe-core/meta-selftest/recipes-test/gitrepotest/gitrepote
>>>>> s
>>>>> t
>>>>> .b
>>>>> b
>>>>> add tag=2.12 in SRC_URI
>>>>> SRC_URI =
>>>>> "git://git.yoctoproject.org/git/matchbox-panel-2;branch=master;tag
>>>>> =
>>>>> 2
>>>>> .1 2;protocol=https \ and comment out SRCREV #SRCREV =
>>>>> "f82ca3f42510fb3ef10f598b393eb373a2c34ca7"
>>>>> 5. run bitbake gitrepotest -c fetch
>>>>> result:
>>>>> NOTE: Tasks Summary: Attempted 2 tasks of which 0 didn't need to be rerun and all succeeded.
>>>>> The fetch goes through although this is not allowed according to the floating tag/branch error statement if not d.getVar("__BBSRCREV_SEEN"): in git.py.
>>>> Could you point to a commit/email/etc about this "no floating tags
>>>> allowed" statement? AFAIK tags without specifying revision is
>>>> allowed and supported. Not recommended, due to mandatory network
>>>> access, and because git tags are not immutable, but I'm surprised
>>>> to hear that they are not supposed to work (which of course can
>>>> also mean that I just missed a memo)
>>>>
>>>>> 6. run bitbake gitrepotest -c cleanall 7. apply patch
>>>>> 0001-base.bbclass-Fix-the-recipe-parsing-race-issue.patch
>>>>> 8. run bitbake gitrepotest -c fetch
>>>>> result:
>>>>> ERROR: gitrepotest-1.0-r0 do_fetch: Bitbake Fetcher Error:
>>>>> FetchError("Recipe uses a floating tag/branch '2.12' for repo
>>>>> 'git.yoctoproject.org/git/matchbox-panel-2' without a fixed SRCREV
>>>>> yet doesn't call bb.fetch2.get_srcrev() (use SRCPV in PV for
>>>>> OE).",
>>>>> None)
>>>>> ERROR: Logfile of failure stored in:
>>>>> .../build/tmp-glibc/work/cortexa53-wrs-linux/gitrepotest/1.0/temp/log.
>>>>> do_fetch.1254999
>>>>> ERROR: Task (.../layers/oe-core/meta-selftest/recipes-test/gitrepotest/gitrepotest.bb:do_fetch) failed with exit code '1'
>>>>> NOTE: Tasks Summary: Attempted 2 tasks of which 0 didn't need to be rerun and 1 failed.
>>>>>
>>>>> Error is the expected result when only having a tag set.
>>>>>
>>>>> Tony
>>>>>
>>>>> -----Original Message-----
>>>>> From: Alexander Kanavin <alex.kanavin@gmail.com>
>>>>> Sent: Tuesday, 19 August 2025 10:53
>>>>> To: Tony Rex <tony.rex@ericsson.com>
>>>>> Cc: Mathieu Dubois-Briand <mathieu.dubois-briand@bootlin.com>;
>>>>> openembedded-core@lists.openembedded.org
>>>>> Subject: Re: [OE-core] [PATCH 0/1] base.bbclass: Fix the recipe
>>>>> parsing race issue
>>>>>
>>>>> [You don't often get email from alex.kanavin@gmail.com. Learn why
>>>>> this is important at https://aka.ms/LearnAboutSenderIdentification
>>>>> ]
>>>>>
>>>>> You need to make things a little bit easier for maintainers and try to save their time.
>>>>>
>>>>> I would suggest that you provide clear steps to observe the issue.
>>>>> Attach the recipe file that would demonstrate the issue, and provide clear steps to trigger it. What is confusing right now is that you say here's a recipe that builds ok, but then you 'apply the patch' and then there's some kind of failure. So which patch needs to be applied?
>>>>>
>>>>> Alex
>>>>>
>>>>> On Fri, 15 Aug 2025 at 17:29, Tony Rex via lists.openembedded.org <tony.rex=ericsson.com@lists.openembedded.org> wrote:
>>>>>> Hi Mathieu!
>>>>>>
>>>>>> I think that this is how it is intended to work, bitbake does not allow floating tag/branch anymore.
>>>>>> Please correct me if I'm wrong @Richard Purdie (who implemented
>>>>>> this for security reasons I think.)
>>>>>>
>>>>>> The patch that I have makes sure that the parsing and setting of
>>>>>> the variables in the recipe are done before executing the do_fetch command, and as a side effect the floating tag/branch protection is working as intended again.
>>>>>>
>>>>>> How does the recipe look like?
>>>>>> devtool add kernel-module-hello-world https://git/.
>>>>>> yoctoproject.org%2Fgit%2Fkernel-module-hello-world&data=05%7C02%7
>>>>>> C
>>>>>> t
>>>>>> on
>>>>>> y
>>>>>> .rex%40ericsson.com%7Cde1ae2aceeda470b381608dddefdd730%7C92e84ceb
>>>>>> f
>>>>>> b
>>>>>> fd
>>>>>> 4
>>>>>> 7abbe52080c6b87953f%7C0%7C0%7C638911904005151487%7CUnknown%7CTWFp
>>>>>> b
>>>>>> G
>>>>>> Zs
>>>>>> b
>>>>>> 3d8eyJFbXB0eU1hcGkiOnRydWUsIlYiOiIwLjAuMDAwMCIsIlAiOiJXaW4zMiIsIk
>>>>>> F
>>>>>> O
>>>>>> Ij
>>>>>> o
>>>>>> iTWFpbCIsIldUIjoyfQ%3D%3D%7C0%7C%7C%7C&sdata=jyCicRQT%2FLtb1YxiFH
>>>>>> K
>>>>>> g
>>>>>> w5
>>>>>> B
>>>>>> nNktN4t9IciiyxtsEwO4%3D&reserved=0
>>>>>> The one I get looks like this:
>>>>>>
>>>>>> cat
>>>>>> build/workspace/recipes/kernel-module-hello-world/kernel-module-h
>>>>>> e
>>>>>> l
>>>>>> lo
>>>>>> -
>>>>>> world_git.bb
>>>>>> # Recipe created by recipetool
>>>>>> # This is the basis of a recipe and may need further editing in order to be fully functional.
>>>>>> # (Feel free to remove these comments when editing.)
>>>>>>
>>>>>> # WARNING: the following LICENSE and LIC_FILES_CHKSUM values are
>>>>>> best guesses - it is # your responsibility to verify that the values are complete and correct.
>>>>>> #
>>>>>> # The following license files were not able to be identified and
>>>>>> are # represented as "Unknown" below, you will need to check them yourself:
>>>>>> #   LICENSE
>>>>>> LICENSE = "Unknown"
>>>>>> LIC_FILES_CHKSUM = "file://LICENSE;md5=7b785be6433a2e4af6a641cc7f0330d1"
>>>>>>
>>>>>> SRC_URI = "git://git.yoctoproject.org/git/kernel-module-hello-world;protocol=https;branch=master"
>>>>>>
>>>>>> # Modify these as desired
>>>>>> PV = "1.0+git"
>>>>>> SRCREV = "b66f0f2bd559a1df81fc1bda240eba07a2ecdd2b"
>>>>>>
>>>>>> S = "${WORKDIR}/git"
>>>>>>
>>>>>> inherit module
>>>>>>
>>>>>> EXTRA_OEMAKE:append:task-install = " -C ${STAGING_KERNEL_DIR} M=${S}"
>>>>>> EXTRA_OEMAKE += "KERNEL_DIR=${STAGING_KERNEL_DIR}"
>>>>>>
>>>>>> And it contains SRCREV = "b66f0f2bd559a1df81fc1bda240eba07a2ecdd2b"
>>>>>>
>>>>>> Tony
>>>>>>
>>>>>> -----Original Message-----
>>>>>> From: Mathieu Dubois-Briand <mathieu.dubois-briand@bootlin.com>
>>>>>> Sent: Friday, 15 August 2025 11:26
>>>>>> To: Tony Rex <tony.rex@ericsson.com>;
>>>>>> openembedded-core@lists.openembedded.org
>>>>>> Subject: Re: [OE-core] [PATCH 0/1] base.bbclass: Fix the recipe
>>>>>> parsing race issue
>>>>>>
>>>>>> [You don't often get email from mathieu.dubois-briand@bootlin.com.
>>>>>> Learn why this is important at
>>>>>> https://aka.ms/LearnAboutSenderIdentification ]
>>>>>>
>>>>>> On Thu Aug 14, 2025 at 7:23 PM CEST, Tony Rex via lists.openembedded.org wrote:
>>>>>>> It seems that the race condition happens when doing do_fetch on
>>>>>>> really small repos and it can be difficult to reproduce so below
>>>>>>> the other case with floating tag/branch that is easier to verify.
>>>>>>>
>>>>>>> To verify the floating tag/branch issue have s simple recipe
>>>>>>> hello_1.0.bb and set the SRC_URI to SRC_URI = "git://<repository>;protocol=https;user=${USER};branch=master;tag=v1.0"
>>>>>>>
>>>>>>> run command bitbake hello everything builds.
>>>>>>>
>>>>>>> Apply the patch
>>>>>>>
>>>>>>> run command bitbake hello
>>>>>>> ERROR: hello-1.0-r1 do_fetch: Bitbake Fetcher Error:
>>>>>>> FetchError("Recipe uses a floating tag/branch 'v1.0' for repo
>>>>>>> '<repository>' without a fixed SRCREV yet doesn't call
>>>>>>> bb.fetch2.get_srcrev() (use SRCPV in PV for OE).", None)
>>>>>>> ERROR: Logfile of failure stored in:
>>>>>>> hello/1.0/temp/log.do_fetch.3289932
>>>>>>> ERROR: Task (hello/hello_1.0.bb:do_fetch) failed with exit code '1'
>>>>>>>
>>>>>>> Tony Rex (1):
>>>>>>>   base.bbclass: Fix the recipe parsing race issue
>>>>>>>
>>>>>>>  meta/classes-global/base.bbclass | 2 +-
>>>>>>>  1 file changed, 1 insertion(+), 1 deletion(-)
>>>>>> Hi Tony,
>>>>>>
>>>>>> Thanks for your patch.
>>>>>>
>>>>>> It looks like this is breaking some builds with the following error:
>>>>>>
>>>>>> oeqa.utils.subprocesstweak.OETestCalledProcessError: Command '. /srv/pokybuild/yocto-worker/qemux86-64/build/build/tmp/work/qemux86_64-poky-linux/core-image-sato/1.0/testsdkext/environment-setup-core2-64-poky-linux > /dev/null; devtool add kernel-module-hello-world https://git.yoctoproject.org/git/kernel-module-hello-world;' returned non-zero exit status 1.
>>>>>> ...
>>>>>> ERROR: Bitbake Fetcher Error: FetchError("Recipe uses a floating
>>>>>> tag/branch 'master' for repo
>>>>>> 'git.yoctoproject.org/git/kernel-module-hello-world' without a
>>>>>> fixed SRCREV yet doesn't call bb.fetch2.get_srcrev() (use SRCPV
>>>>>> in PV for OE).", None)
>>>>>>
>>>>>> https://auto/
>>>>>> builder.yoctoproject.org%2Fvalkyrie%2F%23%2Fbuilders%2F68%2Fbuild
>>>>>> s
>>>>>> %
>>>>>> 2F
>>>>>> 2
>>>>>> 284&data=05%7C02%7Ctony.rex%40ericsson.com%7Cde1ae2aceeda470b3816
>>>>>> 0
>>>>>> 8
>>>>>> dd
>>>>>> d
>>>>>> efdd730%7C92e84cebfbfd47abbe52080c6b87953f%7C0%7C0%7C638911904005
>>>>>> 2
>>>>>> 0
>>>>>> 90
>>>>>> 0
>>>>>> 3%7CUnknown%7CTWFpbGZsb3d8eyJFbXB0eU1hcGkiOnRydWUsIlYiOiIwLjAuMDA
>>>>>> w
>>>>>> M
>>>>>> CI
>>>>>> s
>>>>>> IlAiOiJXaW4zMiIsIkFOIjoiTWFpbCIsIldUIjoyfQ%3D%3D%7C0%7C%7C%7C&sda
>>>>>> t
>>>>>> a
>>>>>> =m
>>>>>> P
>>>>>> UXMdb%2FYOyGIm0rnncaWgcqtwkUIh8364IP3eYDPe4%3D&reserved=0
>>>>>> https://auto/
>>>>>> builder.yoctoproject.org%2Fvalkyrie%2F%23%2Fbuilders%2F30%2Fbuild
>>>>>> s
>>>>>> %
>>>>>> 2F
>>>>>> 2
>>>>>> 199&data=05%7C02%7Ctony.rex%40ericsson.com%7Cde1ae2aceeda470b3816
>>>>>> 0
>>>>>> 8
>>>>>> dd
>>>>>> d
>>>>>> efdd730%7C92e84cebfbfd47abbe52080c6b87953f%7C0%7C0%7C638911904005
>>>>>> 2
>>>>>> 2
>>>>>> 74
>>>>>> 3
>>>>>> 2%7CUnknown%7CTWFpbGZsb3d8eyJFbXB0eU1hcGkiOnRydWUsIlYiOiIwLjAuMDA
>>>>>> w
>>>>>> M
>>>>>> CI
>>>>>> s
>>>>>> IlAiOiJXaW4zMiIsIkFOIjoiTWFpbCIsIldUIjoyfQ%3D%3D%7C0%7C%7C%7C&sda
>>>>>> t
>>>>>> a
>>>>>> =R
>>>>>> 0
>>>>>> HFameGl2LmIVMT9ml%2BDMnj%2FMbG6SW0%2F3yNZHQu2js%3D&reserved=0
>>>>>> https://auto/
>>>>>> builder.yoctoproject.org%2Fvalkyrie%2F%23%2Fbuilders%2F16%2Fbuild
>>>>>> s
>>>>>> %
>>>>>> 2F
>>>>>> 2
>>>>>> 247&data=05%7C02%7Ctony.rex%40ericsson.com%7Cde1ae2aceeda470b3816
>>>>>> 0
>>>>>> 8
>>>>>> dd
>>>>>> d
>>>>>> efdd730%7C92e84cebfbfd47abbe52080c6b87953f%7C0%7C0%7C638911904005
>>>>>> 2
>>>>>> 4
>>>>>> 53
>>>>>> 8
>>>>>> 7%7CUnknown%7CTWFpbGZsb3d8eyJFbXB0eU1hcGkiOnRydWUsIlYiOiIwLjAuMDA
>>>>>> w
>>>>>> M
>>>>>> CI
>>>>>> s
>>>>>> IlAiOiJXaW4zMiIsIkFOIjoiTWFpbCIsIldUIjoyfQ%3D%3D%7C0%7C%7C%7C&sda
>>>>>> t
>>>>>> a
>>>>>> =N
>>>>>> S
>>>>>> FhEupwMkgqUAhpRgFbKnyFl%2FsY%2B4UJMhVQHhqOaNM%3D&reserved=0
>>>>>>
>>>>>> Can you have a look at this error please?
>>>>>>
>>>>>> Best regards,
>>>>>> Mathieu
>>>>>>
>>>>>> --
>>>>>> Mathieu Dubois-Briand, Bootlin
>>>>>> Embedded Linux and Kernel engineering https://boot/
>>>>>> lin.com%2F&data=05%7C02%7Ctony.rex%40ericsson.com%7Cde1ae2aceeda4
>>>>>> 7
>>>>>> 0
>>>>>> b3
>>>>>> 8
>>>>>> 1608dddefdd730%7C92e84cebfbfd47abbe52080c6b87953f%7C0%7C0%7C63891
>>>>>> 1
>>>>>> 9
>>>>>> 04
>>>>>> 0
>>>>>> 05263327%7CUnknown%7CTWFpbGZsb3d8eyJFbXB0eU1hcGkiOnRydWUsIlYiOiIw
>>>>>> L
>>>>>> j
>>>>>> Au
>>>>>> M
>>>>>> DAwMCIsIlAiOiJXaW4zMiIsIkFOIjoiTWFpbCIsIldUIjoyfQ%3D%3D%7C0%7C%7C
>>>>>> %
>>>>>> 7
>>>>>> C&
>>>>>> s
>>>>>> data=oyi7YWsXDVHn7QB4fDdG1sBp7gQsfGWcJTWtX1CjAw4%3D&reserved=0
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>> 
>>>

Re: [OE-core] [PATCH 0/1] base.bbclass: Fix the recipe parsing race issue

[Tony Rex]

It also seems possible to add an include file with the TAGNAME and PV setting to the recipe either by require or include:
cat recipes-test/gitrepotest/gitrepotest.inc
TAGNAME:pn-gitrepotest="2"
PV:pn-gitrepotest="12"

git diff
diff --git a/meta-selftest/recipes-test/gitrepotest/gitrepotest.bb b/meta-selftest/recipes-test/gitrepotest/gitrepotest.bb
index f1b6c55833..f5d70f2c17 100644
--- a/meta-selftest/recipes-test/gitrepotest/gitrepotest.bb
+++ b/meta-selftest/recipes-test/gitrepotest/gitrepotest.bb
@@ -2,15 +2,18 @@ SUMMARY = "Test recipe for git repo initialization"
 HOMEPAGE = "https://git.yoctoproject.org/git/matchbox-panel-2"
 LICENSE = "GPL-2.0-or-later"
 LIC_FILES_CHKSUM = "file://COPYING;md5=94d55d512a9ba36caa9b7df079bae19f"
+require gitrepotest.inc

 INHIBIT_DEFAULT_DEPS = "1"

 PATCHTOOL="git"

-SRC_URI = "git://git.yoctoproject.org/git/matchbox-panel-2;branch=master;protocol=https \
+TAG="${TAGNAME}.${PV}"
+SRC_URI = "git://git.yoctoproject.org/git/matchbox-panel-2;protocol=https;tag=${TAG} \
            file://0001-testpatch.patch \
           "

-SRCREV = "f82ca3f42510fb3ef10f598b393eb373a2c34ca7"
+#SRCREV = "f82ca3f42510fb3ef10f598b393eb373a2c34ca7"

 S = "${WORKDIR}/git"
+BBCLASSEXTEND="nativesdk"

Tony

-----Original Message-----
From: Tony Rex
Sent: Monday, 25 August 2025 17:49
To: Gyorgy Sarvari <skandigraun@gmail.com>; Alexander Kanavin <alex.kanavin@gmail.com>
Cc: Mathieu Dubois-Briand <mathieu.dubois-briand@bootlin.com>; openembedded-core@lists.openembedded.org
Subject: RE: [OE-core] [PATCH 0/1] base.bbclass: Fix the recipe parsing race issue

It is the full error output that is in the mail thread, there is nothing more I think.

But can you please try to reproduce the error by doing the changes in meta-selftest layer described below and evaluate the outcome.
I think the key is the +BBCLASSEXTEND="nativesdk" line.

git diff
diff --git a/meta-selftest/conf/layer.conf b/meta-selftest/conf/layer.conf index 48ca8464f9..3d83072fb0 100644
--- a/meta-selftest/conf/layer.conf
+++ b/meta-selftest/conf/layer.conf
@@ -12,3 +12,6 @@ BBFILE_PRIORITY_selftest = "5"
 addpylib ${LAYERDIR}/lib oeqa

 LAYERSERIES_COMPAT_selftest = "scarthgap"
+
+TAGNAME:pn-gitrepotest="2"
+PV:pn-gitrepotest="12"
diff --git a/meta-selftest/recipes-test/gitrepotest/gitrepotest.bb b/meta-selftest/recipes-test/gitrepotest/gitrepotest.bb
index f1b6c55833..10943584df 100644
--- a/meta-selftest/recipes-test/gitrepotest/gitrepotest.bb
+++ b/meta-selftest/recipes-test/gitrepotest/gitrepotest.bb
@@ -7,10 +7,12 @@ INHIBIT_DEFAULT_DEPS = "1"

 PATCHTOOL="git"

-SRC_URI = "git://git.yoctoproject.org/git/matchbox-panel-2;branch=master;protocol=https \
+TAG="${TAGNAME}.${PV}"
+SRC_URI =
+"git://git.yoctoproject.org/git/matchbox-panel-2;branch=master;protocol
+=https;tag=${TAG} \
            file://0001-testpatch.patch \
           "

-SRCREV = "f82ca3f42510fb3ef10f598b393eb373a2c34ca7"
+#SRCREV = "f82ca3f42510fb3ef10f598b393eb373a2c34ca7"

 S = "${WORKDIR}/git"
+BBCLASSEXTEND="nativesdk"


You should get something like this:
ERROR: ExpansionError during parsing .../oe-core/meta-selftest/recipes-test/gitrepotest/gitrepotest.bb   | ETA:  0:00:35
bb.data_smart.ExpansionError: Failure expanding variable fetcher_hashes_dummyfunc[vardepvalue], expression was ${@bb.fetch.get_hashvalue(d)} which triggered exception FetchError: Fetcher failure: Unable to resolve '${TAGNAME}.1.0' in upstream git repository in git ls-remote output for git.yoctoproject.org/git/matchbox-panel-2
The variable dependency chain for the failure is: fetcher_hashes_dummyfunc[vardepvalue]

ERROR: Parsing halted due to errors, see error messages above


Tony

-----Original Message-----
From: Gyorgy Sarvari <skandigraun@gmail.com>
Sent: Friday, 22 August 2025 16:10
To: Tony Rex <tony.rex@ericsson.com>; Alexander Kanavin <alex.kanavin@gmail.com>
Cc: Mathieu Dubois-Briand <mathieu.dubois-briand@bootlin.com>; openembedded-core@lists.openembedded.org
Subject: Re: [OE-core] [PATCH 0/1] base.bbclass: Fix the recipe parsing race issue

On 8/22/25 13:26, Tony Rex wrote:
> So to me it looks like the fetcher_hashes_dummyfunc makes it possible to catch any faulty or unwanted setups in the parsing stage.
>
> It seems hard to reproduce and expose this error on a common target, but I can reproduce it easily in the local setup.
> So I share my observations instead and hopefully you can make some sense out them.
>
> The error that I'm seeing is that when fetcher_hashes_dummyfunc[vardepvalue] = "${@bb.fetch.get_hashvalue(d)}" is executed the PV is not set, nor is the PN.
> So it seems that if the git ls-remote check happens to quick and early all variables are not set properly.
> The repo in this case is small (140K) and I believe that the access to
> the git server is fast (don't know of it matters) the git ls-remote check does not contain the right information if for example, the tag in the SRC_URI contains the PV value.
>
> When doing printouts from get_hashvalue the revs variable is empty and
> PN = defaultpkgname and PV = 1.0 has default values so the variables doesn't seem to be properly set yet and gives this error:
>
> ERROR: ExpansionError during parsing .../generic-if.bb################################   | ETA:  0:00:00
> bb.data_smart.ExpansionError: Failure expanding variable fetcher_hashes_dummyfunc[vardepvalue], expression was ${@bb.fetch.get_hashvalue(d)} which triggered exception FetchError: Fetcher failure: Unable to resolve '${TAGNAME}-1.0' in upstream git repository in git ls-remote output for gerrit...
> The variable dependency chain for the failure is:
> fetcher_hashes_dummyfunc[vardepvalue]
>
> ERROR: Parsing halted due to errors, see error messages above
>
> The patch that is provided fixes this issue, but there might be other and/or maybe better solutions to this matter.
> Another workaround for this problem that I have tested and that also
> seems to get around this issue is to set the SRC_URI with := or ?=
>
> This is why I consider it to be a race condition during parsing and the git ls-remote check.

Could you please share the _full_ error output, and the recipe that is causing it? Feel free to change confidential things to placeholders, but the more parts would be available, the better it would be.


>
> Tony
>
> -----Original Message-----
> From: Tony Rex
> Sent: Thursday, 21 August 2025 17:48
> To: Gyorgy Sarvari <skandigraun@gmail.com>; Alexander Kanavin
> <alex.kanavin@gmail.com>
> Cc: Mathieu Dubois-Briand <mathieu.dubois-briand@bootlin.com>;
> openembedded-core@lists.openembedded.org
> Subject: RE: [OE-core] [PATCH 0/1] base.bbclass: Fix the recipe
> parsing race issue
>
> I'm not sure how to run your (Gyorgy) code inside a recipe to test it, but in a bash script it works, of course.
>
> The thing that I'm looking at now is  a bit confusing, when commenting out the fetcher_hashes_dummyfunc function from the base.bbclass it seems to work properly.
>
> I don't know what the real purpose of this function is, am I missing something here, like a certain setup or a corner case?
> It seems to work both on whinlatter and scarthgap. Can someone please help out and clarify.
>
> git diff classes-global/base.bbclass
> diff --git a/meta/classes-global/base.bbclass
> b/meta/classes-global/base.bbclass
> index 6c08870571..83f82b2a4d 100644
> --- a/meta/classes-global/base.bbclass
> +++ b/meta/classes-global/base.bbclass
> @@ -128,16 +128,16 @@ def setup_hosttools_dir(dest, toolsvar, d, fatal=True):
>
>  # We can't use vardepvalue against do_fetch directly since that would overwrite  # the other task dependencies so we use an indirect function.
> -python fetcher_hashes_dummyfunc() {
> -    return
> -}
> -fetcher_hashes_dummyfunc[vardepvalue] = "${@bb.fetch.get_hashvalue(d)}"
> +#python fetcher_hashes_dummyfunc() {
> +#    return
> +#}
> +#fetcher_hashes_dummyfunc[vardepvalue] = "${@bb.fetch.get_hashvalue(d)}"
>
>  addtask fetch
>  do_fetch[dirs] = "${DL_DIR}"
>  do_fetch[file-checksums] = "${@bb.fetch.get_checksum_file_list(d)}"
>  do_fetch[file-checksums] += " ${@get_lic_checksum_file_list(d)}"
> -do_fetch[prefuncs] += "fetcher_hashes_dummyfunc"
> +#do_fetch[prefuncs] += "fetcher_hashes_dummyfunc"
>  do_fetch[network] = "1"
>  python base_do_fetch() {
>
> Tony
>
> -----Original Message-----
> From: Gyorgy Sarvari <skandigraun@gmail.com>
> Sent: Thursday, 21 August 2025 09:30
> To: Tony Rex <tony.rex@ericsson.com>; Alexander Kanavin
> <alex.kanavin@gmail.com>
> Cc: Mathieu Dubois-Briand <mathieu.dubois-briand@bootlin.com>;
> openembedded-core@lists.openembedded.org
> Subject: Re: [OE-core] [PATCH 0/1] base.bbclass: Fix the recipe
> parsing race issue
>
> On 8/21/25 09:10, Tony Rex wrote:
>> Yes, I get the same or similar error if the patch is not applied.
>>
>> ERROR: ExpansionError during parsing .../generic-if.bb################################   | ETA:  0:00:00
>> bb.data_smart.ExpansionError: Failure expanding variable fetcher_hashes_dummyfunc[vardepvalue], expression was ${@bb.fetch.get_hashvalue(d)} which triggered exception FetchError: Fetcher failure: Unable to resolve '${TAGNAME}-1.0' in upstream git repository in git ls-remote output for gerrit...
>> The variable dependency chain for the failure is:
>> fetcher_hashes_dummyfunc[vardepvalue]
>>
>> ERROR: Parsing halted due to errors, see error messages above
>>
>> How does your mytest_get.bb look like? Since I have had problem with reproducing the error on in a common setup maybe it can be done with your bb file.
>>
>> Tony
> The failure I reproduced is not recipe dependent. As some diagnostic test, try this script a few times with your failing git repo (change the repo placeholder):
>
> for i in `seq 50`; do
>   git clone $GIT_REPO_URL target_folder
>   git -C ./target_folder ls-remote
>   if [ $? -ne 0 ]; then
>     echo ls-remote failed, iteration: $i
>     break
>   fi
>   rm -rf ./target_folder
> done
>
> If I haven't made a typo, the script should clone your repo, do an ls-remote inside it, delete the folder, and do the same again.
>
> If your problem is the same that I reproduced, then I would expect the above script to fail sometimes - if it does, then the problem is with the infrastructure (network, remote git server, etc... something is flaky).
>
> Just for the record: the script generates a small amount of successive requests which some network admins might consider flooding.
>
>
>
>> -----Original Message-----
>> From: Gyorgy Sarvari <skandigraun@gmail.com>
>> Sent: Wednesday, 20 August 2025 18:12
>> To: Tony Rex <tony.rex@ericsson.com>; Alexander Kanavin
>> <alex.kanavin@gmail.com>
>> Cc: Mathieu Dubois-Briand <mathieu.dubois-briand@bootlin.com>;
>> openembedded-core@lists.openembedded.org
>> Subject: Re: [OE-core] [PATCH 0/1] base.bbclass: Fix the recipe
>> parsing race issue
>>
>> [You don't often get email from skandigraun@gmail.com. Learn why this
>> is important at https://aka.ms/LearnAboutSenderIdentification ]
>>
>> On 8/20/25 17:44, Gyorgy Sarvari via lists.openembedded.org wrote:
>>> On 8/20/25 13:53, Tony Rex wrote:
>>>> It was introduced by this commit:
>>>> https://git/
>>>> .yoctoproject.org%2Fpoky%2Fcommit%2F%3Fid%3Debfa1700f41b3411aec0401
>>>> 4
>>>> 4
>>>> 605166c35b8dd14&data=05%7C02%7Ctony.rex%40ericsson.com%7C7b5c099d17
>>>> 0
>>>> 0
>>>> 43080cca08dde004478d%7C92e84cebfbfd47abbe52080c6b87953f%7C0%7C0%7C6
>>>> 3
>>>> 8
>>>> 913031194171335%7CUnknown%7CTWFpbGZsb3d8eyJFbXB0eU1hcGkiOnRydWUsIlY
>>>> i
>>>> O
>>>> iIwLjAuMDAwMCIsIlAiOiJXaW4zMiIsIkFOIjoiTWFpbCIsIldUIjoyfQ%3D%3D%7C0
>>>> %
>>>> 7
>>>> C%7C%7C&sdata=iYvLGK1DwJk78tEvmL%2FPOL2olj7iNL87eXqdsEraxLw%3D&rese
>>>> r
>>>> v
>>>> ed=0
>>>>
>>>> Maybe "not allowed" is a strong wording, but that was how I interpreted it. Sorry, my bad if I misinterpreted it.
>>> In my understanding (which could be wrong), this is more to avoid
>>> working on a repository, where the exact work-revision can't be
>>> determined. But in your example, it can be determined, if your patch
>>> is not applied.
>>>
>>> Some extra ramblings, which may or may not be unadulterated garbage
>>> from
>>> me: Your initial commit message mentioned an expansion error. I
>>> looked into that a bit, and is it possible that the repository that
>>> is causing this issue has some access rate limit? The only thing
>>> that can fail sometimes in that code path seems to be "git ls-remote".
>>>
>>> Now, when I noticed this, I compiled a git version for myself that
>>> randomly throws an error with "git ls-remote" command, and when I
>>> used it with bitbake, I got this:
>>>
>>> ERROR: ExpansionError during parsing
>>> /home/meee/stuff/ptest-images/poky/meta/recipes-core/ell/mytest_get.
>>> b
>>> b
>>> bb.data_smart.ExpansionError: Failure expanding variable
>>> fetcher_hashes_dummyfunc[vardepvalue], expression was <etc etc etc>
>>>
>>> Is this the same error you got?
>> Sorry for the spam, one thing I forgot - the error on my end also contains details about the command that failed, pointing to git. If it's not the same as you get, do you have a copy of the full bitbake output you get when you see this error?
>>
>>>> Tony
>>>>
>>>> -----Original Message-----
>>>> From: Gyorgy Sarvari <skandigraun@gmail.com>
>>>> Sent: Wednesday, 20 August 2025 12:18
>>>> To: Tony Rex <tony.rex@ericsson.com>; Alexander Kanavin
>>>> <alex.kanavin@gmail.com>
>>>> Cc: Mathieu Dubois-Briand <mathieu.dubois-briand@bootlin.com>;
>>>> openembedded-core@lists.openembedded.org
>>>> Subject: Re: [OE-core] [PATCH 0/1] base.bbclass: Fix the recipe
>>>> parsing race issue
>>>>
>>>> [You don't often get email from skandigraun@gmail.com. Learn why
>>>> this is important at https://aka.ms/LearnAboutSenderIdentification
>>>> ]
>>>>
>>>> On 8/19/25 23:10, Tony Rex via lists.openembedded.org wrote:
>>>>> Hi Alex and Mathieu!
>>>>>
>>>>> Ok, so the patch fixes a bug and then the floating tag/branch function is in place again.
>>>>> The floating tag/branch function is broken due to the bug and the patch fixes it.
>>>>>
>>>>> The mail from Mathieu below might not fully reflect the issue here, since it seems to create a recipe with the help of devtool.
>>>>>
>>>>> To verify:
>>>>> 1. add layers/oe-core/meta-selftest to conf/bblayers.conf 2. run
>>>>> bitbake gitrepotest -c fetch
>>>>> result:
>>>>> NOTE: Tasks Summary: Attempted 2 tasks of which 0 didn't need to be rerun and all succeeded.
>>>>> 3. run bitbake gitrepotest -c cleanall 4. edit
>>>>> ../layers/oe-core/meta-selftest/recipes-test/gitrepotest/gitrepote
>>>>> s
>>>>> t
>>>>> .b
>>>>> b
>>>>> add tag=2.12 in SRC_URI
>>>>> SRC_URI =
>>>>> "git://git.yoctoproject.org/git/matchbox-panel-2;branch=master;tag
>>>>> =
>>>>> 2
>>>>> .1 2;protocol=https \ and comment out SRCREV #SRCREV =
>>>>> "f82ca3f42510fb3ef10f598b393eb373a2c34ca7"
>>>>> 5. run bitbake gitrepotest -c fetch
>>>>> result:
>>>>> NOTE: Tasks Summary: Attempted 2 tasks of which 0 didn't need to be rerun and all succeeded.
>>>>> The fetch goes through although this is not allowed according to the floating tag/branch error statement if not d.getVar("__BBSRCREV_SEEN"): in git.py.
>>>> Could you point to a commit/email/etc about this "no floating tags
>>>> allowed" statement? AFAIK tags without specifying revision is
>>>> allowed and supported. Not recommended, due to mandatory network
>>>> access, and because git tags are not immutable, but I'm surprised
>>>> to hear that they are not supposed to work (which of course can
>>>> also mean that I just missed a memo)
>>>>
>>>>> 6. run bitbake gitrepotest -c cleanall 7. apply patch
>>>>> 0001-base.bbclass-Fix-the-recipe-parsing-race-issue.patch
>>>>> 8. run bitbake gitrepotest -c fetch
>>>>> result:
>>>>> ERROR: gitrepotest-1.0-r0 do_fetch: Bitbake Fetcher Error:
>>>>> FetchError("Recipe uses a floating tag/branch '2.12' for repo
>>>>> 'git.yoctoproject.org/git/matchbox-panel-2' without a fixed SRCREV
>>>>> yet doesn't call bb.fetch2.get_srcrev() (use SRCPV in PV for
>>>>> OE).",
>>>>> None)
>>>>> ERROR: Logfile of failure stored in:
>>>>> .../build/tmp-glibc/work/cortexa53-wrs-linux/gitrepotest/1.0/temp/log.
>>>>> do_fetch.1254999
>>>>> ERROR: Task (.../layers/oe-core/meta-selftest/recipes-test/gitrepotest/gitrepotest.bb:do_fetch) failed with exit code '1'
>>>>> NOTE: Tasks Summary: Attempted 2 tasks of which 0 didn't need to be rerun and 1 failed.
>>>>>
>>>>> Error is the expected result when only having a tag set.
>>>>>
>>>>> Tony
>>>>>
>>>>> -----Original Message-----
>>>>> From: Alexander Kanavin <alex.kanavin@gmail.com>
>>>>> Sent: Tuesday, 19 August 2025 10:53
>>>>> To: Tony Rex <tony.rex@ericsson.com>
>>>>> Cc: Mathieu Dubois-Briand <mathieu.dubois-briand@bootlin.com>;
>>>>> openembedded-core@lists.openembedded.org
>>>>> Subject: Re: [OE-core] [PATCH 0/1] base.bbclass: Fix the recipe
>>>>> parsing race issue
>>>>>
>>>>> [You don't often get email from alex.kanavin@gmail.com. Learn why
>>>>> this is important at https://aka.ms/LearnAboutSenderIdentification
>>>>> ]
>>>>>
>>>>> You need to make things a little bit easier for maintainers and try to save their time.
>>>>>
>>>>> I would suggest that you provide clear steps to observe the issue.
>>>>> Attach the recipe file that would demonstrate the issue, and provide clear steps to trigger it. What is confusing right now is that you say here's a recipe that builds ok, but then you 'apply the patch' and then there's some kind of failure. So which patch needs to be applied?
>>>>>
>>>>> Alex
>>>>>
>>>>> On Fri, 15 Aug 2025 at 17:29, Tony Rex via lists.openembedded.org <tony.rex=ericsson.com@lists.openembedded.org> wrote:
>>>>>> Hi Mathieu!
>>>>>>
>>>>>> I think that this is how it is intended to work, bitbake does not allow floating tag/branch anymore.
>>>>>> Please correct me if I'm wrong @Richard Purdie (who implemented
>>>>>> this for security reasons I think.)
>>>>>>
>>>>>> The patch that I have makes sure that the parsing and setting of
>>>>>> the variables in the recipe are done before executing the do_fetch command, and as a side effect the floating tag/branch protection is working as intended again.
>>>>>>
>>>>>> How does the recipe look like?
>>>>>> devtool add kernel-module-hello-world https://git/.
>>>>>> yoctoproject.org%2Fgit%2Fkernel-module-hello-world&data=05%7C02%7
>>>>>> C
>>>>>> t
>>>>>> on
>>>>>> y
>>>>>> .rex%40ericsson.com%7Cde1ae2aceeda470b381608dddefdd730%7C92e84ceb
>>>>>> f
>>>>>> b
>>>>>> fd
>>>>>> 4
>>>>>> 7abbe52080c6b87953f%7C0%7C0%7C638911904005151487%7CUnknown%7CTWFp
>>>>>> b
>>>>>> G
>>>>>> Zs
>>>>>> b
>>>>>> 3d8eyJFbXB0eU1hcGkiOnRydWUsIlYiOiIwLjAuMDAwMCIsIlAiOiJXaW4zMiIsIk
>>>>>> F
>>>>>> O
>>>>>> Ij
>>>>>> o
>>>>>> iTWFpbCIsIldUIjoyfQ%3D%3D%7C0%7C%7C%7C&sdata=jyCicRQT%2FLtb1YxiFH
>>>>>> K
>>>>>> g
>>>>>> w5
>>>>>> B
>>>>>> nNktN4t9IciiyxtsEwO4%3D&reserved=0
>>>>>> The one I get looks like this:
>>>>>>
>>>>>> cat
>>>>>> build/workspace/recipes/kernel-module-hello-world/kernel-module-h
>>>>>> e
>>>>>> l
>>>>>> lo
>>>>>> -
>>>>>> world_git.bb
>>>>>> # Recipe created by recipetool
>>>>>> # This is the basis of a recipe and may need further editing in order to be fully functional.
>>>>>> # (Feel free to remove these comments when editing.)
>>>>>>
>>>>>> # WARNING: the following LICENSE and LIC_FILES_CHKSUM values are
>>>>>> best guesses - it is # your responsibility to verify that the values are complete and correct.
>>>>>> #
>>>>>> # The following license files were not able to be identified and
>>>>>> are # represented as "Unknown" below, you will need to check them yourself:
>>>>>> #   LICENSE
>>>>>> LICENSE = "Unknown"
>>>>>> LIC_FILES_CHKSUM = "file://LICENSE;md5=7b785be6433a2e4af6a641cc7f0330d1"
>>>>>>
>>>>>> SRC_URI = "git://git.yoctoproject.org/git/kernel-module-hello-world;protocol=https;branch=master"
>>>>>>
>>>>>> # Modify these as desired
>>>>>> PV = "1.0+git"
>>>>>> SRCREV = "b66f0f2bd559a1df81fc1bda240eba07a2ecdd2b"
>>>>>>
>>>>>> S = "${WORKDIR}/git"
>>>>>>
>>>>>> inherit module
>>>>>>
>>>>>> EXTRA_OEMAKE:append:task-install = " -C ${STAGING_KERNEL_DIR} M=${S}"
>>>>>> EXTRA_OEMAKE += "KERNEL_DIR=${STAGING_KERNEL_DIR}"
>>>>>>
>>>>>> And it contains SRCREV = "b66f0f2bd559a1df81fc1bda240eba07a2ecdd2b"
>>>>>>
>>>>>> Tony
>>>>>>
>>>>>> -----Original Message-----
>>>>>> From: Mathieu Dubois-Briand <mathieu.dubois-briand@bootlin.com>
>>>>>> Sent: Friday, 15 August 2025 11:26
>>>>>> To: Tony Rex <tony.rex@ericsson.com>;
>>>>>> openembedded-core@lists.openembedded.org
>>>>>> Subject: Re: [OE-core] [PATCH 0/1] base.bbclass: Fix the recipe
>>>>>> parsing race issue
>>>>>>
>>>>>> [You don't often get email from mathieu.dubois-briand@bootlin.com.
>>>>>> Learn why this is important at
>>>>>> https://aka.ms/LearnAboutSenderIdentification ]
>>>>>>
>>>>>> On Thu Aug 14, 2025 at 7:23 PM CEST, Tony Rex via lists.openembedded.org wrote:
>>>>>>> It seems that the race condition happens when doing do_fetch on
>>>>>>> really small repos and it can be difficult to reproduce so below
>>>>>>> the other case with floating tag/branch that is easier to verify.
>>>>>>>
>>>>>>> To verify the floating tag/branch issue have s simple recipe
>>>>>>> hello_1.0.bb and set the SRC_URI to SRC_URI = "git://<repository>;protocol=https;user=${USER};branch=master;tag=v1.0"
>>>>>>>
>>>>>>> run command bitbake hello everything builds.
>>>>>>>
>>>>>>> Apply the patch
>>>>>>>
>>>>>>> run command bitbake hello
>>>>>>> ERROR: hello-1.0-r1 do_fetch: Bitbake Fetcher Error:
>>>>>>> FetchError("Recipe uses a floating tag/branch 'v1.0' for repo
>>>>>>> '<repository>' without a fixed SRCREV yet doesn't call
>>>>>>> bb.fetch2.get_srcrev() (use SRCPV in PV for OE).", None)
>>>>>>> ERROR: Logfile of failure stored in:
>>>>>>> hello/1.0/temp/log.do_fetch.3289932
>>>>>>> ERROR: Task (hello/hello_1.0.bb:do_fetch) failed with exit code '1'
>>>>>>>
>>>>>>> Tony Rex (1):
>>>>>>>   base.bbclass: Fix the recipe parsing race issue
>>>>>>>
>>>>>>>  meta/classes-global/base.bbclass | 2 +-
>>>>>>>  1 file changed, 1 insertion(+), 1 deletion(-)
>>>>>> Hi Tony,
>>>>>>
>>>>>> Thanks for your patch.
>>>>>>
>>>>>> It looks like this is breaking some builds with the following error:
>>>>>>
>>>>>> oeqa.utils.subprocesstweak.OETestCalledProcessError: Command '. /srv/pokybuild/yocto-worker/qemux86-64/build/build/tmp/work/qemux86_64-poky-linux/core-image-sato/1.0/testsdkext/environment-setup-core2-64-poky-linux > /dev/null; devtool add kernel-module-hello-world https://git.yoctoproject.org/git/kernel-module-hello-world;' returned non-zero exit status 1.
>>>>>> ...
>>>>>> ERROR: Bitbake Fetcher Error: FetchError("Recipe uses a floating
>>>>>> tag/branch 'master' for repo
>>>>>> 'git.yoctoproject.org/git/kernel-module-hello-world' without a
>>>>>> fixed SRCREV yet doesn't call bb.fetch2.get_srcrev() (use SRCPV
>>>>>> in PV for OE).", None)
>>>>>>
>>>>>> https://auto/
>>>>>> builder.yoctoproject.org%2Fvalkyrie%2F%23%2Fbuilders%2F68%2Fbuild
>>>>>> s
>>>>>> %
>>>>>> 2F
>>>>>> 2
>>>>>> 284&data=05%7C02%7Ctony.rex%40ericsson.com%7Cde1ae2aceeda470b3816
>>>>>> 0
>>>>>> 8
>>>>>> dd
>>>>>> d
>>>>>> efdd730%7C92e84cebfbfd47abbe52080c6b87953f%7C0%7C0%7C638911904005
>>>>>> 2
>>>>>> 0
>>>>>> 90
>>>>>> 0
>>>>>> 3%7CUnknown%7CTWFpbGZsb3d8eyJFbXB0eU1hcGkiOnRydWUsIlYiOiIwLjAuMDA
>>>>>> w
>>>>>> M
>>>>>> CI
>>>>>> s
>>>>>> IlAiOiJXaW4zMiIsIkFOIjoiTWFpbCIsIldUIjoyfQ%3D%3D%7C0%7C%7C%7C&sda
>>>>>> t
>>>>>> a
>>>>>> =m
>>>>>> P
>>>>>> UXMdb%2FYOyGIm0rnncaWgcqtwkUIh8364IP3eYDPe4%3D&reserved=0
>>>>>> https://auto/
>>>>>> builder.yoctoproject.org%2Fvalkyrie%2F%23%2Fbuilders%2F30%2Fbuild
>>>>>> s
>>>>>> %
>>>>>> 2F
>>>>>> 2
>>>>>> 199&data=05%7C02%7Ctony.rex%40ericsson.com%7Cde1ae2aceeda470b3816
>>>>>> 0
>>>>>> 8
>>>>>> dd
>>>>>> d
>>>>>> efdd730%7C92e84cebfbfd47abbe52080c6b87953f%7C0%7C0%7C638911904005
>>>>>> 2
>>>>>> 2
>>>>>> 74
>>>>>> 3
>>>>>> 2%7CUnknown%7CTWFpbGZsb3d8eyJFbXB0eU1hcGkiOnRydWUsIlYiOiIwLjAuMDA
>>>>>> w
>>>>>> M
>>>>>> CI
>>>>>> s
>>>>>> IlAiOiJXaW4zMiIsIkFOIjoiTWFpbCIsIldUIjoyfQ%3D%3D%7C0%7C%7C%7C&sda
>>>>>> t
>>>>>> a
>>>>>> =R
>>>>>> 0
>>>>>> HFameGl2LmIVMT9ml%2BDMnj%2FMbG6SW0%2F3yNZHQu2js%3D&reserved=0
>>>>>> https://auto/
>>>>>> builder.yoctoproject.org%2Fvalkyrie%2F%23%2Fbuilders%2F16%2Fbuild
>>>>>> s
>>>>>> %
>>>>>> 2F
>>>>>> 2
>>>>>> 247&data=05%7C02%7Ctony.rex%40ericsson.com%7Cde1ae2aceeda470b3816
>>>>>> 0
>>>>>> 8
>>>>>> dd
>>>>>> d
>>>>>> efdd730%7C92e84cebfbfd47abbe52080c6b87953f%7C0%7C0%7C638911904005
>>>>>> 2
>>>>>> 4
>>>>>> 53
>>>>>> 8
>>>>>> 7%7CUnknown%7CTWFpbGZsb3d8eyJFbXB0eU1hcGkiOnRydWUsIlYiOiIwLjAuMDA
>>>>>> w
>>>>>> M
>>>>>> CI
>>>>>> s
>>>>>> IlAiOiJXaW4zMiIsIkFOIjoiTWFpbCIsIldUIjoyfQ%3D%3D%7C0%7C%7C%7C&sda
>>>>>> t
>>>>>> a
>>>>>> =N
>>>>>> S
>>>>>> FhEupwMkgqUAhpRgFbKnyFl%2FsY%2B4UJMhVQHhqOaNM%3D&reserved=0
>>>>>>
>>>>>> Can you have a look at this error please?
>>>>>>
>>>>>> Best regards,
>>>>>> Mathieu
>>>>>>
>>>>>> --
>>>>>> Mathieu Dubois-Briand, Bootlin
>>>>>> Embedded Linux and Kernel engineering https://boot/
>>>>>> lin.com%2F&data=05%7C02%7Ctony.rex%40ericsson.com%7Cde1ae2aceeda4
>>>>>> 7
>>>>>> 0
>>>>>> b3
>>>>>> 8
>>>>>> 1608dddefdd730%7C92e84cebfbfd47abbe52080c6b87953f%7C0%7C0%7C63891
>>>>>> 1
>>>>>> 9
>>>>>> 04
>>>>>> 0
>>>>>> 05263327%7CUnknown%7CTWFpbGZsb3d8eyJFbXB0eU1hcGkiOnRydWUsIlYiOiIw
>>>>>> L
>>>>>> j
>>>>>> Au
>>>>>> M
>>>>>> DAwMCIsIlAiOiJXaW4zMiIsIkFOIjoiTWFpbCIsIldUIjoyfQ%3D%3D%7C0%7C%7C
>>>>>> %
>>>>>> 7
>>>>>> C&
>>>>>> s
>>>>>> data=oyi7YWsXDVHn7QB4fDdG1sBp7gQsfGWcJTWtX1CjAw4%3D&reserved=0
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>> 
>>>

Re: [OE-core] [PATCH 0/1] base.bbclass: Fix the recipe parsing race issue

[Gyorgy Sarvari]

On 8/25/25 17:49, Tony Rex wrote:
> It is the full error output that is in the mail thread, there is nothing more I think.
>
> But can you please try to reproduce the error by doing the changes in meta-selftest layer described below and evaluate the outcome.
> I think the key is the +BBCLASSEXTEND="nativesdk" line.

I can confirm that I can reproduce it. I don't think that your proposed
patch is the correct solution, but your bug looks real.

>
> git diff
> diff --git a/meta-selftest/conf/layer.conf b/meta-selftest/conf/layer.conf
> index 48ca8464f9..3d83072fb0 100644
> --- a/meta-selftest/conf/layer.conf
> +++ b/meta-selftest/conf/layer.conf
> @@ -12,3 +12,6 @@ BBFILE_PRIORITY_selftest = "5"
>  addpylib ${LAYERDIR}/lib oeqa
>
>  LAYERSERIES_COMPAT_selftest = "scarthgap"
> +
> +TAGNAME:pn-gitrepotest="2"
> +PV:pn-gitrepotest="12"
> diff --git a/meta-selftest/recipes-test/gitrepotest/gitrepotest.bb b/meta-selftest/recipes-test/gitrepotest/gitrepotest.bb
> index f1b6c55833..10943584df 100644
> --- a/meta-selftest/recipes-test/gitrepotest/gitrepotest.bb
> +++ b/meta-selftest/recipes-test/gitrepotest/gitrepotest.bb
> @@ -7,10 +7,12 @@ INHIBIT_DEFAULT_DEPS = "1"
>
>  PATCHTOOL="git"
>
> -SRC_URI = "git://git.yoctoproject.org/git/matchbox-panel-2;branch=master;protocol=https \
> +TAG="${TAGNAME}.${PV}"
> +SRC_URI = "git://git.yoctoproject.org/git/matchbox-panel-2;branch=master;protocol=https;tag=${TAG} \
>             file://0001-testpatch.patch \
>            "
>
> -SRCREV = "f82ca3f42510fb3ef10f598b393eb373a2c34ca7"
> +#SRCREV = "f82ca3f42510fb3ef10f598b393eb373a2c34ca7"
>
>  S = "${WORKDIR}/git"
> +BBCLASSEXTEND="nativesdk"
>
>
> You should get something like this:
> ERROR: ExpansionError during parsing .../oe-core/meta-selftest/recipes-test/gitrepotest/gitrepotest.bb   | ETA:  0:00:35
> bb.data_smart.ExpansionError: Failure expanding variable fetcher_hashes_dummyfunc[vardepvalue], expression was ${@bb.fetch.get_hashvalue(d)} which triggered exception FetchError: Fetcher failure: Unable to resolve '${TAGNAME}.1.0' in upstream git repository in git ls-remote output for git.yoctoproject.org/git/matchbox-panel-2
> The variable dependency chain for the failure is: fetcher_hashes_dummyfunc[vardepvalue]
>
> ERROR: Parsing halted due to errors, see error messages above
>
>
> Tony
>
> -----Original Message-----
> From: Gyorgy Sarvari <skandigraun@gmail.com>
> Sent: Friday, 22 August 2025 16:10
> To: Tony Rex <tony.rex@ericsson.com>; Alexander Kanavin <alex.kanavin@gmail.com>
> Cc: Mathieu Dubois-Briand <mathieu.dubois-briand@bootlin.com>; openembedded-core@lists.openembedded.org
> Subject: Re: [OE-core] [PATCH 0/1] base.bbclass: Fix the recipe parsing race issue
>
> On 8/22/25 13:26, Tony Rex wrote:
>> So to me it looks like the fetcher_hashes_dummyfunc makes it possible to catch any faulty or unwanted setups in the parsing stage.
>>
>> It seems hard to reproduce and expose this error on a common target, but I can reproduce it easily in the local setup.
>> So I share my observations instead and hopefully you can make some sense out them.
>>
>> The error that I'm seeing is that when fetcher_hashes_dummyfunc[vardepvalue] = "${@bb.fetch.get_hashvalue(d)}" is executed the PV is not set, nor is the PN.
>> So it seems that if the git ls-remote check happens to quick and early all variables are not set properly.
>> The repo in this case is small (140K) and I believe that the access to
>> the git server is fast (don't know of it matters) the git ls-remote check does not contain the right information if for example, the tag in the SRC_URI contains the PV value.
>>
>> When doing printouts from get_hashvalue the revs variable is empty and
>> PN = defaultpkgname and PV = 1.0 has default values so the variables doesn't seem to be properly set yet and gives this error:
>>
>> ERROR: ExpansionError during parsing .../generic-if.bb################################   | ETA:  0:00:00
>> bb.data_smart.ExpansionError: Failure expanding variable fetcher_hashes_dummyfunc[vardepvalue], expression was ${@bb.fetch.get_hashvalue(d)} which triggered exception FetchError: Fetcher failure: Unable to resolve '${TAGNAME}-1.0' in upstream git repository in git ls-remote output for gerrit...
>> The variable dependency chain for the failure is:
>> fetcher_hashes_dummyfunc[vardepvalue]
>>
>> ERROR: Parsing halted due to errors, see error messages above
>>
>> The patch that is provided fixes this issue, but there might be other and/or maybe better solutions to this matter.
>> Another workaround for this problem that I have tested and that also
>> seems to get around this issue is to set the SRC_URI with := or ?=
>>
>> This is why I consider it to be a race condition during parsing and the git ls-remote check.
> Could you please share the _full_ error output, and the recipe that is causing it? Feel free to change confidential things to placeholders, but the more parts would be available, the better it would be.
>
>
>> Tony
>>
>> -----Original Message-----
>> From: Tony Rex
>> Sent: Thursday, 21 August 2025 17:48
>> To: Gyorgy Sarvari <skandigraun@gmail.com>; Alexander Kanavin
>> <alex.kanavin@gmail.com>
>> Cc: Mathieu Dubois-Briand <mathieu.dubois-briand@bootlin.com>;
>> openembedded-core@lists.openembedded.org
>> Subject: RE: [OE-core] [PATCH 0/1] base.bbclass: Fix the recipe
>> parsing race issue
>>
>> I'm not sure how to run your (Gyorgy) code inside a recipe to test it, but in a bash script it works, of course.
>>
>> The thing that I'm looking at now is  a bit confusing, when commenting out the fetcher_hashes_dummyfunc function from the base.bbclass it seems to work properly.
>>
>> I don't know what the real purpose of this function is, am I missing something here, like a certain setup or a corner case?
>> It seems to work both on whinlatter and scarthgap. Can someone please help out and clarify.
>>
>> git diff classes-global/base.bbclass
>> diff --git a/meta/classes-global/base.bbclass
>> b/meta/classes-global/base.bbclass
>> index 6c08870571..83f82b2a4d 100644
>> --- a/meta/classes-global/base.bbclass
>> +++ b/meta/classes-global/base.bbclass
>> @@ -128,16 +128,16 @@ def setup_hosttools_dir(dest, toolsvar, d, fatal=True):
>>
>>  # We can't use vardepvalue against do_fetch directly since that would overwrite  # the other task dependencies so we use an indirect function.
>> -python fetcher_hashes_dummyfunc() {
>> -    return
>> -}
>> -fetcher_hashes_dummyfunc[vardepvalue] = "${@bb.fetch.get_hashvalue(d)}"
>> +#python fetcher_hashes_dummyfunc() {
>> +#    return
>> +#}
>> +#fetcher_hashes_dummyfunc[vardepvalue] = "${@bb.fetch.get_hashvalue(d)}"
>>
>>  addtask fetch
>>  do_fetch[dirs] = "${DL_DIR}"
>>  do_fetch[file-checksums] = "${@bb.fetch.get_checksum_file_list(d)}"
>>  do_fetch[file-checksums] += " ${@get_lic_checksum_file_list(d)}"
>> -do_fetch[prefuncs] += "fetcher_hashes_dummyfunc"
>> +#do_fetch[prefuncs] += "fetcher_hashes_dummyfunc"
>>  do_fetch[network] = "1"
>>  python base_do_fetch() {
>>
>> Tony
>>
>> -----Original Message-----
>> From: Gyorgy Sarvari <skandigraun@gmail.com>
>> Sent: Thursday, 21 August 2025 09:30
>> To: Tony Rex <tony.rex@ericsson.com>; Alexander Kanavin
>> <alex.kanavin@gmail.com>
>> Cc: Mathieu Dubois-Briand <mathieu.dubois-briand@bootlin.com>;
>> openembedded-core@lists.openembedded.org
>> Subject: Re: [OE-core] [PATCH 0/1] base.bbclass: Fix the recipe
>> parsing race issue
>>
>> On 8/21/25 09:10, Tony Rex wrote:
>>> Yes, I get the same or similar error if the patch is not applied.
>>>
>>> ERROR: ExpansionError during parsing .../generic-if.bb################################   | ETA:  0:00:00
>>> bb.data_smart.ExpansionError: Failure expanding variable fetcher_hashes_dummyfunc[vardepvalue], expression was ${@bb.fetch.get_hashvalue(d)} which triggered exception FetchError: Fetcher failure: Unable to resolve '${TAGNAME}-1.0' in upstream git repository in git ls-remote output for gerrit...
>>> The variable dependency chain for the failure is:
>>> fetcher_hashes_dummyfunc[vardepvalue]
>>>
>>> ERROR: Parsing halted due to errors, see error messages above
>>>
>>> How does your mytest_get.bb look like? Since I have had problem with reproducing the error on in a common setup maybe it can be done with your bb file.
>>>
>>> Tony
>> The failure I reproduced is not recipe dependent. As some diagnostic test, try this script a few times with your failing git repo (change the repo placeholder):
>>
>> for i in `seq 50`; do
>>   git clone $GIT_REPO_URL target_folder
>>   git -C ./target_folder ls-remote
>>   if [ $? -ne 0 ]; then
>>     echo ls-remote failed, iteration: $i
>>     break
>>   fi
>>   rm -rf ./target_folder
>> done
>>
>> If I haven't made a typo, the script should clone your repo, do an ls-remote inside it, delete the folder, and do the same again.
>>
>> If your problem is the same that I reproduced, then I would expect the above script to fail sometimes - if it does, then the problem is with the infrastructure (network, remote git server, etc... something is flaky).
>>
>> Just for the record: the script generates a small amount of successive requests which some network admins might consider flooding.
>>
>>
>>
>>> -----Original Message-----
>>> From: Gyorgy Sarvari <skandigraun@gmail.com>
>>> Sent: Wednesday, 20 August 2025 18:12
>>> To: Tony Rex <tony.rex@ericsson.com>; Alexander Kanavin
>>> <alex.kanavin@gmail.com>
>>> Cc: Mathieu Dubois-Briand <mathieu.dubois-briand@bootlin.com>;
>>> openembedded-core@lists.openembedded.org
>>> Subject: Re: [OE-core] [PATCH 0/1] base.bbclass: Fix the recipe
>>> parsing race issue
>>>
>>> [You don't often get email from skandigraun@gmail.com. Learn why this
>>> is important at https://aka.ms/LearnAboutSenderIdentification ]
>>>
>>> On 8/20/25 17:44, Gyorgy Sarvari via lists.openembedded.org wrote:
>>>> On 8/20/25 13:53, Tony Rex wrote:
>>>>> It was introduced by this commit:
>>>>> https://git/
>>>>> .yoctoproject.org%2Fpoky%2Fcommit%2F%3Fid%3Debfa1700f41b3411aec0401
>>>>> 4
>>>>> 4
>>>>> 605166c35b8dd14&data=05%7C02%7Ctony.rex%40ericsson.com%7C7b5c099d17
>>>>> 0
>>>>> 0
>>>>> 43080cca08dde004478d%7C92e84cebfbfd47abbe52080c6b87953f%7C0%7C0%7C6
>>>>> 3
>>>>> 8
>>>>> 913031194171335%7CUnknown%7CTWFpbGZsb3d8eyJFbXB0eU1hcGkiOnRydWUsIlY
>>>>> i
>>>>> O
>>>>> iIwLjAuMDAwMCIsIlAiOiJXaW4zMiIsIkFOIjoiTWFpbCIsIldUIjoyfQ%3D%3D%7C0
>>>>> %
>>>>> 7
>>>>> C%7C%7C&sdata=iYvLGK1DwJk78tEvmL%2FPOL2olj7iNL87eXqdsEraxLw%3D&rese
>>>>> r
>>>>> v
>>>>> ed=0
>>>>>
>>>>> Maybe "not allowed" is a strong wording, but that was how I interpreted it. Sorry, my bad if I misinterpreted it.
>>>> In my understanding (which could be wrong), this is more to avoid
>>>> working on a repository, where the exact work-revision can't be
>>>> determined. But in your example, it can be determined, if your patch
>>>> is not applied.
>>>>
>>>> Some extra ramblings, which may or may not be unadulterated garbage
>>>> from
>>>> me: Your initial commit message mentioned an expansion error. I
>>>> looked into that a bit, and is it possible that the repository that
>>>> is causing this issue has some access rate limit? The only thing
>>>> that can fail sometimes in that code path seems to be "git ls-remote".
>>>>
>>>> Now, when I noticed this, I compiled a git version for myself that
>>>> randomly throws an error with "git ls-remote" command, and when I
>>>> used it with bitbake, I got this:
>>>>
>>>> ERROR: ExpansionError during parsing
>>>> /home/meee/stuff/ptest-images/poky/meta/recipes-core/ell/mytest_get.
>>>> b
>>>> b
>>>> bb.data_smart.ExpansionError: Failure expanding variable
>>>> fetcher_hashes_dummyfunc[vardepvalue], expression was <etc etc etc>
>>>>
>>>> Is this the same error you got?
>>> Sorry for the spam, one thing I forgot - the error on my end also contains details about the command that failed, pointing to git. If it's not the same as you get, do you have a copy of the full bitbake output you get when you see this error?
>>>
>>>>> Tony
>>>>>
>>>>> -----Original Message-----
>>>>> From: Gyorgy Sarvari <skandigraun@gmail.com>
>>>>> Sent: Wednesday, 20 August 2025 12:18
>>>>> To: Tony Rex <tony.rex@ericsson.com>; Alexander Kanavin
>>>>> <alex.kanavin@gmail.com>
>>>>> Cc: Mathieu Dubois-Briand <mathieu.dubois-briand@bootlin.com>;
>>>>> openembedded-core@lists.openembedded.org
>>>>> Subject: Re: [OE-core] [PATCH 0/1] base.bbclass: Fix the recipe
>>>>> parsing race issue
>>>>>
>>>>> [You don't often get email from skandigraun@gmail.com. Learn why
>>>>> this is important at https://aka.ms/LearnAboutSenderIdentification
>>>>> ]
>>>>>
>>>>> On 8/19/25 23:10, Tony Rex via lists.openembedded.org wrote:
>>>>>> Hi Alex and Mathieu!
>>>>>>
>>>>>> Ok, so the patch fixes a bug and then the floating tag/branch function is in place again.
>>>>>> The floating tag/branch function is broken due to the bug and the patch fixes it.
>>>>>>
>>>>>> The mail from Mathieu below might not fully reflect the issue here, since it seems to create a recipe with the help of devtool.
>>>>>>
>>>>>> To verify:
>>>>>> 1. add layers/oe-core/meta-selftest to conf/bblayers.conf 2. run
>>>>>> bitbake gitrepotest -c fetch
>>>>>> result:
>>>>>> NOTE: Tasks Summary: Attempted 2 tasks of which 0 didn't need to be rerun and all succeeded.
>>>>>> 3. run bitbake gitrepotest -c cleanall 4. edit
>>>>>> ../layers/oe-core/meta-selftest/recipes-test/gitrepotest/gitrepote
>>>>>> s
>>>>>> t
>>>>>> .b
>>>>>> b
>>>>>> add tag=2.12 in SRC_URI
>>>>>> SRC_URI =
>>>>>> "git://git.yoctoproject.org/git/matchbox-panel-2;branch=master;tag
>>>>>> =
>>>>>> 2
>>>>>> .1 2;protocol=https \ and comment out SRCREV #SRCREV =
>>>>>> "f82ca3f42510fb3ef10f598b393eb373a2c34ca7"
>>>>>> 5. run bitbake gitrepotest -c fetch
>>>>>> result:
>>>>>> NOTE: Tasks Summary: Attempted 2 tasks of which 0 didn't need to be rerun and all succeeded.
>>>>>> The fetch goes through although this is not allowed according to the floating tag/branch error statement if not d.getVar("__BBSRCREV_SEEN"): in git.py.
>>>>> Could you point to a commit/email/etc about this "no floating tags
>>>>> allowed" statement? AFAIK tags without specifying revision is
>>>>> allowed and supported. Not recommended, due to mandatory network
>>>>> access, and because git tags are not immutable, but I'm surprised
>>>>> to hear that they are not supposed to work (which of course can
>>>>> also mean that I just missed a memo)
>>>>>
>>>>>> 6. run bitbake gitrepotest -c cleanall 7. apply patch
>>>>>> 0001-base.bbclass-Fix-the-recipe-parsing-race-issue.patch
>>>>>> 8. run bitbake gitrepotest -c fetch
>>>>>> result:
>>>>>> ERROR: gitrepotest-1.0-r0 do_fetch: Bitbake Fetcher Error:
>>>>>> FetchError("Recipe uses a floating tag/branch '2.12' for repo
>>>>>> 'git.yoctoproject.org/git/matchbox-panel-2' without a fixed SRCREV
>>>>>> yet doesn't call bb.fetch2.get_srcrev() (use SRCPV in PV for
>>>>>> OE).",
>>>>>> None)
>>>>>> ERROR: Logfile of failure stored in:
>>>>>> .../build/tmp-glibc/work/cortexa53-wrs-linux/gitrepotest/1.0/temp/log.
>>>>>> do_fetch.1254999
>>>>>> ERROR: Task (.../layers/oe-core/meta-selftest/recipes-test/gitrepotest/gitrepotest.bb:do_fetch) failed with exit code '1'
>>>>>> NOTE: Tasks Summary: Attempted 2 tasks of which 0 didn't need to be rerun and 1 failed.
>>>>>>
>>>>>> Error is the expected result when only having a tag set.
>>>>>>
>>>>>> Tony
>>>>>>
>>>>>> -----Original Message-----
>>>>>> From: Alexander Kanavin <alex.kanavin@gmail.com>
>>>>>> Sent: Tuesday, 19 August 2025 10:53
>>>>>> To: Tony Rex <tony.rex@ericsson.com>
>>>>>> Cc: Mathieu Dubois-Briand <mathieu.dubois-briand@bootlin.com>;
>>>>>> openembedded-core@lists.openembedded.org
>>>>>> Subject: Re: [OE-core] [PATCH 0/1] base.bbclass: Fix the recipe
>>>>>> parsing race issue
>>>>>>
>>>>>> [You don't often get email from alex.kanavin@gmail.com. Learn why
>>>>>> this is important at https://aka.ms/LearnAboutSenderIdentification
>>>>>> ]
>>>>>>
>>>>>> You need to make things a little bit easier for maintainers and try to save their time.
>>>>>>
>>>>>> I would suggest that you provide clear steps to observe the issue.
>>>>>> Attach the recipe file that would demonstrate the issue, and provide clear steps to trigger it. What is confusing right now is that you say here's a recipe that builds ok, but then you 'apply the patch' and then there's some kind of failure. So which patch needs to be applied?
>>>>>>
>>>>>> Alex
>>>>>>
>>>>>> On Fri, 15 Aug 2025 at 17:29, Tony Rex via lists.openembedded.org <tony.rex=ericsson.com@lists.openembedded.org> wrote:
>>>>>>> Hi Mathieu!
>>>>>>>
>>>>>>> I think that this is how it is intended to work, bitbake does not allow floating tag/branch anymore.
>>>>>>> Please correct me if I'm wrong @Richard Purdie (who implemented
>>>>>>> this for security reasons I think.)
>>>>>>>
>>>>>>> The patch that I have makes sure that the parsing and setting of
>>>>>>> the variables in the recipe are done before executing the do_fetch command, and as a side effect the floating tag/branch protection is working as intended again.
>>>>>>>
>>>>>>> How does the recipe look like?
>>>>>>> devtool add kernel-module-hello-world https://git/.
>>>>>>> yoctoproject.org%2Fgit%2Fkernel-module-hello-world&data=05%7C02%7
>>>>>>> C
>>>>>>> t
>>>>>>> on
>>>>>>> y
>>>>>>> .rex%40ericsson.com%7Cde1ae2aceeda470b381608dddefdd730%7C92e84ceb
>>>>>>> f
>>>>>>> b
>>>>>>> fd
>>>>>>> 4
>>>>>>> 7abbe52080c6b87953f%7C0%7C0%7C638911904005151487%7CUnknown%7CTWFp
>>>>>>> b
>>>>>>> G
>>>>>>> Zs
>>>>>>> b
>>>>>>> 3d8eyJFbXB0eU1hcGkiOnRydWUsIlYiOiIwLjAuMDAwMCIsIlAiOiJXaW4zMiIsIk
>>>>>>> F
>>>>>>> O
>>>>>>> Ij
>>>>>>> o
>>>>>>> iTWFpbCIsIldUIjoyfQ%3D%3D%7C0%7C%7C%7C&sdata=jyCicRQT%2FLtb1YxiFH
>>>>>>> K
>>>>>>> g
>>>>>>> w5
>>>>>>> B
>>>>>>> nNktN4t9IciiyxtsEwO4%3D&reserved=0
>>>>>>> The one I get looks like this:
>>>>>>>
>>>>>>> cat
>>>>>>> build/workspace/recipes/kernel-module-hello-world/kernel-module-h
>>>>>>> e
>>>>>>> l
>>>>>>> lo
>>>>>>> -
>>>>>>> world_git.bb
>>>>>>> # Recipe created by recipetool
>>>>>>> # This is the basis of a recipe and may need further editing in order to be fully functional.
>>>>>>> # (Feel free to remove these comments when editing.)
>>>>>>>
>>>>>>> # WARNING: the following LICENSE and LIC_FILES_CHKSUM values are
>>>>>>> best guesses - it is # your responsibility to verify that the values are complete and correct.
>>>>>>> #
>>>>>>> # The following license files were not able to be identified and
>>>>>>> are # represented as "Unknown" below, you will need to check them yourself:
>>>>>>> #   LICENSE
>>>>>>> LICENSE = "Unknown"
>>>>>>> LIC_FILES_CHKSUM = "file://LICENSE;md5=7b785be6433a2e4af6a641cc7f0330d1"
>>>>>>>
>>>>>>> SRC_URI = "git://git.yoctoproject.org/git/kernel-module-hello-world;protocol=https;branch=master"
>>>>>>>
>>>>>>> # Modify these as desired
>>>>>>> PV = "1.0+git"
>>>>>>> SRCREV = "b66f0f2bd559a1df81fc1bda240eba07a2ecdd2b"
>>>>>>>
>>>>>>> S = "${WORKDIR}/git"
>>>>>>>
>>>>>>> inherit module
>>>>>>>
>>>>>>> EXTRA_OEMAKE:append:task-install = " -C ${STAGING_KERNEL_DIR} M=${S}"
>>>>>>> EXTRA_OEMAKE += "KERNEL_DIR=${STAGING_KERNEL_DIR}"
>>>>>>>
>>>>>>> And it contains SRCREV = "b66f0f2bd559a1df81fc1bda240eba07a2ecdd2b"
>>>>>>>
>>>>>>> Tony
>>>>>>>
>>>>>>> -----Original Message-----
>>>>>>> From: Mathieu Dubois-Briand <mathieu.dubois-briand@bootlin.com>
>>>>>>> Sent: Friday, 15 August 2025 11:26
>>>>>>> To: Tony Rex <tony.rex@ericsson.com>;
>>>>>>> openembedded-core@lists.openembedded.org
>>>>>>> Subject: Re: [OE-core] [PATCH 0/1] base.bbclass: Fix the recipe
>>>>>>> parsing race issue
>>>>>>>
>>>>>>> [You don't often get email from mathieu.dubois-briand@bootlin.com.
>>>>>>> Learn why this is important at
>>>>>>> https://aka.ms/LearnAboutSenderIdentification ]
>>>>>>>
>>>>>>> On Thu Aug 14, 2025 at 7:23 PM CEST, Tony Rex via lists.openembedded.org wrote:
>>>>>>>> It seems that the race condition happens when doing do_fetch on
>>>>>>>> really small repos and it can be difficult to reproduce so below
>>>>>>>> the other case with floating tag/branch that is easier to verify.
>>>>>>>>
>>>>>>>> To verify the floating tag/branch issue have s simple recipe
>>>>>>>> hello_1.0.bb and set the SRC_URI to SRC_URI = "git://<repository>;protocol=https;user=${USER};branch=master;tag=v1.0"
>>>>>>>>
>>>>>>>> run command bitbake hello everything builds.
>>>>>>>>
>>>>>>>> Apply the patch
>>>>>>>>
>>>>>>>> run command bitbake hello
>>>>>>>> ERROR: hello-1.0-r1 do_fetch: Bitbake Fetcher Error:
>>>>>>>> FetchError("Recipe uses a floating tag/branch 'v1.0' for repo
>>>>>>>> '<repository>' without a fixed SRCREV yet doesn't call
>>>>>>>> bb.fetch2.get_srcrev() (use SRCPV in PV for OE).", None)
>>>>>>>> ERROR: Logfile of failure stored in:
>>>>>>>> hello/1.0/temp/log.do_fetch.3289932
>>>>>>>> ERROR: Task (hello/hello_1.0.bb:do_fetch) failed with exit code '1'
>>>>>>>>
>>>>>>>> Tony Rex (1):
>>>>>>>>   base.bbclass: Fix the recipe parsing race issue
>>>>>>>>
>>>>>>>>  meta/classes-global/base.bbclass | 2 +-
>>>>>>>>  1 file changed, 1 insertion(+), 1 deletion(-)
>>>>>>> Hi Tony,
>>>>>>>
>>>>>>> Thanks for your patch.
>>>>>>>
>>>>>>> It looks like this is breaking some builds with the following error:
>>>>>>>
>>>>>>> oeqa.utils.subprocesstweak.OETestCalledProcessError: Command '. /srv/pokybuild/yocto-worker/qemux86-64/build/build/tmp/work/qemux86_64-poky-linux/core-image-sato/1.0/testsdkext/environment-setup-core2-64-poky-linux > /dev/null; devtool add kernel-module-hello-world https://git.yoctoproject.org/git/kernel-module-hello-world;' returned non-zero exit status 1.
>>>>>>> ...
>>>>>>> ERROR: Bitbake Fetcher Error: FetchError("Recipe uses a floating
>>>>>>> tag/branch 'master' for repo
>>>>>>> 'git.yoctoproject.org/git/kernel-module-hello-world' without a
>>>>>>> fixed SRCREV yet doesn't call bb.fetch2.get_srcrev() (use SRCPV
>>>>>>> in PV for OE).", None)
>>>>>>>
>>>>>>> https://auto/
>>>>>>> builder.yoctoproject.org%2Fvalkyrie%2F%23%2Fbuilders%2F68%2Fbuild
>>>>>>> s
>>>>>>> %
>>>>>>> 2F
>>>>>>> 2
>>>>>>> 284&data=05%7C02%7Ctony.rex%40ericsson.com%7Cde1ae2aceeda470b3816
>>>>>>> 0
>>>>>>> 8
>>>>>>> dd
>>>>>>> d
>>>>>>> efdd730%7C92e84cebfbfd47abbe52080c6b87953f%7C0%7C0%7C638911904005
>>>>>>> 2
>>>>>>> 0
>>>>>>> 90
>>>>>>> 0
>>>>>>> 3%7CUnknown%7CTWFpbGZsb3d8eyJFbXB0eU1hcGkiOnRydWUsIlYiOiIwLjAuMDA
>>>>>>> w
>>>>>>> M
>>>>>>> CI
>>>>>>> s
>>>>>>> IlAiOiJXaW4zMiIsIkFOIjoiTWFpbCIsIldUIjoyfQ%3D%3D%7C0%7C%7C%7C&sda
>>>>>>> t
>>>>>>> a
>>>>>>> =m
>>>>>>> P
>>>>>>> UXMdb%2FYOyGIm0rnncaWgcqtwkUIh8364IP3eYDPe4%3D&reserved=0
>>>>>>> https://auto/
>>>>>>> builder.yoctoproject.org%2Fvalkyrie%2F%23%2Fbuilders%2F30%2Fbuild
>>>>>>> s
>>>>>>> %
>>>>>>> 2F
>>>>>>> 2
>>>>>>> 199&data=05%7C02%7Ctony.rex%40ericsson.com%7Cde1ae2aceeda470b3816
>>>>>>> 0
>>>>>>> 8
>>>>>>> dd
>>>>>>> d
>>>>>>> efdd730%7C92e84cebfbfd47abbe52080c6b87953f%7C0%7C0%7C638911904005
>>>>>>> 2
>>>>>>> 2
>>>>>>> 74
>>>>>>> 3
>>>>>>> 2%7CUnknown%7CTWFpbGZsb3d8eyJFbXB0eU1hcGkiOnRydWUsIlYiOiIwLjAuMDA
>>>>>>> w
>>>>>>> M
>>>>>>> CI
>>>>>>> s
>>>>>>> IlAiOiJXaW4zMiIsIkFOIjoiTWFpbCIsIldUIjoyfQ%3D%3D%7C0%7C%7C%7C&sda
>>>>>>> t
>>>>>>> a
>>>>>>> =R
>>>>>>> 0
>>>>>>> HFameGl2LmIVMT9ml%2BDMnj%2FMbG6SW0%2F3yNZHQu2js%3D&reserved=0
>>>>>>> https://auto/
>>>>>>> builder.yoctoproject.org%2Fvalkyrie%2F%23%2Fbuilders%2F16%2Fbuild
>>>>>>> s
>>>>>>> %
>>>>>>> 2F
>>>>>>> 2
>>>>>>> 247&data=05%7C02%7Ctony.rex%40ericsson.com%7Cde1ae2aceeda470b3816
>>>>>>> 0
>>>>>>> 8
>>>>>>> dd
>>>>>>> d
>>>>>>> efdd730%7C92e84cebfbfd47abbe52080c6b87953f%7C0%7C0%7C638911904005
>>>>>>> 2
>>>>>>> 4
>>>>>>> 53
>>>>>>> 8
>>>>>>> 7%7CUnknown%7CTWFpbGZsb3d8eyJFbXB0eU1hcGkiOnRydWUsIlYiOiIwLjAuMDA
>>>>>>> w
>>>>>>> M
>>>>>>> CI
>>>>>>> s
>>>>>>> IlAiOiJXaW4zMiIsIkFOIjoiTWFpbCIsIldUIjoyfQ%3D%3D%7C0%7C%7C%7C&sda
>>>>>>> t
>>>>>>> a
>>>>>>> =N
>>>>>>> S
>>>>>>> FhEupwMkgqUAhpRgFbKnyFl%2FsY%2B4UJMhVQHhqOaNM%3D&reserved=0
>>>>>>>
>>>>>>> Can you have a look at this error please?
>>>>>>>
>>>>>>> Best regards,
>>>>>>> Mathieu
>>>>>>>
>>>>>>> --
>>>>>>> Mathieu Dubois-Briand, Bootlin
>>>>>>> Embedded Linux and Kernel engineering https://boot/
>>>>>>> lin.com%2F&data=05%7C02%7Ctony.rex%40ericsson.com%7Cde1ae2aceeda4
>>>>>>> 7
>>>>>>> 0
>>>>>>> b3
>>>>>>> 8
>>>>>>> 1608dddefdd730%7C92e84cebfbfd47abbe52080c6b87953f%7C0%7C0%7C63891
>>>>>>> 1
>>>>>>> 9
>>>>>>> 04
>>>>>>> 0
>>>>>>> 05263327%7CUnknown%7CTWFpbGZsb3d8eyJFbXB0eU1hcGkiOnRydWUsIlYiOiIw
>>>>>>> L
>>>>>>> j
>>>>>>> Au
>>>>>>> M
>>>>>>> DAwMCIsIlAiOiJXaW4zMiIsIkFOIjoiTWFpbCIsIldUIjoyfQ%3D%3D%7C0%7C%7C
>>>>>>> %
>>>>>>> 7
>>>>>>> C&
>>>>>>> s
>>>>>>> data=oyi7YWsXDVHn7QB4fDdG1sBp7gQsfGWcJTWtX1CjAw4%3D&reserved=0
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>> -=-=-=-=-=-=-=-=-=-=-=-
>>>> Links: You receive all messages sent to this group.
>>>> View/Reply Online (#222190):
>>>> https://list/
>>>> s.openembedded.org%2Fg%2Fopenembedded-core%2Fmessage%2F222190&data=0
>>>> 5
>>>> %
>>>> 7C02%7Ctony.rex%40ericsson.com%7C7b5c099d170043080cca08dde004478d%7C
>>>> 9
>>>> 2
>>>> e84cebfbfd47abbe52080c6b87953f%7C0%7C0%7C638913031194226137%7CUnknow
>>>> n
>>>> %
>>>> 7CTWFpbGZsb3d8eyJFbXB0eU1hcGkiOnRydWUsIlYiOiIwLjAuMDAwMCIsIlAiOiJXaW
>>>> 4
>>>> z
>>>> MiIsIkFOIjoiTWFpbCIsIldUIjoyfQ%3D%3D%7C0%7C%7C%7C&sdata=WbB87ExjlYOS
>>>> m
>>>> q
>>>> I0%2BEee6LbpupPfWxzk0vNYlbCUP5g%3D&reserved=0
>>>> Mute This Topic:
>>>> https://list/
>>>> s.openembedded.org%2Fmt%2F114706197%2F6084445&data=05%7C02%7Ctony.re
>>>> x
>>>> %
>>>> 40ericsson.com%7C7b5c099d170043080cca08dde004478d%7C92e84cebfbfd47ab
>>>> b
>>>> e
>>>> 52080c6b87953f%7C0%7C0%7C638913031194241729%7CUnknown%7CTWFpbGZsb3d8
>>>> e
>>>> y
>>>> JFbXB0eU1hcGkiOnRydWUsIlYiOiIwLjAuMDAwMCIsIlAiOiJXaW4zMiIsIkFOIjoiTW
>>>> F
>>>> p
>>>> bCIsIldUIjoyfQ%3D%3D%7C0%7C%7C%7C&sdata=bHfsli%2Fba3g8lov%2Bxmfovaal
>>>> 5
>>>> C
>>>> uBGuFnZVXs8AjHGIY%3D&reserved=0 Group Owner:
>>>> openembedded-core+owner@lists.openembedded.org
>>>> Unsubscribe:
>>>> https://list/
>>>> s.openembedded.org%2Fg%2Fopenembedded-core%2Funsub&data=05%7C02%7Cto
>>>> n
>>>> y
>>>> .rex%40ericsson.com%7C7b5c099d170043080cca08dde004478d%7C92e84cebfbf
>>>> d
>>>> 4
>>>> 7abbe52080c6b87953f%7C0%7C0%7C638913031194543958%7CUnknown%7CTWFpbGZ
>>>> s
>>>> b
>>>> 3d8eyJFbXB0eU1hcGkiOnRydWUsIlYiOiIwLjAuMDAwMCIsIlAiOiJXaW4zMiIsIkFOI
>>>> j
>>>> o
>>>> iTWFpbCIsIldUIjoyfQ%3D%3D%7C0%7C%7C%7C&sdata=gy4aai6oflVendGBp8nbYAL
>>>> 8
>>>> K
>>>> CGj77ObOMy2rYDnGN0%3D&reserved=0 [skandigraun@gmail.com]
>>>> -=-=-=-=-=-=-=-=-=-=-=-
>>>>

Re: [OE-core] [PATCH 0/1] base.bbclass: Fix the recipe parsing race issue

[Gyorgy Sarvari]

On 8/25/25 20:31, Gyorgy Sarvari wrote:
> On 8/25/25 17:49, Tony Rex wrote:
>> It is the full error output that is in the mail thread, there is nothing more I think.
>>
>> But can you please try to reproduce the error by doing the changes in meta-selftest layer described below and evaluate the outcome.
>> I think the key is the +BBCLASSEXTEND="nativesdk" line.
> I can confirm that I can reproduce it. I don't think that your proposed
> patch is the correct solution, but your bug looks real.

Looking at it it, I'm not 100% sure if it's a bug, but rather maybe a
bit surprising behavior, by design?  I guess other people will share
their opinion on this in the coming days - this a personal opinion after
running it in a debugger. Your finding about the BBCLASSEXTEND seems to
corroborate this too.

When you set an override with TAGNAME:pn-gitrepotest, that will be valid
only for the gitrepotest package. But when you enable nativesdk, it will
also check the nativesdk-gitrepotest package, for which this variable
isn't defined - there is no default value, and the pn-girepotest
override doesn't match. If you also define
TAGNAME:pn-nativesdk-gitrepotest, then it is defined, and it seems to be
resolved correctly.


>
>> git diff
>> diff --git a/meta-selftest/conf/layer.conf b/meta-selftest/conf/layer.conf
>> index 48ca8464f9..3d83072fb0 100644
>> --- a/meta-selftest/conf/layer.conf
>> +++ b/meta-selftest/conf/layer.conf
>> @@ -12,3 +12,6 @@ BBFILE_PRIORITY_selftest = "5"
>>  addpylib ${LAYERDIR}/lib oeqa
>>
>>  LAYERSERIES_COMPAT_selftest = "scarthgap"
>> +
>> +TAGNAME:pn-gitrepotest="2"
>> +PV:pn-gitrepotest="12"
>> diff --git a/meta-selftest/recipes-test/gitrepotest/gitrepotest.bb b/meta-selftest/recipes-test/gitrepotest/gitrepotest.bb
>> index f1b6c55833..10943584df 100644
>> --- a/meta-selftest/recipes-test/gitrepotest/gitrepotest.bb
>> +++ b/meta-selftest/recipes-test/gitrepotest/gitrepotest.bb
>> @@ -7,10 +7,12 @@ INHIBIT_DEFAULT_DEPS = "1"
>>
>>  PATCHTOOL="git"
>>
>> -SRC_URI = "git://git.yoctoproject.org/git/matchbox-panel-2;branch=master;protocol=https \
>> +TAG="${TAGNAME}.${PV}"
>> +SRC_URI = "git://git.yoctoproject.org/git/matchbox-panel-2;branch=master;protocol=https;tag=${TAG} \
>>             file://0001-testpatch.patch \
>>            "
>>
>> -SRCREV = "f82ca3f42510fb3ef10f598b393eb373a2c34ca7"
>> +#SRCREV = "f82ca3f42510fb3ef10f598b393eb373a2c34ca7"
>>
>>  S = "${WORKDIR}/git"
>> +BBCLASSEXTEND="nativesdk"
>>
>>
>> You should get something like this:
>> ERROR: ExpansionError during parsing .../oe-core/meta-selftest/recipes-test/gitrepotest/gitrepotest.bb   | ETA:  0:00:35
>> bb.data_smart.ExpansionError: Failure expanding variable fetcher_hashes_dummyfunc[vardepvalue], expression was ${@bb.fetch.get_hashvalue(d)} which triggered exception FetchError: Fetcher failure: Unable to resolve '${TAGNAME}.1.0' in upstream git repository in git ls-remote output for git.yoctoproject.org/git/matchbox-panel-2
>> The variable dependency chain for the failure is: fetcher_hashes_dummyfunc[vardepvalue]
>>
>> ERROR: Parsing halted due to errors, see error messages above
>>
>>
>> Tony
>>
>> -----Original Message-----
>> From: Gyorgy Sarvari <skandigraun@gmail.com>
>> Sent: Friday, 22 August 2025 16:10
>> To: Tony Rex <tony.rex@ericsson.com>; Alexander Kanavin <alex.kanavin@gmail.com>
>> Cc: Mathieu Dubois-Briand <mathieu.dubois-briand@bootlin.com>; openembedded-core@lists.openembedded.org
>> Subject: Re: [OE-core] [PATCH 0/1] base.bbclass: Fix the recipe parsing race issue
>>
>> On 8/22/25 13:26, Tony Rex wrote:
>>> So to me it looks like the fetcher_hashes_dummyfunc makes it possible to catch any faulty or unwanted setups in the parsing stage.
>>>
>>> It seems hard to reproduce and expose this error on a common target, but I can reproduce it easily in the local setup.
>>> So I share my observations instead and hopefully you can make some sense out them.
>>>
>>> The error that I'm seeing is that when fetcher_hashes_dummyfunc[vardepvalue] = "${@bb.fetch.get_hashvalue(d)}" is executed the PV is not set, nor is the PN.
>>> So it seems that if the git ls-remote check happens to quick and early all variables are not set properly.
>>> The repo in this case is small (140K) and I believe that the access to
>>> the git server is fast (don't know of it matters) the git ls-remote check does not contain the right information if for example, the tag in the SRC_URI contains the PV value.
>>>
>>> When doing printouts from get_hashvalue the revs variable is empty and
>>> PN = defaultpkgname and PV = 1.0 has default values so the variables doesn't seem to be properly set yet and gives this error:
>>>
>>> ERROR: ExpansionError during parsing .../generic-if.bb################################   | ETA:  0:00:00
>>> bb.data_smart.ExpansionError: Failure expanding variable fetcher_hashes_dummyfunc[vardepvalue], expression was ${@bb.fetch.get_hashvalue(d)} which triggered exception FetchError: Fetcher failure: Unable to resolve '${TAGNAME}-1.0' in upstream git repository in git ls-remote output for gerrit...
>>> The variable dependency chain for the failure is:
>>> fetcher_hashes_dummyfunc[vardepvalue]
>>>
>>> ERROR: Parsing halted due to errors, see error messages above
>>>
>>> The patch that is provided fixes this issue, but there might be other and/or maybe better solutions to this matter.
>>> Another workaround for this problem that I have tested and that also
>>> seems to get around this issue is to set the SRC_URI with := or ?=
>>>
>>> This is why I consider it to be a race condition during parsing and the git ls-remote check.
>> Could you please share the _full_ error output, and the recipe that is causing it? Feel free to change confidential things to placeholders, but the more parts would be available, the better it would be.
>>
>>
>>> Tony
>>>
>>> -----Original Message-----
>>> From: Tony Rex
>>> Sent: Thursday, 21 August 2025 17:48
>>> To: Gyorgy Sarvari <skandigraun@gmail.com>; Alexander Kanavin
>>> <alex.kanavin@gmail.com>
>>> Cc: Mathieu Dubois-Briand <mathieu.dubois-briand@bootlin.com>;
>>> openembedded-core@lists.openembedded.org
>>> Subject: RE: [OE-core] [PATCH 0/1] base.bbclass: Fix the recipe
>>> parsing race issue
>>>
>>> I'm not sure how to run your (Gyorgy) code inside a recipe to test it, but in a bash script it works, of course.
>>>
>>> The thing that I'm looking at now is  a bit confusing, when commenting out the fetcher_hashes_dummyfunc function from the base.bbclass it seems to work properly.
>>>
>>> I don't know what the real purpose of this function is, am I missing something here, like a certain setup or a corner case?
>>> It seems to work both on whinlatter and scarthgap. Can someone please help out and clarify.
>>>
>>> git diff classes-global/base.bbclass
>>> diff --git a/meta/classes-global/base.bbclass
>>> b/meta/classes-global/base.bbclass
>>> index 6c08870571..83f82b2a4d 100644
>>> --- a/meta/classes-global/base.bbclass
>>> +++ b/meta/classes-global/base.bbclass
>>> @@ -128,16 +128,16 @@ def setup_hosttools_dir(dest, toolsvar, d, fatal=True):
>>>
>>>  # We can't use vardepvalue against do_fetch directly since that would overwrite  # the other task dependencies so we use an indirect function.
>>> -python fetcher_hashes_dummyfunc() {
>>> -    return
>>> -}
>>> -fetcher_hashes_dummyfunc[vardepvalue] = "${@bb.fetch.get_hashvalue(d)}"
>>> +#python fetcher_hashes_dummyfunc() {
>>> +#    return
>>> +#}
>>> +#fetcher_hashes_dummyfunc[vardepvalue] = "${@bb.fetch.get_hashvalue(d)}"
>>>
>>>  addtask fetch
>>>  do_fetch[dirs] = "${DL_DIR}"
>>>  do_fetch[file-checksums] = "${@bb.fetch.get_checksum_file_list(d)}"
>>>  do_fetch[file-checksums] += " ${@get_lic_checksum_file_list(d)}"
>>> -do_fetch[prefuncs] += "fetcher_hashes_dummyfunc"
>>> +#do_fetch[prefuncs] += "fetcher_hashes_dummyfunc"
>>>  do_fetch[network] = "1"
>>>  python base_do_fetch() {
>>>
>>> Tony
>>>
>>> -----Original Message-----
>>> From: Gyorgy Sarvari <skandigraun@gmail.com>
>>> Sent: Thursday, 21 August 2025 09:30
>>> To: Tony Rex <tony.rex@ericsson.com>; Alexander Kanavin
>>> <alex.kanavin@gmail.com>
>>> Cc: Mathieu Dubois-Briand <mathieu.dubois-briand@bootlin.com>;
>>> openembedded-core@lists.openembedded.org
>>> Subject: Re: [OE-core] [PATCH 0/1] base.bbclass: Fix the recipe
>>> parsing race issue
>>>
>>> On 8/21/25 09:10, Tony Rex wrote:
>>>> Yes, I get the same or similar error if the patch is not applied.
>>>>
>>>> ERROR: ExpansionError during parsing .../generic-if.bb################################   | ETA:  0:00:00
>>>> bb.data_smart.ExpansionError: Failure expanding variable fetcher_hashes_dummyfunc[vardepvalue], expression was ${@bb.fetch.get_hashvalue(d)} which triggered exception FetchError: Fetcher failure: Unable to resolve '${TAGNAME}-1.0' in upstream git repository in git ls-remote output for gerrit...
>>>> The variable dependency chain for the failure is:
>>>> fetcher_hashes_dummyfunc[vardepvalue]
>>>>
>>>> ERROR: Parsing halted due to errors, see error messages above
>>>>
>>>> How does your mytest_get.bb look like? Since I have had problem with reproducing the error on in a common setup maybe it can be done with your bb file.
>>>>
>>>> Tony
>>> The failure I reproduced is not recipe dependent. As some diagnostic test, try this script a few times with your failing git repo (change the repo placeholder):
>>>
>>> for i in `seq 50`; do
>>>   git clone $GIT_REPO_URL target_folder
>>>   git -C ./target_folder ls-remote
>>>   if [ $? -ne 0 ]; then
>>>     echo ls-remote failed, iteration: $i
>>>     break
>>>   fi
>>>   rm -rf ./target_folder
>>> done
>>>
>>> If I haven't made a typo, the script should clone your repo, do an ls-remote inside it, delete the folder, and do the same again.
>>>
>>> If your problem is the same that I reproduced, then I would expect the above script to fail sometimes - if it does, then the problem is with the infrastructure (network, remote git server, etc... something is flaky).
>>>
>>> Just for the record: the script generates a small amount of successive requests which some network admins might consider flooding.
>>>
>>>
>>>
>>>> -----Original Message-----
>>>> From: Gyorgy Sarvari <skandigraun@gmail.com>
>>>> Sent: Wednesday, 20 August 2025 18:12
>>>> To: Tony Rex <tony.rex@ericsson.com>; Alexander Kanavin
>>>> <alex.kanavin@gmail.com>
>>>> Cc: Mathieu Dubois-Briand <mathieu.dubois-briand@bootlin.com>;
>>>> openembedded-core@lists.openembedded.org
>>>> Subject: Re: [OE-core] [PATCH 0/1] base.bbclass: Fix the recipe
>>>> parsing race issue
>>>>
>>>> [You don't often get email from skandigraun@gmail.com. Learn why this
>>>> is important at https://aka.ms/LearnAboutSenderIdentification ]
>>>>
>>>> On 8/20/25 17:44, Gyorgy Sarvari via lists.openembedded.org wrote:
>>>>> On 8/20/25 13:53, Tony Rex wrote:
>>>>>> It was introduced by this commit:
>>>>>> https://git/
>>>>>> .yoctoproject.org%2Fpoky%2Fcommit%2F%3Fid%3Debfa1700f41b3411aec0401
>>>>>> 4
>>>>>> 4
>>>>>> 605166c35b8dd14&data=05%7C02%7Ctony.rex%40ericsson.com%7C7b5c099d17
>>>>>> 0
>>>>>> 0
>>>>>> 43080cca08dde004478d%7C92e84cebfbfd47abbe52080c6b87953f%7C0%7C0%7C6
>>>>>> 3
>>>>>> 8
>>>>>> 913031194171335%7CUnknown%7CTWFpbGZsb3d8eyJFbXB0eU1hcGkiOnRydWUsIlY
>>>>>> i
>>>>>> O
>>>>>> iIwLjAuMDAwMCIsIlAiOiJXaW4zMiIsIkFOIjoiTWFpbCIsIldUIjoyfQ%3D%3D%7C0
>>>>>> %
>>>>>> 7
>>>>>> C%7C%7C&sdata=iYvLGK1DwJk78tEvmL%2FPOL2olj7iNL87eXqdsEraxLw%3D&rese
>>>>>> r
>>>>>> v
>>>>>> ed=0
>>>>>>
>>>>>> Maybe "not allowed" is a strong wording, but that was how I interpreted it. Sorry, my bad if I misinterpreted it.
>>>>> In my understanding (which could be wrong), this is more to avoid
>>>>> working on a repository, where the exact work-revision can't be
>>>>> determined. But in your example, it can be determined, if your patch
>>>>> is not applied.
>>>>>
>>>>> Some extra ramblings, which may or may not be unadulterated garbage
>>>>> from
>>>>> me: Your initial commit message mentioned an expansion error. I
>>>>> looked into that a bit, and is it possible that the repository that
>>>>> is causing this issue has some access rate limit? The only thing
>>>>> that can fail sometimes in that code path seems to be "git ls-remote".
>>>>>
>>>>> Now, when I noticed this, I compiled a git version for myself that
>>>>> randomly throws an error with "git ls-remote" command, and when I
>>>>> used it with bitbake, I got this:
>>>>>
>>>>> ERROR: ExpansionError during parsing
>>>>> /home/meee/stuff/ptest-images/poky/meta/recipes-core/ell/mytest_get.
>>>>> b
>>>>> b
>>>>> bb.data_smart.ExpansionError: Failure expanding variable
>>>>> fetcher_hashes_dummyfunc[vardepvalue], expression was <etc etc etc>
>>>>>
>>>>> Is this the same error you got?
>>>> Sorry for the spam, one thing I forgot - the error on my end also contains details about the command that failed, pointing to git. If it's not the same as you get, do you have a copy of the full bitbake output you get when you see this error?
>>>>
>>>>>> Tony
>>>>>>
>>>>>> -----Original Message-----
>>>>>> From: Gyorgy Sarvari <skandigraun@gmail.com>
>>>>>> Sent: Wednesday, 20 August 2025 12:18
>>>>>> To: Tony Rex <tony.rex@ericsson.com>; Alexander Kanavin
>>>>>> <alex.kanavin@gmail.com>
>>>>>> Cc: Mathieu Dubois-Briand <mathieu.dubois-briand@bootlin.com>;
>>>>>> openembedded-core@lists.openembedded.org
>>>>>> Subject: Re: [OE-core] [PATCH 0/1] base.bbclass: Fix the recipe
>>>>>> parsing race issue
>>>>>>
>>>>>> [You don't often get email from skandigraun@gmail.com. Learn why
>>>>>> this is important at https://aka.ms/LearnAboutSenderIdentification
>>>>>> ]
>>>>>>
>>>>>> On 8/19/25 23:10, Tony Rex via lists.openembedded.org wrote:
>>>>>>> Hi Alex and Mathieu!
>>>>>>>
>>>>>>> Ok, so the patch fixes a bug and then the floating tag/branch function is in place again.
>>>>>>> The floating tag/branch function is broken due to the bug and the patch fixes it.
>>>>>>>
>>>>>>> The mail from Mathieu below might not fully reflect the issue here, since it seems to create a recipe with the help of devtool.
>>>>>>>
>>>>>>> To verify:
>>>>>>> 1. add layers/oe-core/meta-selftest to conf/bblayers.conf 2. run
>>>>>>> bitbake gitrepotest -c fetch
>>>>>>> result:
>>>>>>> NOTE: Tasks Summary: Attempted 2 tasks of which 0 didn't need to be rerun and all succeeded.
>>>>>>> 3. run bitbake gitrepotest -c cleanall 4. edit
>>>>>>> ../layers/oe-core/meta-selftest/recipes-test/gitrepotest/gitrepote
>>>>>>> s
>>>>>>> t
>>>>>>> .b
>>>>>>> b
>>>>>>> add tag=2.12 in SRC_URI
>>>>>>> SRC_URI =
>>>>>>> "git://git.yoctoproject.org/git/matchbox-panel-2;branch=master;tag
>>>>>>> =
>>>>>>> 2
>>>>>>> .1 2;protocol=https \ and comment out SRCREV #SRCREV =
>>>>>>> "f82ca3f42510fb3ef10f598b393eb373a2c34ca7"
>>>>>>> 5. run bitbake gitrepotest -c fetch
>>>>>>> result:
>>>>>>> NOTE: Tasks Summary: Attempted 2 tasks of which 0 didn't need to be rerun and all succeeded.
>>>>>>> The fetch goes through although this is not allowed according to the floating tag/branch error statement if not d.getVar("__BBSRCREV_SEEN"): in git.py.
>>>>>> Could you point to a commit/email/etc about this "no floating tags
>>>>>> allowed" statement? AFAIK tags without specifying revision is
>>>>>> allowed and supported. Not recommended, due to mandatory network
>>>>>> access, and because git tags are not immutable, but I'm surprised
>>>>>> to hear that they are not supposed to work (which of course can
>>>>>> also mean that I just missed a memo)
>>>>>>
>>>>>>> 6. run bitbake gitrepotest -c cleanall 7. apply patch
>>>>>>> 0001-base.bbclass-Fix-the-recipe-parsing-race-issue.patch
>>>>>>> 8. run bitbake gitrepotest -c fetch
>>>>>>> result:
>>>>>>> ERROR: gitrepotest-1.0-r0 do_fetch: Bitbake Fetcher Error:
>>>>>>> FetchError("Recipe uses a floating tag/branch '2.12' for repo
>>>>>>> 'git.yoctoproject.org/git/matchbox-panel-2' without a fixed SRCREV
>>>>>>> yet doesn't call bb.fetch2.get_srcrev() (use SRCPV in PV for
>>>>>>> OE).",
>>>>>>> None)
>>>>>>> ERROR: Logfile of failure stored in:
>>>>>>> .../build/tmp-glibc/work/cortexa53-wrs-linux/gitrepotest/1.0/temp/log.
>>>>>>> do_fetch.1254999
>>>>>>> ERROR: Task (.../layers/oe-core/meta-selftest/recipes-test/gitrepotest/gitrepotest.bb:do_fetch) failed with exit code '1'
>>>>>>> NOTE: Tasks Summary: Attempted 2 tasks of which 0 didn't need to be rerun and 1 failed.
>>>>>>>
>>>>>>> Error is the expected result when only having a tag set.
>>>>>>>
>>>>>>> Tony
>>>>>>>
>>>>>>> -----Original Message-----
>>>>>>> From: Alexander Kanavin <alex.kanavin@gmail.com>
>>>>>>> Sent: Tuesday, 19 August 2025 10:53
>>>>>>> To: Tony Rex <tony.rex@ericsson.com>
>>>>>>> Cc: Mathieu Dubois-Briand <mathieu.dubois-briand@bootlin.com>;
>>>>>>> openembedded-core@lists.openembedded.org
>>>>>>> Subject: Re: [OE-core] [PATCH 0/1] base.bbclass: Fix the recipe
>>>>>>> parsing race issue
>>>>>>>
>>>>>>> [You don't often get email from alex.kanavin@gmail.com. Learn why
>>>>>>> this is important at https://aka.ms/LearnAboutSenderIdentification
>>>>>>> ]
>>>>>>>
>>>>>>> You need to make things a little bit easier for maintainers and try to save their time.
>>>>>>>
>>>>>>> I would suggest that you provide clear steps to observe the issue.
>>>>>>> Attach the recipe file that would demonstrate the issue, and provide clear steps to trigger it. What is confusing right now is that you say here's a recipe that builds ok, but then you 'apply the patch' and then there's some kind of failure. So which patch needs to be applied?
>>>>>>>
>>>>>>> Alex
>>>>>>>
>>>>>>> On Fri, 15 Aug 2025 at 17:29, Tony Rex via lists.openembedded.org <tony.rex=ericsson.com@lists.openembedded.org> wrote:
>>>>>>>> Hi Mathieu!
>>>>>>>>
>>>>>>>> I think that this is how it is intended to work, bitbake does not allow floating tag/branch anymore.
>>>>>>>> Please correct me if I'm wrong @Richard Purdie (who implemented
>>>>>>>> this for security reasons I think.)
>>>>>>>>
>>>>>>>> The patch that I have makes sure that the parsing and setting of
>>>>>>>> the variables in the recipe are done before executing the do_fetch command, and as a side effect the floating tag/branch protection is working as intended again.
>>>>>>>>
>>>>>>>> How does the recipe look like?
>>>>>>>> devtool add kernel-module-hello-world https://git/.
>>>>>>>> yoctoproject.org%2Fgit%2Fkernel-module-hello-world&data=05%7C02%7
>>>>>>>> C
>>>>>>>> t
>>>>>>>> on
>>>>>>>> y
>>>>>>>> .rex%40ericsson.com%7Cde1ae2aceeda470b381608dddefdd730%7C92e84ceb
>>>>>>>> f
>>>>>>>> b
>>>>>>>> fd
>>>>>>>> 4
>>>>>>>> 7abbe52080c6b87953f%7C0%7C0%7C638911904005151487%7CUnknown%7CTWFp
>>>>>>>> b
>>>>>>>> G
>>>>>>>> Zs
>>>>>>>> b
>>>>>>>> 3d8eyJFbXB0eU1hcGkiOnRydWUsIlYiOiIwLjAuMDAwMCIsIlAiOiJXaW4zMiIsIk
>>>>>>>> F
>>>>>>>> O
>>>>>>>> Ij
>>>>>>>> o
>>>>>>>> iTWFpbCIsIldUIjoyfQ%3D%3D%7C0%7C%7C%7C&sdata=jyCicRQT%2FLtb1YxiFH
>>>>>>>> K
>>>>>>>> g
>>>>>>>> w5
>>>>>>>> B
>>>>>>>> nNktN4t9IciiyxtsEwO4%3D&reserved=0
>>>>>>>> The one I get looks like this:
>>>>>>>>
>>>>>>>> cat
>>>>>>>> build/workspace/recipes/kernel-module-hello-world/kernel-module-h
>>>>>>>> e
>>>>>>>> l
>>>>>>>> lo
>>>>>>>> -
>>>>>>>> world_git.bb
>>>>>>>> # Recipe created by recipetool
>>>>>>>> # This is the basis of a recipe and may need further editing in order to be fully functional.
>>>>>>>> # (Feel free to remove these comments when editing.)
>>>>>>>>
>>>>>>>> # WARNING: the following LICENSE and LIC_FILES_CHKSUM values are
>>>>>>>> best guesses - it is # your responsibility to verify that the values are complete and correct.
>>>>>>>> #
>>>>>>>> # The following license files were not able to be identified and
>>>>>>>> are # represented as "Unknown" below, you will need to check them yourself:
>>>>>>>> #   LICENSE
>>>>>>>> LICENSE = "Unknown"
>>>>>>>> LIC_FILES_CHKSUM = "file://LICENSE;md5=7b785be6433a2e4af6a641cc7f0330d1"
>>>>>>>>
>>>>>>>> SRC_URI = "git://git.yoctoproject.org/git/kernel-module-hello-world;protocol=https;branch=master"
>>>>>>>>
>>>>>>>> # Modify these as desired
>>>>>>>> PV = "1.0+git"
>>>>>>>> SRCREV = "b66f0f2bd559a1df81fc1bda240eba07a2ecdd2b"
>>>>>>>>
>>>>>>>> S = "${WORKDIR}/git"
>>>>>>>>
>>>>>>>> inherit module
>>>>>>>>
>>>>>>>> EXTRA_OEMAKE:append:task-install = " -C ${STAGING_KERNEL_DIR} M=${S}"
>>>>>>>> EXTRA_OEMAKE += "KERNEL_DIR=${STAGING_KERNEL_DIR}"
>>>>>>>>
>>>>>>>> And it contains SRCREV = "b66f0f2bd559a1df81fc1bda240eba07a2ecdd2b"
>>>>>>>>
>>>>>>>> Tony
>>>>>>>>
>>>>>>>> -----Original Message-----
>>>>>>>> From: Mathieu Dubois-Briand <mathieu.dubois-briand@bootlin.com>
>>>>>>>> Sent: Friday, 15 August 2025 11:26
>>>>>>>> To: Tony Rex <tony.rex@ericsson.com>;
>>>>>>>> openembedded-core@lists.openembedded.org
>>>>>>>> Subject: Re: [OE-core] [PATCH 0/1] base.bbclass: Fix the recipe
>>>>>>>> parsing race issue
>>>>>>>>
>>>>>>>> [You don't often get email from mathieu.dubois-briand@bootlin.com.
>>>>>>>> Learn why this is important at
>>>>>>>> https://aka.ms/LearnAboutSenderIdentification ]
>>>>>>>>
>>>>>>>> On Thu Aug 14, 2025 at 7:23 PM CEST, Tony Rex via lists.openembedded.org wrote:
>>>>>>>>> It seems that the race condition happens when doing do_fetch on
>>>>>>>>> really small repos and it can be difficult to reproduce so below
>>>>>>>>> the other case with floating tag/branch that is easier to verify.
>>>>>>>>>
>>>>>>>>> To verify the floating tag/branch issue have s simple recipe
>>>>>>>>> hello_1.0.bb and set the SRC_URI to SRC_URI = "git://<repository>;protocol=https;user=${USER};branch=master;tag=v1.0"
>>>>>>>>>
>>>>>>>>> run command bitbake hello everything builds.
>>>>>>>>>
>>>>>>>>> Apply the patch
>>>>>>>>>
>>>>>>>>> run command bitbake hello
>>>>>>>>> ERROR: hello-1.0-r1 do_fetch: Bitbake Fetcher Error:
>>>>>>>>> FetchError("Recipe uses a floating tag/branch 'v1.0' for repo
>>>>>>>>> '<repository>' without a fixed SRCREV yet doesn't call
>>>>>>>>> bb.fetch2.get_srcrev() (use SRCPV in PV for OE).", None)
>>>>>>>>> ERROR: Logfile of failure stored in:
>>>>>>>>> hello/1.0/temp/log.do_fetch.3289932
>>>>>>>>> ERROR: Task (hello/hello_1.0.bb:do_fetch) failed with exit code '1'
>>>>>>>>>
>>>>>>>>> Tony Rex (1):
>>>>>>>>>   base.bbclass: Fix the recipe parsing race issue
>>>>>>>>>
>>>>>>>>>  meta/classes-global/base.bbclass | 2 +-
>>>>>>>>>  1 file changed, 1 insertion(+), 1 deletion(-)
>>>>>>>> Hi Tony,
>>>>>>>>
>>>>>>>> Thanks for your patch.
>>>>>>>>
>>>>>>>> It looks like this is breaking some builds with the following error:
>>>>>>>>
>>>>>>>> oeqa.utils.subprocesstweak.OETestCalledProcessError: Command '. /srv/pokybuild/yocto-worker/qemux86-64/build/build/tmp/work/qemux86_64-poky-linux/core-image-sato/1.0/testsdkext/environment-setup-core2-64-poky-linux > /dev/null; devtool add kernel-module-hello-world https://git.yoctoproject.org/git/kernel-module-hello-world;' returned non-zero exit status 1.
>>>>>>>> ...
>>>>>>>> ERROR: Bitbake Fetcher Error: FetchError("Recipe uses a floating
>>>>>>>> tag/branch 'master' for repo
>>>>>>>> 'git.yoctoproject.org/git/kernel-module-hello-world' without a
>>>>>>>> fixed SRCREV yet doesn't call bb.fetch2.get_srcrev() (use SRCPV
>>>>>>>> in PV for OE).", None)
>>>>>>>>
>>>>>>>> https://auto/
>>>>>>>> builder.yoctoproject.org%2Fvalkyrie%2F%23%2Fbuilders%2F68%2Fbuild
>>>>>>>> s
>>>>>>>> %
>>>>>>>> 2F
>>>>>>>> 2
>>>>>>>> 284&data=05%7C02%7Ctony.rex%40ericsson.com%7Cde1ae2aceeda470b3816
>>>>>>>> 0
>>>>>>>> 8
>>>>>>>> dd
>>>>>>>> d
>>>>>>>> efdd730%7C92e84cebfbfd47abbe52080c6b87953f%7C0%7C0%7C638911904005
>>>>>>>> 2
>>>>>>>> 0
>>>>>>>> 90
>>>>>>>> 0
>>>>>>>> 3%7CUnknown%7CTWFpbGZsb3d8eyJFbXB0eU1hcGkiOnRydWUsIlYiOiIwLjAuMDA
>>>>>>>> w
>>>>>>>> M
>>>>>>>> CI
>>>>>>>> s
>>>>>>>> IlAiOiJXaW4zMiIsIkFOIjoiTWFpbCIsIldUIjoyfQ%3D%3D%7C0%7C%7C%7C&sda
>>>>>>>> t
>>>>>>>> a
>>>>>>>> =m
>>>>>>>> P
>>>>>>>> UXMdb%2FYOyGIm0rnncaWgcqtwkUIh8364IP3eYDPe4%3D&reserved=0
>>>>>>>> https://auto/
>>>>>>>> builder.yoctoproject.org%2Fvalkyrie%2F%23%2Fbuilders%2F30%2Fbuild
>>>>>>>> s
>>>>>>>> %
>>>>>>>> 2F
>>>>>>>> 2
>>>>>>>> 199&data=05%7C02%7Ctony.rex%40ericsson.com%7Cde1ae2aceeda470b3816
>>>>>>>> 0
>>>>>>>> 8
>>>>>>>> dd
>>>>>>>> d
>>>>>>>> efdd730%7C92e84cebfbfd47abbe52080c6b87953f%7C0%7C0%7C638911904005
>>>>>>>> 2
>>>>>>>> 2
>>>>>>>> 74
>>>>>>>> 3
>>>>>>>> 2%7CUnknown%7CTWFpbGZsb3d8eyJFbXB0eU1hcGkiOnRydWUsIlYiOiIwLjAuMDA
>>>>>>>> w
>>>>>>>> M
>>>>>>>> CI
>>>>>>>> s
>>>>>>>> IlAiOiJXaW4zMiIsIkFOIjoiTWFpbCIsIldUIjoyfQ%3D%3D%7C0%7C%7C%7C&sda
>>>>>>>> t
>>>>>>>> a
>>>>>>>> =R
>>>>>>>> 0
>>>>>>>> HFameGl2LmIVMT9ml%2BDMnj%2FMbG6SW0%2F3yNZHQu2js%3D&reserved=0
>>>>>>>> https://auto/
>>>>>>>> builder.yoctoproject.org%2Fvalkyrie%2F%23%2Fbuilders%2F16%2Fbuild
>>>>>>>> s
>>>>>>>> %
>>>>>>>> 2F
>>>>>>>> 2
>>>>>>>> 247&data=05%7C02%7Ctony.rex%40ericsson.com%7Cde1ae2aceeda470b3816
>>>>>>>> 0
>>>>>>>> 8
>>>>>>>> dd
>>>>>>>> d
>>>>>>>> efdd730%7C92e84cebfbfd47abbe52080c6b87953f%7C0%7C0%7C638911904005
>>>>>>>> 2
>>>>>>>> 4
>>>>>>>> 53
>>>>>>>> 8
>>>>>>>> 7%7CUnknown%7CTWFpbGZsb3d8eyJFbXB0eU1hcGkiOnRydWUsIlYiOiIwLjAuMDA
>>>>>>>> w
>>>>>>>> M
>>>>>>>> CI
>>>>>>>> s
>>>>>>>> IlAiOiJXaW4zMiIsIkFOIjoiTWFpbCIsIldUIjoyfQ%3D%3D%7C0%7C%7C%7C&sda
>>>>>>>> t
>>>>>>>> a
>>>>>>>> =N
>>>>>>>> S
>>>>>>>> FhEupwMkgqUAhpRgFbKnyFl%2FsY%2B4UJMhVQHhqOaNM%3D&reserved=0
>>>>>>>>
>>>>>>>> Can you have a look at this error please?
>>>>>>>>
>>>>>>>> Best regards,
>>>>>>>> Mathieu
>>>>>>>>
>>>>>>>> --
>>>>>>>> Mathieu Dubois-Briand, Bootlin
>>>>>>>> Embedded Linux and Kernel engineering https://boot/
>>>>>>>> lin.com%2F&data=05%7C02%7Ctony.rex%40ericsson.com%7Cde1ae2aceeda4
>>>>>>>> 7
>>>>>>>> 0
>>>>>>>> b3
>>>>>>>> 8
>>>>>>>> 1608dddefdd730%7C92e84cebfbfd47abbe52080c6b87953f%7C0%7C0%7C63891
>>>>>>>> 1
>>>>>>>> 9
>>>>>>>> 04
>>>>>>>> 0
>>>>>>>> 05263327%7CUnknown%7CTWFpbGZsb3d8eyJFbXB0eU1hcGkiOnRydWUsIlYiOiIw
>>>>>>>> L
>>>>>>>> j
>>>>>>>> Au
>>>>>>>> M
>>>>>>>> DAwMCIsIlAiOiJXaW4zMiIsIkFOIjoiTWFpbCIsIldUIjoyfQ%3D%3D%7C0%7C%7C
>>>>>>>> %
>>>>>>>> 7
>>>>>>>> C&
>>>>>>>> s
>>>>>>>> data=oyi7YWsXDVHn7QB4fDdG1sBp7gQsfGWcJTWtX1CjAw4%3D&reserved=0
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>> -=-=-=-=-=-=-=-=-=-=-=-
>>>>> Links: You receive all messages sent to this group.
>>>>> View/Reply Online (#222190):
>>>>> https://list/
>>>>> s.openembedded.org%2Fg%2Fopenembedded-core%2Fmessage%2F222190&data=0
>>>>> 5
>>>>> %
>>>>> 7C02%7Ctony.rex%40ericsson.com%7C7b5c099d170043080cca08dde004478d%7C
>>>>> 9
>>>>> 2
>>>>> e84cebfbfd47abbe52080c6b87953f%7C0%7C0%7C638913031194226137%7CUnknow
>>>>> n
>>>>> %
>>>>> 7CTWFpbGZsb3d8eyJFbXB0eU1hcGkiOnRydWUsIlYiOiIwLjAuMDAwMCIsIlAiOiJXaW
>>>>> 4
>>>>> z
>>>>> MiIsIkFOIjoiTWFpbCIsIldUIjoyfQ%3D%3D%7C0%7C%7C%7C&sdata=WbB87ExjlYOS
>>>>> m
>>>>> q
>>>>> I0%2BEee6LbpupPfWxzk0vNYlbCUP5g%3D&reserved=0
>>>>> Mute This Topic:
>>>>> https://list/
>>>>> s.openembedded.org%2Fmt%2F114706197%2F6084445&data=05%7C02%7Ctony.re
>>>>> x
>>>>> %
>>>>> 40ericsson.com%7C7b5c099d170043080cca08dde004478d%7C92e84cebfbfd47ab
>>>>> b
>>>>> e
>>>>> 52080c6b87953f%7C0%7C0%7C638913031194241729%7CUnknown%7CTWFpbGZsb3d8
>>>>> e
>>>>> y
>>>>> JFbXB0eU1hcGkiOnRydWUsIlYiOiIwLjAuMDAwMCIsIlAiOiJXaW4zMiIsIkFOIjoiTW
>>>>> F
>>>>> p
>>>>> bCIsIldUIjoyfQ%3D%3D%7C0%7C%7C%7C&sdata=bHfsli%2Fba3g8lov%2Bxmfovaal
>>>>> 5
>>>>> C
>>>>> uBGuFnZVXs8AjHGIY%3D&reserved=0 Group Owner:
>>>>> openembedded-core+owner@lists.openembedded.org
>>>>> Unsubscribe:
>>>>> https://list/
>>>>> s.openembedded.org%2Fg%2Fopenembedded-core%2Funsub&data=05%7C02%7Cto
>>>>> n
>>>>> y
>>>>> .rex%40ericsson.com%7C7b5c099d170043080cca08dde004478d%7C92e84cebfbf
>>>>> d
>>>>> 4
>>>>> 7abbe52080c6b87953f%7C0%7C0%7C638913031194543958%7CUnknown%7CTWFpbGZ
>>>>> s
>>>>> b
>>>>> 3d8eyJFbXB0eU1hcGkiOnRydWUsIlYiOiIwLjAuMDAwMCIsIlAiOiJXaW4zMiIsIkFOI
>>>>> j
>>>>> o
>>>>> iTWFpbCIsIldUIjoyfQ%3D%3D%7C0%7C%7C%7C&sdata=gy4aai6oflVendGBp8nbYAL
>>>>> 8
>>>>> K
>>>>> CGj77ObOMy2rYDnGN0%3D&reserved=0 [skandigraun@gmail.com]
>>>>> -=-=-=-=-=-=-=-=-=-=-=-
>>>>>

Re: [OE-core] [PATCH 0/1] base.bbclass: Fix the recipe parsing race issue

[Tony Rex]

Yes, I think you're right, and that this might not be a bug, but a change in the behavior that is not seen in previous releases such as hardknott.
So the question is if this changed behavior is wanted/intended or a side effect, like you stated.
In earlier releases it seems that the PV and variables like TAGNAME was set before the override pn-nativesdk-gitrepotest was used/needed in
the parsing. However if the setting is done earlier like with the patch  fetcher_hashes_dummyfunc[vardepvalue] := "${@bb.fetch.get_hashvalue(d)}"
or SRC_URI := or ?= the pn-nativesdk-gitrepotest seems to pick up the value from pn-gitrepotest without complaints.

And I also think that the change in this area breaks the floating branch/tag "requirement" in git.py, if not d.getVar("__BBSRCREV_SEEN"):

Tony

-----Original Message-----
From: Gyorgy Sarvari <skandigraun@gmail.com>
Sent: Monday, 25 August 2025 22:14
To: Tony Rex <tony.rex@ericsson.com>; Alexander Kanavin <alex.kanavin@gmail.com>
Cc: Mathieu Dubois-Briand <mathieu.dubois-briand@bootlin.com>; openembedded-core@lists.openembedded.org
Subject: Re: [OE-core] [PATCH 0/1] base.bbclass: Fix the recipe parsing race issue

On 8/25/25 20:31, Gyorgy Sarvari wrote:
> On 8/25/25 17:49, Tony Rex wrote:
>> It is the full error output that is in the mail thread, there is nothing more I think.
>>
>> But can you please try to reproduce the error by doing the changes in meta-selftest layer described below and evaluate the outcome.
>> I think the key is the +BBCLASSEXTEND="nativesdk" line.
> I can confirm that I can reproduce it. I don't think that your
> proposed patch is the correct solution, but your bug looks real.

Looking at it it, I'm not 100% sure if it's a bug, but rather maybe a bit surprising behavior, by design?  I guess other people will share their opinion on this in the coming days - this a personal opinion after running it in a debugger. Your finding about the BBCLASSEXTEND seems to corroborate this too.

When you set an override with TAGNAME:pn-gitrepotest, that will be valid only for the gitrepotest package. But when you enable nativesdk, it will also check the nativesdk-gitrepotest package, for which this variable isn't defined - there is no default value, and the pn-girepotest override doesn't match. If you also define TAGNAME:pn-nativesdk-gitrepotest, then it is defined, and it seems to be resolved correctly.


>
>> git diff
>> diff --git a/meta-selftest/conf/layer.conf
>> b/meta-selftest/conf/layer.conf index 48ca8464f9..3d83072fb0 100644
>> --- a/meta-selftest/conf/layer.conf
>> +++ b/meta-selftest/conf/layer.conf
>> @@ -12,3 +12,6 @@ BBFILE_PRIORITY_selftest = "5"
>>  addpylib ${LAYERDIR}/lib oeqa
>>
>>  LAYERSERIES_COMPAT_selftest = "scarthgap"
>> +
>> +TAGNAME:pn-gitrepotest="2"
>> +PV:pn-gitrepotest="12"
>> diff --git a/meta-selftest/recipes-test/gitrepotest/gitrepotest.bb
>> b/meta-selftest/recipes-test/gitrepotest/gitrepotest.bb
>> index f1b6c55833..10943584df 100644
>> --- a/meta-selftest/recipes-test/gitrepotest/gitrepotest.bb
>> +++ b/meta-selftest/recipes-test/gitrepotest/gitrepotest.bb
>> @@ -7,10 +7,12 @@ INHIBIT_DEFAULT_DEPS = "1"
>>
>>  PATCHTOOL="git"
>>
>> -SRC_URI =
>> "git://git.yoctoproject.org/git/matchbox-panel-2;branch=master;protoc
>> ol=https \
>> +TAG="${TAGNAME}.${PV}"
>> +SRC_URI =
>> +"git://git.yoctoproject.org/git/matchbox-panel-2;branch=master;proto
>> +col=https;tag=${TAG} \
>>             file://0001-testpatch.patch \
>>            "
>>
>> -SRCREV = "f82ca3f42510fb3ef10f598b393eb373a2c34ca7"
>> +#SRCREV = "f82ca3f42510fb3ef10f598b393eb373a2c34ca7"
>>
>>  S = "${WORKDIR}/git"
>> +BBCLASSEXTEND="nativesdk"
>>
>>
>> You should get something like this:
>> ERROR: ExpansionError during parsing .../oe-core/meta-selftest/recipes-test/gitrepotest/gitrepotest.bb   | ETA:  0:00:35
>> bb.data_smart.ExpansionError: Failure expanding variable
>> fetcher_hashes_dummyfunc[vardepvalue], expression was
>> ${@bb.fetch.get_hashvalue(d)} which triggered exception FetchError:
>> Fetcher failure: Unable to resolve '${TAGNAME}.1.0' in upstream git
>> repository in git ls-remote output for
>> git.yoctoproject.org/git/matchbox-panel-2
>> The variable dependency chain for the failure is:
>> fetcher_hashes_dummyfunc[vardepvalue]
>>
>> ERROR: Parsing halted due to errors, see error messages above
>>
>>
>> Tony
>>
>> -----Original Message-----
>> From: Gyorgy Sarvari <skandigraun@gmail.com>
>> Sent: Friday, 22 August 2025 16:10
>> To: Tony Rex <tony.rex@ericsson.com>; Alexander Kanavin
>> <alex.kanavin@gmail.com>
>> Cc: Mathieu Dubois-Briand <mathieu.dubois-briand@bootlin.com>;
>> openembedded-core@lists.openembedded.org
>> Subject: Re: [OE-core] [PATCH 0/1] base.bbclass: Fix the recipe
>> parsing race issue
>>
>> On 8/22/25 13:26, Tony Rex wrote:
>>> So to me it looks like the fetcher_hashes_dummyfunc makes it possible to catch any faulty or unwanted setups in the parsing stage.
>>>
>>> It seems hard to reproduce and expose this error on a common target, but I can reproduce it easily in the local setup.
>>> So I share my observations instead and hopefully you can make some sense out them.
>>>
>>> The error that I'm seeing is that when fetcher_hashes_dummyfunc[vardepvalue] = "${@bb.fetch.get_hashvalue(d)}" is executed the PV is not set, nor is the PN.
>>> So it seems that if the git ls-remote check happens to quick and early all variables are not set properly.
>>> The repo in this case is small (140K) and I believe that the access
>>> to the git server is fast (don't know of it matters) the git ls-remote check does not contain the right information if for example, the tag in the SRC_URI contains the PV value.
>>>
>>> When doing printouts from get_hashvalue the revs variable is empty
>>> and PN = defaultpkgname and PV = 1.0 has default values so the variables doesn't seem to be properly set yet and gives this error:
>>>
>>> ERROR: ExpansionError during parsing .../generic-if.bb################################   | ETA:  0:00:00
>>> bb.data_smart.ExpansionError: Failure expanding variable fetcher_hashes_dummyfunc[vardepvalue], expression was ${@bb.fetch.get_hashvalue(d)} which triggered exception FetchError: Fetcher failure: Unable to resolve '${TAGNAME}-1.0' in upstream git repository in git ls-remote output for gerrit...
>>> The variable dependency chain for the failure is:
>>> fetcher_hashes_dummyfunc[vardepvalue]
>>>
>>> ERROR: Parsing halted due to errors, see error messages above
>>>
>>> The patch that is provided fixes this issue, but there might be other and/or maybe better solutions to this matter.
>>> Another workaround for this problem that I have tested and that also
>>> seems to get around this issue is to set the SRC_URI with := or ?=
>>>
>>> This is why I consider it to be a race condition during parsing and the git ls-remote check.
>> Could you please share the _full_ error output, and the recipe that is causing it? Feel free to change confidential things to placeholders, but the more parts would be available, the better it would be.
>>
>>
>>> Tony
>>>
>>> -----Original Message-----
>>> From: Tony Rex
>>> Sent: Thursday, 21 August 2025 17:48
>>> To: Gyorgy Sarvari <skandigraun@gmail.com>; Alexander Kanavin
>>> <alex.kanavin@gmail.com>
>>> Cc: Mathieu Dubois-Briand <mathieu.dubois-briand@bootlin.com>;
>>> openembedded-core@lists.openembedded.org
>>> Subject: RE: [OE-core] [PATCH 0/1] base.bbclass: Fix the recipe
>>> parsing race issue
>>>
>>> I'm not sure how to run your (Gyorgy) code inside a recipe to test it, but in a bash script it works, of course.
>>>
>>> The thing that I'm looking at now is  a bit confusing, when commenting out the fetcher_hashes_dummyfunc function from the base.bbclass it seems to work properly.
>>>
>>> I don't know what the real purpose of this function is, am I missing something here, like a certain setup or a corner case?
>>> It seems to work both on whinlatter and scarthgap. Can someone please help out and clarify.
>>>
>>> git diff classes-global/base.bbclass diff --git
>>> a/meta/classes-global/base.bbclass
>>> b/meta/classes-global/base.bbclass
>>> index 6c08870571..83f82b2a4d 100644
>>> --- a/meta/classes-global/base.bbclass
>>> +++ b/meta/classes-global/base.bbclass
>>> @@ -128,16 +128,16 @@ def setup_hosttools_dir(dest, toolsvar, d, fatal=True):
>>>
>>>  # We can't use vardepvalue against do_fetch directly since that would overwrite  # the other task dependencies so we use an indirect function.
>>> -python fetcher_hashes_dummyfunc() {
>>> -    return
>>> -}
>>> -fetcher_hashes_dummyfunc[vardepvalue] = "${@bb.fetch.get_hashvalue(d)}"
>>> +#python fetcher_hashes_dummyfunc() {
>>> +#    return
>>> +#}
>>> +#fetcher_hashes_dummyfunc[vardepvalue] = "${@bb.fetch.get_hashvalue(d)}"
>>>
>>>  addtask fetch
>>>  do_fetch[dirs] = "${DL_DIR}"
>>>  do_fetch[file-checksums] = "${@bb.fetch.get_checksum_file_list(d)}"
>>>  do_fetch[file-checksums] += " ${@get_lic_checksum_file_list(d)}"
>>> -do_fetch[prefuncs] += "fetcher_hashes_dummyfunc"
>>> +#do_fetch[prefuncs] += "fetcher_hashes_dummyfunc"
>>>  do_fetch[network] = "1"
>>>  python base_do_fetch() {
>>>
>>> Tony
>>>
>>> -----Original Message-----
>>> From: Gyorgy Sarvari <skandigraun@gmail.com>
>>> Sent: Thursday, 21 August 2025 09:30
>>> To: Tony Rex <tony.rex@ericsson.com>; Alexander Kanavin
>>> <alex.kanavin@gmail.com>
>>> Cc: Mathieu Dubois-Briand <mathieu.dubois-briand@bootlin.com>;
>>> openembedded-core@lists.openembedded.org
>>> Subject: Re: [OE-core] [PATCH 0/1] base.bbclass: Fix the recipe
>>> parsing race issue
>>>
>>> On 8/21/25 09:10, Tony Rex wrote:
>>>> Yes, I get the same or similar error if the patch is not applied.
>>>>
>>>> ERROR: ExpansionError during parsing .../generic-if.bb################################   | ETA:  0:00:00
>>>> bb.data_smart.ExpansionError: Failure expanding variable fetcher_hashes_dummyfunc[vardepvalue], expression was ${@bb.fetch.get_hashvalue(d)} which triggered exception FetchError: Fetcher failure: Unable to resolve '${TAGNAME}-1.0' in upstream git repository in git ls-remote output for gerrit...
>>>> The variable dependency chain for the failure is:
>>>> fetcher_hashes_dummyfunc[vardepvalue]
>>>>
>>>> ERROR: Parsing halted due to errors, see error messages above
>>>>
>>>> How does your mytest_get.bb look like? Since I have had problem with reproducing the error on in a common setup maybe it can be done with your bb file.
>>>>
>>>> Tony
>>> The failure I reproduced is not recipe dependent. As some diagnostic test, try this script a few times with your failing git repo (change the repo placeholder):
>>>
>>> for i in `seq 50`; do
>>>   git clone $GIT_REPO_URL target_folder
>>>   git -C ./target_folder ls-remote
>>>   if [ $? -ne 0 ]; then
>>>     echo ls-remote failed, iteration: $i
>>>     break
>>>   fi
>>>   rm -rf ./target_folder
>>> done
>>>
>>> If I haven't made a typo, the script should clone your repo, do an ls-remote inside it, delete the folder, and do the same again.
>>>
>>> If your problem is the same that I reproduced, then I would expect the above script to fail sometimes - if it does, then the problem is with the infrastructure (network, remote git server, etc... something is flaky).
>>>
>>> Just for the record: the script generates a small amount of successive requests which some network admins might consider flooding.
>>>
>>>
>>>
>>>> -----Original Message-----
>>>> From: Gyorgy Sarvari <skandigraun@gmail.com>
>>>> Sent: Wednesday, 20 August 2025 18:12
>>>> To: Tony Rex <tony.rex@ericsson.com>; Alexander Kanavin
>>>> <alex.kanavin@gmail.com>
>>>> Cc: Mathieu Dubois-Briand <mathieu.dubois-briand@bootlin.com>;
>>>> openembedded-core@lists.openembedded.org
>>>> Subject: Re: [OE-core] [PATCH 0/1] base.bbclass: Fix the recipe
>>>> parsing race issue
>>>>
>>>> [You don't often get email from skandigraun@gmail.com. Learn why
>>>> this is important at https://aka.ms/LearnAboutSenderIdentification
>>>> ]
>>>>
>>>> On 8/20/25 17:44, Gyorgy Sarvari via lists.openembedded.org wrote:
>>>>> On 8/20/25 13:53, Tony Rex wrote:
>>>>>> It was introduced by this commit:
>>>>>> https://git/
>>>>>> .yoctoproject.org%2Fpoky%2Fcommit%2F%3Fid%3Debfa1700f41b3411aec04
>>>>>> 01
>>>>>> 4
>>>>>> 4
>>>>>> 605166c35b8dd14&data=05%7C02%7Ctony.rex%40ericsson.com%7C7b5c099d
>>>>>> 17
>>>>>> 0
>>>>>> 0
>>>>>> 43080cca08dde004478d%7C92e84cebfbfd47abbe52080c6b87953f%7C0%7C0%7
>>>>>> C6
>>>>>> 3
>>>>>> 8
>>>>>> 913031194171335%7CUnknown%7CTWFpbGZsb3d8eyJFbXB0eU1hcGkiOnRydWUsI
>>>>>> lY
>>>>>> i
>>>>>> O
>>>>>> iIwLjAuMDAwMCIsIlAiOiJXaW4zMiIsIkFOIjoiTWFpbCIsIldUIjoyfQ%3D%3D%7
>>>>>> C0
>>>>>> %
>>>>>> 7
>>>>>> C%7C%7C&sdata=iYvLGK1DwJk78tEvmL%2FPOL2olj7iNL87eXqdsEraxLw%3D&re
>>>>>> se
>>>>>> r
>>>>>> v
>>>>>> ed=0
>>>>>>
>>>>>> Maybe "not allowed" is a strong wording, but that was how I interpreted it. Sorry, my bad if I misinterpreted it.
>>>>> In my understanding (which could be wrong), this is more to avoid
>>>>> working on a repository, where the exact work-revision can't be
>>>>> determined. But in your example, it can be determined, if your
>>>>> patch is not applied.
>>>>>
>>>>> Some extra ramblings, which may or may not be unadulterated
>>>>> garbage from
>>>>> me: Your initial commit message mentioned an expansion error. I
>>>>> looked into that a bit, and is it possible that the repository
>>>>> that is causing this issue has some access rate limit? The only
>>>>> thing that can fail sometimes in that code path seems to be "git ls-remote".
>>>>>
>>>>> Now, when I noticed this, I compiled a git version for myself that
>>>>> randomly throws an error with "git ls-remote" command, and when I
>>>>> used it with bitbake, I got this:
>>>>>
>>>>> ERROR: ExpansionError during parsing
>>>>> /home/meee/stuff/ptest-images/poky/meta/recipes-core/ell/mytest_get.
>>>>> b
>>>>> b
>>>>> bb.data_smart.ExpansionError: Failure expanding variable
>>>>> fetcher_hashes_dummyfunc[vardepvalue], expression was <etc etc
>>>>> etc>
>>>>>
>>>>> Is this the same error you got?
>>>> Sorry for the spam, one thing I forgot - the error on my end also contains details about the command that failed, pointing to git. If it's not the same as you get, do you have a copy of the full bitbake output you get when you see this error?
>>>>
>>>>>> Tony
>>>>>>
>>>>>> -----Original Message-----
>>>>>> From: Gyorgy Sarvari <skandigraun@gmail.com>
>>>>>> Sent: Wednesday, 20 August 2025 12:18
>>>>>> To: Tony Rex <tony.rex@ericsson.com>; Alexander Kanavin
>>>>>> <alex.kanavin@gmail.com>
>>>>>> Cc: Mathieu Dubois-Briand <mathieu.dubois-briand@bootlin.com>;
>>>>>> openembedded-core@lists.openembedded.org
>>>>>> Subject: Re: [OE-core] [PATCH 0/1] base.bbclass: Fix the recipe
>>>>>> parsing race issue
>>>>>>
>>>>>> [You don't often get email from skandigraun@gmail.com. Learn why
>>>>>> this is important at
>>>>>> https://aka.ms/LearnAboutSenderIdentification
>>>>>> ]
>>>>>>
>>>>>> On 8/19/25 23:10, Tony Rex via lists.openembedded.org wrote:
>>>>>>> Hi Alex and Mathieu!
>>>>>>>
>>>>>>> Ok, so the patch fixes a bug and then the floating tag/branch function is in place again.
>>>>>>> The floating tag/branch function is broken due to the bug and the patch fixes it.
>>>>>>>
>>>>>>> The mail from Mathieu below might not fully reflect the issue here, since it seems to create a recipe with the help of devtool.
>>>>>>>
>>>>>>> To verify:
>>>>>>> 1. add layers/oe-core/meta-selftest to conf/bblayers.conf 2. run
>>>>>>> bitbake gitrepotest -c fetch
>>>>>>> result:
>>>>>>> NOTE: Tasks Summary: Attempted 2 tasks of which 0 didn't need to be rerun and all succeeded.
>>>>>>> 3. run bitbake gitrepotest -c cleanall 4. edit
>>>>>>> ../layers/oe-core/meta-selftest/recipes-test/gitrepotest/gitrepo
>>>>>>> te
>>>>>>> s
>>>>>>> t
>>>>>>> .b
>>>>>>> b
>>>>>>> add tag=2.12 in SRC_URI
>>>>>>> SRC_URI =
>>>>>>> "git://git.yoctoproject.org/git/matchbox-panel-2;branch=master;t
>>>>>>> ag
>>>>>>> =
>>>>>>> 2
>>>>>>> .1 2;protocol=https \ and comment out SRCREV #SRCREV =
>>>>>>> "f82ca3f42510fb3ef10f598b393eb373a2c34ca7"
>>>>>>> 5. run bitbake gitrepotest -c fetch
>>>>>>> result:
>>>>>>> NOTE: Tasks Summary: Attempted 2 tasks of which 0 didn't need to be rerun and all succeeded.
>>>>>>> The fetch goes through although this is not allowed according to the floating tag/branch error statement if not d.getVar("__BBSRCREV_SEEN"): in git.py.
>>>>>> Could you point to a commit/email/etc about this "no floating
>>>>>> tags allowed" statement? AFAIK tags without specifying revision
>>>>>> is allowed and supported. Not recommended, due to mandatory
>>>>>> network access, and because git tags are not immutable, but I'm
>>>>>> surprised to hear that they are not supposed to work (which of
>>>>>> course can also mean that I just missed a memo)
>>>>>>
>>>>>>> 6. run bitbake gitrepotest -c cleanall 7. apply patch
>>>>>>> 0001-base.bbclass-Fix-the-recipe-parsing-race-issue.patch
>>>>>>> 8. run bitbake gitrepotest -c fetch
>>>>>>> result:
>>>>>>> ERROR: gitrepotest-1.0-r0 do_fetch: Bitbake Fetcher Error:
>>>>>>> FetchError("Recipe uses a floating tag/branch '2.12' for repo
>>>>>>> 'git.yoctoproject.org/git/matchbox-panel-2' without a fixed
>>>>>>> SRCREV yet doesn't call bb.fetch2.get_srcrev() (use SRCPV in PV
>>>>>>> for OE).",
>>>>>>> None)
>>>>>>> ERROR: Logfile of failure stored in:
>>>>>>> .../build/tmp-glibc/work/cortexa53-wrs-linux/gitrepotest/1.0/temp/log.
>>>>>>> do_fetch.1254999
>>>>>>> ERROR: Task (.../layers/oe-core/meta-selftest/recipes-test/gitrepotest/gitrepotest.bb:do_fetch) failed with exit code '1'
>>>>>>> NOTE: Tasks Summary: Attempted 2 tasks of which 0 didn't need to be rerun and 1 failed.
>>>>>>>
>>>>>>> Error is the expected result when only having a tag set.
>>>>>>>
>>>>>>> Tony
>>>>>>>
>>>>>>> -----Original Message-----
>>>>>>> From: Alexander Kanavin <alex.kanavin@gmail.com>
>>>>>>> Sent: Tuesday, 19 August 2025 10:53
>>>>>>> To: Tony Rex <tony.rex@ericsson.com>
>>>>>>> Cc: Mathieu Dubois-Briand <mathieu.dubois-briand@bootlin.com>;
>>>>>>> openembedded-core@lists.openembedded.org
>>>>>>> Subject: Re: [OE-core] [PATCH 0/1] base.bbclass: Fix the recipe
>>>>>>> parsing race issue
>>>>>>>
>>>>>>> [You don't often get email from alex.kanavin@gmail.com. Learn
>>>>>>> why this is important at
>>>>>>> https://aka.ms/LearnAboutSenderIdentification
>>>>>>> ]
>>>>>>>
>>>>>>> You need to make things a little bit easier for maintainers and try to save their time.
>>>>>>>
>>>>>>> I would suggest that you provide clear steps to observe the issue.
>>>>>>> Attach the recipe file that would demonstrate the issue, and provide clear steps to trigger it. What is confusing right now is that you say here's a recipe that builds ok, but then you 'apply the patch' and then there's some kind of failure. So which patch needs to be applied?
>>>>>>>
>>>>>>> Alex
>>>>>>>
>>>>>>> On Fri, 15 Aug 2025 at 17:29, Tony Rex via lists.openembedded.org <tony.rex=ericsson.com@lists.openembedded.org> wrote:
>>>>>>>> Hi Mathieu!
>>>>>>>>
>>>>>>>> I think that this is how it is intended to work, bitbake does not allow floating tag/branch anymore.
>>>>>>>> Please correct me if I'm wrong @Richard Purdie (who implemented
>>>>>>>> this for security reasons I think.)
>>>>>>>>
>>>>>>>> The patch that I have makes sure that the parsing and setting
>>>>>>>> of the variables in the recipe are done before executing the do_fetch command, and as a side effect the floating tag/branch protection is working as intended again.
>>>>>>>>
>>>>>>>> How does the recipe look like?
>>>>>>>> devtool add kernel-module-hello-world https://git/.
>>>>>>>> yoctoproject.org%2Fgit%2Fkernel-module-hello-world&data=05%7C02
>>>>>>>> %7
>>>>>>>> C
>>>>>>>> t
>>>>>>>> on
>>>>>>>> y
>>>>>>>> .rex%40ericsson.com%7Cde1ae2aceeda470b381608dddefdd730%7C92e84c
>>>>>>>> eb
>>>>>>>> f
>>>>>>>> b
>>>>>>>> fd
>>>>>>>> 4
>>>>>>>> 7abbe52080c6b87953f%7C0%7C0%7C638911904005151487%7CUnknown%7CTW
>>>>>>>> Fp
>>>>>>>> b
>>>>>>>> G
>>>>>>>> Zs
>>>>>>>> b
>>>>>>>> 3d8eyJFbXB0eU1hcGkiOnRydWUsIlYiOiIwLjAuMDAwMCIsIlAiOiJXaW4zMiIs
>>>>>>>> Ik
>>>>>>>> F
>>>>>>>> O
>>>>>>>> Ij
>>>>>>>> o
>>>>>>>> iTWFpbCIsIldUIjoyfQ%3D%3D%7C0%7C%7C%7C&sdata=jyCicRQT%2FLtb1Yxi
>>>>>>>> FH
>>>>>>>> K
>>>>>>>> g
>>>>>>>> w5
>>>>>>>> B
>>>>>>>> nNktN4t9IciiyxtsEwO4%3D&reserved=0
>>>>>>>> The one I get looks like this:
>>>>>>>>
>>>>>>>> cat
>>>>>>>> build/workspace/recipes/kernel-module-hello-world/kernel-module
>>>>>>>> -h
>>>>>>>> e
>>>>>>>> l
>>>>>>>> lo
>>>>>>>> -
>>>>>>>> world_git.bb
>>>>>>>> # Recipe created by recipetool
>>>>>>>> # This is the basis of a recipe and may need further editing in order to be fully functional.
>>>>>>>> # (Feel free to remove these comments when editing.)
>>>>>>>>
>>>>>>>> # WARNING: the following LICENSE and LIC_FILES_CHKSUM values
>>>>>>>> are best guesses - it is # your responsibility to verify that the values are complete and correct.
>>>>>>>> #
>>>>>>>> # The following license files were not able to be identified
>>>>>>>> and are # represented as "Unknown" below, you will need to check them yourself:
>>>>>>>> #   LICENSE
>>>>>>>> LICENSE = "Unknown"
>>>>>>>> LIC_FILES_CHKSUM = "file://LICENSE;md5=7b785be6433a2e4af6a641cc7f0330d1"
>>>>>>>>
>>>>>>>> SRC_URI = "git://git.yoctoproject.org/git/kernel-module-hello-world;protocol=https;branch=master"
>>>>>>>>
>>>>>>>> # Modify these as desired
>>>>>>>> PV = "1.0+git"
>>>>>>>> SRCREV = "b66f0f2bd559a1df81fc1bda240eba07a2ecdd2b"
>>>>>>>>
>>>>>>>> S = "${WORKDIR}/git"
>>>>>>>>
>>>>>>>> inherit module
>>>>>>>>
>>>>>>>> EXTRA_OEMAKE:append:task-install = " -C ${STAGING_KERNEL_DIR} M=${S}"
>>>>>>>> EXTRA_OEMAKE += "KERNEL_DIR=${STAGING_KERNEL_DIR}"
>>>>>>>>
>>>>>>>> And it contains SRCREV = "b66f0f2bd559a1df81fc1bda240eba07a2ecdd2b"
>>>>>>>>
>>>>>>>> Tony
>>>>>>>>
>>>>>>>> -----Original Message-----
>>>>>>>> From: Mathieu Dubois-Briand <mathieu.dubois-briand@bootlin.com>
>>>>>>>> Sent: Friday, 15 August 2025 11:26
>>>>>>>> To: Tony Rex <tony.rex@ericsson.com>;
>>>>>>>> openembedded-core@lists.openembedded.org
>>>>>>>> Subject: Re: [OE-core] [PATCH 0/1] base.bbclass: Fix the recipe
>>>>>>>> parsing race issue
>>>>>>>>
>>>>>>>> [You don't often get email from mathieu.dubois-briand@bootlin.com.
>>>>>>>> Learn why this is important at
>>>>>>>> https://aka.ms/LearnAboutSenderIdentification ]
>>>>>>>>
>>>>>>>> On Thu Aug 14, 2025 at 7:23 PM CEST, Tony Rex via lists.openembedded.org wrote:
>>>>>>>>> It seems that the race condition happens when doing do_fetch
>>>>>>>>> on really small repos and it can be difficult to reproduce so
>>>>>>>>> below the other case with floating tag/branch that is easier to verify.
>>>>>>>>>
>>>>>>>>> To verify the floating tag/branch issue have s simple recipe
>>>>>>>>> hello_1.0.bb and set the SRC_URI to SRC_URI = "git://<repository>;protocol=https;user=${USER};branch=master;tag=v1.0"
>>>>>>>>>
>>>>>>>>> run command bitbake hello everything builds.
>>>>>>>>>
>>>>>>>>> Apply the patch
>>>>>>>>>
>>>>>>>>> run command bitbake hello
>>>>>>>>> ERROR: hello-1.0-r1 do_fetch: Bitbake Fetcher Error:
>>>>>>>>> FetchError("Recipe uses a floating tag/branch 'v1.0' for repo
>>>>>>>>> '<repository>' without a fixed SRCREV yet doesn't call
>>>>>>>>> bb.fetch2.get_srcrev() (use SRCPV in PV for OE).", None)
>>>>>>>>> ERROR: Logfile of failure stored in:
>>>>>>>>> hello/1.0/temp/log.do_fetch.3289932
>>>>>>>>> ERROR: Task (hello/hello_1.0.bb:do_fetch) failed with exit code '1'
>>>>>>>>>
>>>>>>>>> Tony Rex (1):
>>>>>>>>>   base.bbclass: Fix the recipe parsing race issue
>>>>>>>>>
>>>>>>>>>  meta/classes-global/base.bbclass | 2 +-
>>>>>>>>>  1 file changed, 1 insertion(+), 1 deletion(-)
>>>>>>>> Hi Tony,
>>>>>>>>
>>>>>>>> Thanks for your patch.
>>>>>>>>
>>>>>>>> It looks like this is breaking some builds with the following error:
>>>>>>>>
>>>>>>>> oeqa.utils.subprocesstweak.OETestCalledProcessError: Command '. /srv/pokybuild/yocto-worker/qemux86-64/build/build/tmp/work/qemux86_64-poky-linux/core-image-sato/1.0/testsdkext/environment-setup-core2-64-poky-linux > /dev/null; devtool add kernel-module-hello-world https://git.yoctoproject.org/git/kernel-module-hello-world;' returned non-zero exit status 1.
>>>>>>>> ...
>>>>>>>> ERROR: Bitbake Fetcher Error: FetchError("Recipe uses a
>>>>>>>> floating tag/branch 'master' for repo
>>>>>>>> 'git.yoctoproject.org/git/kernel-module-hello-world' without a
>>>>>>>> fixed SRCREV yet doesn't call bb.fetch2.get_srcrev() (use SRCPV
>>>>>>>> in PV for OE).", None)
>>>>>>>>
>>>>>>>> https://auto/
>>>>>>>> builder.yoctoproject.org%2Fvalkyrie%2F%23%2Fbuilders%2F68%2Fbui
>>>>>>>> ld
>>>>>>>> s
>>>>>>>> %
>>>>>>>> 2F
>>>>>>>> 2
>>>>>>>> 284&data=05%7C02%7Ctony.rex%40ericsson.com%7Cde1ae2aceeda470b38
>>>>>>>> 16
>>>>>>>> 0
>>>>>>>> 8
>>>>>>>> dd
>>>>>>>> d
>>>>>>>> efdd730%7C92e84cebfbfd47abbe52080c6b87953f%7C0%7C0%7C6389119040
>>>>>>>> 05
>>>>>>>> 2
>>>>>>>> 0
>>>>>>>> 90
>>>>>>>> 0
>>>>>>>> 3%7CUnknown%7CTWFpbGZsb3d8eyJFbXB0eU1hcGkiOnRydWUsIlYiOiIwLjAuM
>>>>>>>> DA
>>>>>>>> w
>>>>>>>> M
>>>>>>>> CI
>>>>>>>> s
>>>>>>>> IlAiOiJXaW4zMiIsIkFOIjoiTWFpbCIsIldUIjoyfQ%3D%3D%7C0%7C%7C%7C&s
>>>>>>>> da
>>>>>>>> t
>>>>>>>> a
>>>>>>>> =m
>>>>>>>> P
>>>>>>>> UXMdb%2FYOyGIm0rnncaWgcqtwkUIh8364IP3eYDPe4%3D&reserved=0
>>>>>>>> https://auto/
>>>>>>>> builder.yoctoproject.org%2Fvalkyrie%2F%23%2Fbuilders%2F30%2Fbui
>>>>>>>> ld
>>>>>>>> s
>>>>>>>> %
>>>>>>>> 2F
>>>>>>>> 2
>>>>>>>> 199&data=05%7C02%7Ctony.rex%40ericsson.com%7Cde1ae2aceeda470b38
>>>>>>>> 16
>>>>>>>> 0
>>>>>>>> 8
>>>>>>>> dd
>>>>>>>> d
>>>>>>>> efdd730%7C92e84cebfbfd47abbe52080c6b87953f%7C0%7C0%7C6389119040
>>>>>>>> 05
>>>>>>>> 2
>>>>>>>> 2
>>>>>>>> 74
>>>>>>>> 3
>>>>>>>> 2%7CUnknown%7CTWFpbGZsb3d8eyJFbXB0eU1hcGkiOnRydWUsIlYiOiIwLjAuM
>>>>>>>> DA
>>>>>>>> w
>>>>>>>> M
>>>>>>>> CI
>>>>>>>> s
>>>>>>>> IlAiOiJXaW4zMiIsIkFOIjoiTWFpbCIsIldUIjoyfQ%3D%3D%7C0%7C%7C%7C&s
>>>>>>>> da
>>>>>>>> t
>>>>>>>> a
>>>>>>>> =R
>>>>>>>> 0
>>>>>>>> HFameGl2LmIVMT9ml%2BDMnj%2FMbG6SW0%2F3yNZHQu2js%3D&reserved=0
>>>>>>>> https://auto/
>>>>>>>> builder.yoctoproject.org%2Fvalkyrie%2F%23%2Fbuilders%2F16%2Fbui
>>>>>>>> ld
>>>>>>>> s
>>>>>>>> %
>>>>>>>> 2F
>>>>>>>> 2
>>>>>>>> 247&data=05%7C02%7Ctony.rex%40ericsson.com%7Cde1ae2aceeda470b38
>>>>>>>> 16
>>>>>>>> 0
>>>>>>>> 8
>>>>>>>> dd
>>>>>>>> d
>>>>>>>> efdd730%7C92e84cebfbfd47abbe52080c6b87953f%7C0%7C0%7C6389119040
>>>>>>>> 05
>>>>>>>> 2
>>>>>>>> 4
>>>>>>>> 53
>>>>>>>> 8
>>>>>>>> 7%7CUnknown%7CTWFpbGZsb3d8eyJFbXB0eU1hcGkiOnRydWUsIlYiOiIwLjAuM
>>>>>>>> DA
>>>>>>>> w
>>>>>>>> M
>>>>>>>> CI
>>>>>>>> s
>>>>>>>> IlAiOiJXaW4zMiIsIkFOIjoiTWFpbCIsIldUIjoyfQ%3D%3D%7C0%7C%7C%7C&s
>>>>>>>> da
>>>>>>>> t
>>>>>>>> a
>>>>>>>> =N
>>>>>>>> S
>>>>>>>> FhEupwMkgqUAhpRgFbKnyFl%2FsY%2B4UJMhVQHhqOaNM%3D&reserved=0
>>>>>>>>
>>>>>>>> Can you have a look at this error please?
>>>>>>>>
>>>>>>>> Best regards,
>>>>>>>> Mathieu
>>>>>>>>
>>>>>>>> --
>>>>>>>> Mathieu Dubois-Briand, Bootlin
>>>>>>>> Embedded Linux and Kernel engineering https://boot/
>>>>>>>> lin.com%2F&data=05%7C02%7Ctony.rex%40ericsson.com%7Cde1ae2aceed
>>>>>>>> a4
>>>>>>>> 7
>>>>>>>> 0
>>>>>>>> b3
>>>>>>>> 8
>>>>>>>> 1608dddefdd730%7C92e84cebfbfd47abbe52080c6b87953f%7C0%7C0%7C638
>>>>>>>> 91
>>>>>>>> 1
>>>>>>>> 9
>>>>>>>> 04
>>>>>>>> 0
>>>>>>>> 05263327%7CUnknown%7CTWFpbGZsb3d8eyJFbXB0eU1hcGkiOnRydWUsIlYiOi
>>>>>>>> Iw
>>>>>>>> L
>>>>>>>> j
>>>>>>>> Au
>>>>>>>> M
>>>>>>>> DAwMCIsIlAiOiJXaW4zMiIsIkFOIjoiTWFpbCIsIldUIjoyfQ%3D%3D%7C0%7C%
>>>>>>>> 7C
>>>>>>>> %
>>>>>>>> 7
>>>>>>>> C&
>>>>>>>> s
>>>>>>>> data=oyi7YWsXDVHn7QB4fDdG1sBp7gQsfGWcJTWtX1CjAw4%3D&reserved=0
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>> 
>>>>>

Re: [OE-core] [PATCH 0/1] base.bbclass: Fix the recipe parsing race issue

[Tony Rex]

Hi Richard!

Sorry to bother you but can you please have a look at this. There are two things that I have been observing regarding parsing of recipes and
the git ls-remote fetch sequence, it seems to be some unexpected behavior after the introduction of base/package: Move source revision information from PV to PKGV:
https://patchwork.yoctoproject.org/project/oe-core/patch/20230811130448.29263-1-richard.purdie@linuxfoundation.org/

First if the gitrepotest.bb is changed

git diff
diff --git a/meta-selftest/recipes-test/gitrepotest/gitrepotest.bb b/meta-selftest/recipes-test/gitrepotest/gitrepotest.bb
index f1b6c55833..258c6f0557 100644
--- a/meta-selftest/recipes-test/gitrepotest/gitrepotest.bb
+++ b/meta-selftest/recipes-test/gitrepotest/gitrepotest.bb
@@ -7,10 +7,10 @@ INHIBIT_DEFAULT_DEPS = "1"

 PATCHTOOL="git"

-SRC_URI = "git://git.yoctoproject.org/git/matchbox-panel-2;branch=master;protocol=https \
+SRC_URI = "git://git.yoctoproject.org/git/matchbox-panel-2;branch=master;tag=2.12;protocol=https \
            file://0001-testpatch.patch \
           "

-SRCREV = "f82ca3f42510fb3ef10f598b393eb373a2c34ca7"
+#SRCREV = "f82ca3f42510fb3ef10f598b393eb373a2c34ca7"

 S = "${WORKDIR}/git"

run bitbake gitrepotest, it works and builds although there is a floating branch/tag

Change the base.bbclass like it is done when applying the patch 0001-base.bbclass-Fix-the-recipe-parsing-race-issue.patch:

Signed-off-by: Tony Rex <tony.rex@ericsson.com>
---
 meta/classes-global/base.bbclass | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/meta/classes-global/base.bbclass b/meta/classes-global/base.bbclass
index 6be1f5c2df..c955bb861f 100644
--- a/meta/classes-global/base.bbclass
+++ b/meta/classes-global/base.bbclass
@@ -147,7 +147,7 @@ def setup_hosttools_dir(dest, toolsvar, d, fatal=True):
 python fetcher_hashes_dummyfunc() {
     return
 }
-fetcher_hashes_dummyfunc[vardepvalue] = "${@bb.fetch.get_hashvalue(d)}"
+fetcher_hashes_dummyfunc[vardepvalue] := "${@bb.fetch.get_hashvalue(d)}"

 addtask fetch
 do_fetch[dirs] = "${DL_DIR}"

Clear all sstate and run bitbake gitrepotest again and the following build error is triggered:

ERROR: gitrepotest-1.0-r0 do_fetch: Bitbake Fetcher Error: FetchError("Recipe uses a floating tag/branch '2.12' for repo 'git.yoctoproject.org/git/matchbox-panel-2' without a fixed SRCREV yet doesn't call bb.fetch2.get_srcrev() (use SRCPV in PV for OE).", None)
ERROR: Logfile of failure stored in: .../build/tmp/work/cortexa53-wrs-linux/gitrepotest/1.0/temp/log.do_fetch.1802224
ERROR: Task (.../layers/oe-core/meta-selftest/recipes-test/gitrepotest/gitrepotest.bb:do_fetch) failed with exit code '1'

For the second observation revert the patch 0001-base.bbclass-Fix-the-recipe-parsing-race-issue.patch and change gitrepotest.bb:

git diff
diff --git a/meta-selftest/recipes-test/gitrepotest/gitrepotest.bb b/meta-selftest/recipes-test/gitrepotest/gitrepotest.bb
index f1b6c55833..a5c67121ab 100644
--- a/meta-selftest/recipes-test/gitrepotest/gitrepotest.bb
+++ b/meta-selftest/recipes-test/gitrepotest/gitrepotest.bb
@@ -7,10 +7,15 @@ INHIBIT_DEFAULT_DEPS = "1"

 PATCHTOOL="git"

-SRC_URI = "git://git.yoctoproject.org/git/matchbox-panel-2;branch=master;protocol=https \
+TAGNR:pn-gitrepotest = "2"
+PV:pn-gitrepotest = "12"
+TAG = "${TAGNR}.${PV}"
+
+SRC_URI = "git://git.yoctoproject.org/git/matchbox-panel-2;branch=master;tag=${TAG};protocol=https \
            file://0001-testpatch.patch \
           "

-SRCREV = "f82ca3f42510fb3ef10f598b393eb373a2c34ca7"
+#SRCREV = "f82ca3f42510fb3ef10f598b393eb373a2c34ca7"

 S = "${WORKDIR}/git"
+BBCLASSEXTEND="nativesdk"

Clear all sstate and run bitbake gitrepotest again and the following build error is triggered:

ERROR: ExpansionError during parsing .../oe-core/meta-selftest/recipes-test/gitrepotest/gitrepotest.bb
bb.data_smart.ExpansionError: Failure expanding variable fetcher_hashes_dummyfunc[vardepvalue], expression was ${@bb.fetch.get_hashvalue(d)} which triggered exception FetchError: Fetcher failure: Unable to resolve '${TAGNR}.1.0' in upstream git repository in git ls-remote output for git.yoctoproject.org/git/matchbox-panel-2
The variable dependency chain for the failure is: fetcher_hashes_dummyfunc[vardepvalue]

ERROR: Parsing halted due to errors, see error messages above

Change the base.bbclass by applying the patch 0001-base.bbclass-Fix-the-recipe-parsing-race-issue.patch
Clear all sstate and run bitbake gitrepotest again and the following build error is triggered:

ERROR: gitrepotest-12-r0 do_fetch: Bitbake Fetcher Error: FetchError("Recipe uses a floating tag/branch '2.12' for repo 'git.yoctoproject.org/git/matchbox-panel-2' without a fixed SRCREV yet doesn't call bb.fetch2.get_srcrev() (use SRCPV in PV for OE).", None)
ERROR: Logfile of failure stored in: .../build/tmp/work/cortexa53-wrs-linux/gitrepotest/12/temp/log.do_fetch.2058688
ERROR: Task (.../oe-core/meta-selftest/recipes-test/gitrepotest/gitrepotest.bb:do_fetch) failed with exit code '1'

Fyi: Another observation and workaround for the parsing problem with BBCLASSEXTEND="nativesdk" that I have tested without applying the patch and that also seems to get
around the parsing  issue is to set the SRC_URI with :=  then the floating tag/branch error will not take effect either.

Tony

-----Original Message-----
From: Tony Rex
Sent: Tuesday, 26 August 2025 08:34
To: Gyorgy Sarvari <skandigraun@gmail.com>; Alexander Kanavin <alex.kanavin@gmail.com>
Cc: Mathieu Dubois-Briand <mathieu.dubois-briand@bootlin.com>; openembedded-core@lists.openembedded.org
Subject: RE: [OE-core] [PATCH 0/1] base.bbclass: Fix the recipe parsing race issue

Yes, I think you're right, and that this might not be a bug, but a change in the behavior that is not seen in previous releases such as hardknott.
So the question is if this changed behavior is wanted/intended or a side effect, like you stated.
In earlier releases it seems that the PV and variables like TAGNAME was set before the override pn-nativesdk-gitrepotest was used/needed in the parsing. However if the setting is done earlier like with the patch  fetcher_hashes_dummyfunc[vardepvalue] := "${@bb.fetch.get_hashvalue(d)}"
or SRC_URI := or ?= the pn-nativesdk-gitrepotest seems to pick up the value from pn-gitrepotest without complaints.

And I also think that the change in this area breaks the floating branch/tag "requirement" in git.py, if not d.getVar("__BBSRCREV_SEEN"):

Tony

-----Original Message-----
From: Gyorgy Sarvari <skandigraun@gmail.com>
Sent: Monday, 25 August 2025 22:14
To: Tony Rex <tony.rex@ericsson.com>; Alexander Kanavin <alex.kanavin@gmail.com>
Cc: Mathieu Dubois-Briand <mathieu.dubois-briand@bootlin.com>; openembedded-core@lists.openembedded.org
Subject: Re: [OE-core] [PATCH 0/1] base.bbclass: Fix the recipe parsing race issue

On 8/25/25 20:31, Gyorgy Sarvari wrote:
> On 8/25/25 17:49, Tony Rex wrote:
>> It is the full error output that is in the mail thread, there is nothing more I think.
>>
>> But can you please try to reproduce the error by doing the changes in meta-selftest layer described below and evaluate the outcome.
>> I think the key is the +BBCLASSEXTEND="nativesdk" line.
> I can confirm that I can reproduce it. I don't think that your
> proposed patch is the correct solution, but your bug looks real.

Looking at it it, I'm not 100% sure if it's a bug, but rather maybe a bit surprising behavior, by design?  I guess other people will share their opinion on this in the coming days - this a personal opinion after running it in a debugger. Your finding about the BBCLASSEXTEND seems to corroborate this too.

When you set an override with TAGNAME:pn-gitrepotest, that will be valid only for the gitrepotest package. But when you enable nativesdk, it will also check the nativesdk-gitrepotest package, for which this variable isn't defined - there is no default value, and the pn-girepotest override doesn't match. If you also define TAGNAME:pn-nativesdk-gitrepotest, then it is defined, and it seems to be resolved correctly.


>
>> git diff
>> diff --git a/meta-selftest/conf/layer.conf
>> b/meta-selftest/conf/layer.conf index 48ca8464f9..3d83072fb0 100644
>> --- a/meta-selftest/conf/layer.conf
>> +++ b/meta-selftest/conf/layer.conf
>> @@ -12,3 +12,6 @@ BBFILE_PRIORITY_selftest = "5"
>>  addpylib ${LAYERDIR}/lib oeqa
>>
>>  LAYERSERIES_COMPAT_selftest = "scarthgap"
>> +
>> +TAGNAME:pn-gitrepotest="2"
>> +PV:pn-gitrepotest="12"
>> diff --git a/meta-selftest/recipes-test/gitrepotest/gitrepotest.bb
>> b/meta-selftest/recipes-test/gitrepotest/gitrepotest.bb
>> index f1b6c55833..10943584df 100644
>> --- a/meta-selftest/recipes-test/gitrepotest/gitrepotest.bb
>> +++ b/meta-selftest/recipes-test/gitrepotest/gitrepotest.bb
>> @@ -7,10 +7,12 @@ INHIBIT_DEFAULT_DEPS = "1"
>>
>>  PATCHTOOL="git"
>>
>> -SRC_URI =
>> "git://git.yoctoproject.org/git/matchbox-panel-2;branch=master;protoc
>> ol=https \
>> +TAG="${TAGNAME}.${PV}"
>> +SRC_URI =
>> +"git://git.yoctoproject.org/git/matchbox-panel-2;branch=master;proto
>> +col=https;tag=${TAG} \
>>             file://0001-testpatch.patch \
>>            "
>>
>> -SRCREV = "f82ca3f42510fb3ef10f598b393eb373a2c34ca7"
>> +#SRCREV = "f82ca3f42510fb3ef10f598b393eb373a2c34ca7"
>>
>>  S = "${WORKDIR}/git"
>> +BBCLASSEXTEND="nativesdk"
>>
>>
>> You should get something like this:
>> ERROR: ExpansionError during parsing .../oe-core/meta-selftest/recipes-test/gitrepotest/gitrepotest.bb   | ETA:  0:00:35
>> bb.data_smart.ExpansionError: Failure expanding variable
>> fetcher_hashes_dummyfunc[vardepvalue], expression was
>> ${@bb.fetch.get_hashvalue(d)} which triggered exception FetchError:
>> Fetcher failure: Unable to resolve '${TAGNAME}.1.0' in upstream git
>> repository in git ls-remote output for
>> git.yoctoproject.org/git/matchbox-panel-2
>> The variable dependency chain for the failure is:
>> fetcher_hashes_dummyfunc[vardepvalue]
>>
>> ERROR: Parsing halted due to errors, see error messages above
>>
>>
>> Tony
>>
>> -----Original Message-----
>> From: Gyorgy Sarvari <skandigraun@gmail.com>
>> Sent: Friday, 22 August 2025 16:10
>> To: Tony Rex <tony.rex@ericsson.com>; Alexander Kanavin
>> <alex.kanavin@gmail.com>
>> Cc: Mathieu Dubois-Briand <mathieu.dubois-briand@bootlin.com>;
>> openembedded-core@lists.openembedded.org
>> Subject: Re: [OE-core] [PATCH 0/1] base.bbclass: Fix the recipe
>> parsing race issue
>>
>> On 8/22/25 13:26, Tony Rex wrote:
>>> So to me it looks like the fetcher_hashes_dummyfunc makes it possible to catch any faulty or unwanted setups in the parsing stage.
>>>
>>> It seems hard to reproduce and expose this error on a common target, but I can reproduce it easily in the local setup.
>>> So I share my observations instead and hopefully you can make some sense out them.
>>>
>>> The error that I'm seeing is that when fetcher_hashes_dummyfunc[vardepvalue] = "${@bb.fetch.get_hashvalue(d)}" is executed the PV is not set, nor is the PN.
>>> So it seems that if the git ls-remote check happens to quick and early all variables are not set properly.
>>> The repo in this case is small (140K) and I believe that the access
>>> to the git server is fast (don't know of it matters) the git ls-remote check does not contain the right information if for example, the tag in the SRC_URI contains the PV value.
>>>
>>> When doing printouts from get_hashvalue the revs variable is empty
>>> and PN = defaultpkgname and PV = 1.0 has default values so the variables doesn't seem to be properly set yet and gives this error:
>>>
>>> ERROR: ExpansionError during parsing .../generic-if.bb################################   | ETA:  0:00:00
>>> bb.data_smart.ExpansionError: Failure expanding variable fetcher_hashes_dummyfunc[vardepvalue], expression was ${@bb.fetch.get_hashvalue(d)} which triggered exception FetchError: Fetcher failure: Unable to resolve '${TAGNAME}-1.0' in upstream git repository in git ls-remote output for gerrit...
>>> The variable dependency chain for the failure is:
>>> fetcher_hashes_dummyfunc[vardepvalue]
>>>
>>> ERROR: Parsing halted due to errors, see error messages above
>>>
>>> The patch that is provided fixes this issue, but there might be other and/or maybe better solutions to this matter.
>>> Another workaround for this problem that I have tested and that also
>>> seems to get around this issue is to set the SRC_URI with := or ?=
>>>
>>> This is why I consider it to be a race condition during parsing and the git ls-remote check.
>> Could you please share the _full_ error output, and the recipe that is causing it? Feel free to change confidential things to placeholders, but the more parts would be available, the better it would be.
>>
>>
>>> Tony
>>>
>>> -----Original Message-----
>>> From: Tony Rex
>>> Sent: Thursday, 21 August 2025 17:48
>>> To: Gyorgy Sarvari <skandigraun@gmail.com>; Alexander Kanavin
>>> <alex.kanavin@gmail.com>
>>> Cc: Mathieu Dubois-Briand <mathieu.dubois-briand@bootlin.com>;
>>> openembedded-core@lists.openembedded.org
>>> Subject: RE: [OE-core] [PATCH 0/1] base.bbclass: Fix the recipe
>>> parsing race issue
>>>
>>> I'm not sure how to run your (Gyorgy) code inside a recipe to test it, but in a bash script it works, of course.
>>>
>>> The thing that I'm looking at now is  a bit confusing, when commenting out the fetcher_hashes_dummyfunc function from the base.bbclass it seems to work properly.
>>>
>>> I don't know what the real purpose of this function is, am I missing something here, like a certain setup or a corner case?
>>> It seems to work both on whinlatter and scarthgap. Can someone please help out and clarify.
>>>
>>> git diff classes-global/base.bbclass diff --git
>>> a/meta/classes-global/base.bbclass
>>> b/meta/classes-global/base.bbclass
>>> index 6c08870571..83f82b2a4d 100644
>>> --- a/meta/classes-global/base.bbclass
>>> +++ b/meta/classes-global/base.bbclass
>>> @@ -128,16 +128,16 @@ def setup_hosttools_dir(dest, toolsvar, d, fatal=True):
>>>
>>>  # We can't use vardepvalue against do_fetch directly since that would overwrite  # the other task dependencies so we use an indirect function.
>>> -python fetcher_hashes_dummyfunc() {
>>> -    return
>>> -}
>>> -fetcher_hashes_dummyfunc[vardepvalue] = "${@bb.fetch.get_hashvalue(d)}"
>>> +#python fetcher_hashes_dummyfunc() {
>>> +#    return
>>> +#}
>>> +#fetcher_hashes_dummyfunc[vardepvalue] = "${@bb.fetch.get_hashvalue(d)}"
>>>
>>>  addtask fetch
>>>  do_fetch[dirs] = "${DL_DIR}"
>>>  do_fetch[file-checksums] = "${@bb.fetch.get_checksum_file_list(d)}"
>>>  do_fetch[file-checksums] += " ${@get_lic_checksum_file_list(d)}"
>>> -do_fetch[prefuncs] += "fetcher_hashes_dummyfunc"
>>> +#do_fetch[prefuncs] += "fetcher_hashes_dummyfunc"
>>>  do_fetch[network] = "1"
>>>  python base_do_fetch() {
>>>
>>> Tony
>>>
>>> -----Original Message-----
>>> From: Gyorgy Sarvari <skandigraun@gmail.com>
>>> Sent: Thursday, 21 August 2025 09:30
>>> To: Tony Rex <tony.rex@ericsson.com>; Alexander Kanavin
>>> <alex.kanavin@gmail.com>
>>> Cc: Mathieu Dubois-Briand <mathieu.dubois-briand@bootlin.com>;
>>> openembedded-core@lists.openembedded.org
>>> Subject: Re: [OE-core] [PATCH 0/1] base.bbclass: Fix the recipe
>>> parsing race issue
>>>
>>> On 8/21/25 09:10, Tony Rex wrote:
>>>> Yes, I get the same or similar error if the patch is not applied.
>>>>
>>>> ERROR: ExpansionError during parsing .../generic-if.bb################################   | ETA:  0:00:00
>>>> bb.data_smart.ExpansionError: Failure expanding variable fetcher_hashes_dummyfunc[vardepvalue], expression was ${@bb.fetch.get_hashvalue(d)} which triggered exception FetchError: Fetcher failure: Unable to resolve '${TAGNAME}-1.0' in upstream git repository in git ls-remote output for gerrit...
>>>> The variable dependency chain for the failure is:
>>>> fetcher_hashes_dummyfunc[vardepvalue]
>>>>
>>>> ERROR: Parsing halted due to errors, see error messages above
>>>>
>>>> How does your mytest_get.bb look like? Since I have had problem with reproducing the error on in a common setup maybe it can be done with your bb file.
>>>>
>>>> Tony
>>> The failure I reproduced is not recipe dependent. As some diagnostic test, try this script a few times with your failing git repo (change the repo placeholder):
>>>
>>> for i in `seq 50`; do
>>>   git clone $GIT_REPO_URL target_folder
>>>   git -C ./target_folder ls-remote
>>>   if [ $? -ne 0 ]; then
>>>     echo ls-remote failed, iteration: $i
>>>     break
>>>   fi
>>>   rm -rf ./target_folder
>>> done
>>>
>>> If I haven't made a typo, the script should clone your repo, do an ls-remote inside it, delete the folder, and do the same again.
>>>
>>> If your problem is the same that I reproduced, then I would expect the above script to fail sometimes - if it does, then the problem is with the infrastructure (network, remote git server, etc... something is flaky).
>>>
>>> Just for the record: the script generates a small amount of successive requests which some network admins might consider flooding.
>>>
>>>
>>>
>>>> -----Original Message-----
>>>> From: Gyorgy Sarvari <skandigraun@gmail.com>
>>>> Sent: Wednesday, 20 August 2025 18:12
>>>> To: Tony Rex <tony.rex@ericsson.com>; Alexander Kanavin
>>>> <alex.kanavin@gmail.com>
>>>> Cc: Mathieu Dubois-Briand <mathieu.dubois-briand@bootlin.com>;
>>>> openembedded-core@lists.openembedded.org
>>>> Subject: Re: [OE-core] [PATCH 0/1] base.bbclass: Fix the recipe
>>>> parsing race issue
>>>>
>>>> [You don't often get email from skandigraun@gmail.com. Learn why
>>>> this is important at https://aka.ms/LearnAboutSenderIdentification
>>>> ]
>>>>
>>>> On 8/20/25 17:44, Gyorgy Sarvari via lists.openembedded.org wrote:
>>>>> On 8/20/25 13:53, Tony Rex wrote:
>>>>>> It was introduced by this commit:
>>>>>> https://git/
>>>>>> .yoctoproject.org%2Fpoky%2Fcommit%2F%3Fid%3Debfa1700f41b3411aec04
>>>>>> 01
>>>>>> 4
>>>>>> 4
>>>>>> 605166c35b8dd14&data=05%7C02%7Ctony.rex%40ericsson.com%7C7b5c099d
>>>>>> 17
>>>>>> 0
>>>>>> 0
>>>>>> 43080cca08dde004478d%7C92e84cebfbfd47abbe52080c6b87953f%7C0%7C0%7
>>>>>> C6
>>>>>> 3
>>>>>> 8
>>>>>> 913031194171335%7CUnknown%7CTWFpbGZsb3d8eyJFbXB0eU1hcGkiOnRydWUsI
>>>>>> lY
>>>>>> i
>>>>>> O
>>>>>> iIwLjAuMDAwMCIsIlAiOiJXaW4zMiIsIkFOIjoiTWFpbCIsIldUIjoyfQ%3D%3D%7
>>>>>> C0
>>>>>> %
>>>>>> 7
>>>>>> C%7C%7C&sdata=iYvLGK1DwJk78tEvmL%2FPOL2olj7iNL87eXqdsEraxLw%3D&re
>>>>>> se
>>>>>> r
>>>>>> v
>>>>>> ed=0
>>>>>>
>>>>>> Maybe "not allowed" is a strong wording, but that was how I interpreted it. Sorry, my bad if I misinterpreted it.
>>>>> In my understanding (which could be wrong), this is more to avoid
>>>>> working on a repository, where the exact work-revision can't be
>>>>> determined. But in your example, it can be determined, if your
>>>>> patch is not applied.
>>>>>
>>>>> Some extra ramblings, which may or may not be unadulterated
>>>>> garbage from
>>>>> me: Your initial commit message mentioned an expansion error. I
>>>>> looked into that a bit, and is it possible that the repository
>>>>> that is causing this issue has some access rate limit? The only
>>>>> thing that can fail sometimes in that code path seems to be "git ls-remote".
>>>>>
>>>>> Now, when I noticed this, I compiled a git version for myself that
>>>>> randomly throws an error with "git ls-remote" command, and when I
>>>>> used it with bitbake, I got this:
>>>>>
>>>>> ERROR: ExpansionError during parsing
>>>>> /home/meee/stuff/ptest-images/poky/meta/recipes-core/ell/mytest_get.
>>>>> b
>>>>> b
>>>>> bb.data_smart.ExpansionError: Failure expanding variable
>>>>> fetcher_hashes_dummyfunc[vardepvalue], expression was <etc etc
>>>>> etc>
>>>>>
>>>>> Is this the same error you got?
>>>> Sorry for the spam, one thing I forgot - the error on my end also contains details about the command that failed, pointing to git. If it's not the same as you get, do you have a copy of the full bitbake output you get when you see this error?
>>>>
>>>>>> Tony
>>>>>>
>>>>>> -----Original Message-----
>>>>>> From: Gyorgy Sarvari <skandigraun@gmail.com>
>>>>>> Sent: Wednesday, 20 August 2025 12:18
>>>>>> To: Tony Rex <tony.rex@ericsson.com>; Alexander Kanavin
>>>>>> <alex.kanavin@gmail.com>
>>>>>> Cc: Mathieu Dubois-Briand <mathieu.dubois-briand@bootlin.com>;
>>>>>> openembedded-core@lists.openembedded.org
>>>>>> Subject: Re: [OE-core] [PATCH 0/1] base.bbclass: Fix the recipe
>>>>>> parsing race issue
>>>>>>
>>>>>> [You don't often get email from skandigraun@gmail.com. Learn why
>>>>>> this is important at
>>>>>> https://aka.ms/LearnAboutSenderIdentification
>>>>>> ]
>>>>>>
>>>>>> On 8/19/25 23:10, Tony Rex via lists.openembedded.org wrote:
>>>>>>> Hi Alex and Mathieu!
>>>>>>>
>>>>>>> Ok, so the patch fixes a bug and then the floating tag/branch function is in place again.
>>>>>>> The floating tag/branch function is broken due to the bug and the patch fixes it.
>>>>>>>
>>>>>>> The mail from Mathieu below might not fully reflect the issue here, since it seems to create a recipe with the help of devtool.
>>>>>>>
>>>>>>> To verify:
>>>>>>> 1. add layers/oe-core/meta-selftest to conf/bblayers.conf 2. run
>>>>>>> bitbake gitrepotest -c fetch
>>>>>>> result:
>>>>>>> NOTE: Tasks Summary: Attempted 2 tasks of which 0 didn't need to be rerun and all succeeded.
>>>>>>> 3. run bitbake gitrepotest -c cleanall 4. edit
>>>>>>> ../layers/oe-core/meta-selftest/recipes-test/gitrepotest/gitrepo
>>>>>>> te
>>>>>>> s
>>>>>>> t
>>>>>>> .b
>>>>>>> b
>>>>>>> add tag=2.12 in SRC_URI
>>>>>>> SRC_URI =
>>>>>>> "git://git.yoctoproject.org/git/matchbox-panel-2;branch=master;t
>>>>>>> ag
>>>>>>> =
>>>>>>> 2
>>>>>>> .1 2;protocol=https \ and comment out SRCREV #SRCREV =
>>>>>>> "f82ca3f42510fb3ef10f598b393eb373a2c34ca7"
>>>>>>> 5. run bitbake gitrepotest -c fetch
>>>>>>> result:
>>>>>>> NOTE: Tasks Summary: Attempted 2 tasks of which 0 didn't need to be rerun and all succeeded.
>>>>>>> The fetch goes through although this is not allowed according to the floating tag/branch error statement if not d.getVar("__BBSRCREV_SEEN"): in git.py.
>>>>>> Could you point to a commit/email/etc about this "no floating
>>>>>> tags allowed" statement? AFAIK tags without specifying revision
>>>>>> is allowed and supported. Not recommended, due to mandatory
>>>>>> network access, and because git tags are not immutable, but I'm
>>>>>> surprised to hear that they are not supposed to work (which of
>>>>>> course can also mean that I just missed a memo)
>>>>>>
>>>>>>> 6. run bitbake gitrepotest -c cleanall 7. apply patch
>>>>>>> 0001-base.bbclass-Fix-the-recipe-parsing-race-issue.patch
>>>>>>> 8. run bitbake gitrepotest -c fetch
>>>>>>> result:
>>>>>>> ERROR: gitrepotest-1.0-r0 do_fetch: Bitbake Fetcher Error:
>>>>>>> FetchError("Recipe uses a floating tag/branch '2.12' for repo
>>>>>>> 'git.yoctoproject.org/git/matchbox-panel-2' without a fixed
>>>>>>> SRCREV yet doesn't call bb.fetch2.get_srcrev() (use SRCPV in PV
>>>>>>> for OE).",
>>>>>>> None)
>>>>>>> ERROR: Logfile of failure stored in:
>>>>>>> .../build/tmp-glibc/work/cortexa53-wrs-linux/gitrepotest/1.0/temp/log.
>>>>>>> do_fetch.1254999
>>>>>>> ERROR: Task (.../layers/oe-core/meta-selftest/recipes-test/gitrepotest/gitrepotest.bb:do_fetch) failed with exit code '1'
>>>>>>> NOTE: Tasks Summary: Attempted 2 tasks of which 0 didn't need to be rerun and 1 failed.
>>>>>>>
>>>>>>> Error is the expected result when only having a tag set.
>>>>>>>
>>>>>>> Tony
>>>>>>>
>>>>>>> -----Original Message-----
>>>>>>> From: Alexander Kanavin <alex.kanavin@gmail.com>
>>>>>>> Sent: Tuesday, 19 August 2025 10:53
>>>>>>> To: Tony Rex <tony.rex@ericsson.com>
>>>>>>> Cc: Mathieu Dubois-Briand <mathieu.dubois-briand@bootlin.com>;
>>>>>>> openembedded-core@lists.openembedded.org
>>>>>>> Subject: Re: [OE-core] [PATCH 0/1] base.bbclass: Fix the recipe
>>>>>>> parsing race issue
>>>>>>>
>>>>>>> [You don't often get email from alex.kanavin@gmail.com. Learn
>>>>>>> why this is important at
>>>>>>> https://aka.ms/LearnAboutSenderIdentification
>>>>>>> ]
>>>>>>>
>>>>>>> You need to make things a little bit easier for maintainers and try to save their time.
>>>>>>>
>>>>>>> I would suggest that you provide clear steps to observe the issue.
>>>>>>> Attach the recipe file that would demonstrate the issue, and provide clear steps to trigger it. What is confusing right now is that you say here's a recipe that builds ok, but then you 'apply the patch' and then there's some kind of failure. So which patch needs to be applied?
>>>>>>>
>>>>>>> Alex
>>>>>>>
>>>>>>> On Fri, 15 Aug 2025 at 17:29, Tony Rex via lists.openembedded.org <tony.rex=ericsson.com@lists.openembedded.org> wrote:
>>>>>>>> Hi Mathieu!
>>>>>>>>
>>>>>>>> I think that this is how it is intended to work, bitbake does not allow floating tag/branch anymore.
>>>>>>>> Please correct me if I'm wrong @Richard Purdie (who implemented
>>>>>>>> this for security reasons I think.)
>>>>>>>>
>>>>>>>> The patch that I have makes sure that the parsing and setting
>>>>>>>> of the variables in the recipe are done before executing the do_fetch command, and as a side effect the floating tag/branch protection is working as intended again.
>>>>>>>>
>>>>>>>> How does the recipe look like?
>>>>>>>> devtool add kernel-module-hello-world https://git/.
>>>>>>>> yoctoproject.org%2Fgit%2Fkernel-module-hello-world&data=05%7C02
>>>>>>>> %7
>>>>>>>> C
>>>>>>>> t
>>>>>>>> on
>>>>>>>> y
>>>>>>>> .rex%40ericsson.com%7Cde1ae2aceeda470b381608dddefdd730%7C92e84c
>>>>>>>> eb
>>>>>>>> f
>>>>>>>> b
>>>>>>>> fd
>>>>>>>> 4
>>>>>>>> 7abbe52080c6b87953f%7C0%7C0%7C638911904005151487%7CUnknown%7CTW
>>>>>>>> Fp
>>>>>>>> b
>>>>>>>> G
>>>>>>>> Zs
>>>>>>>> b
>>>>>>>> 3d8eyJFbXB0eU1hcGkiOnRydWUsIlYiOiIwLjAuMDAwMCIsIlAiOiJXaW4zMiIs
>>>>>>>> Ik
>>>>>>>> F
>>>>>>>> O
>>>>>>>> Ij
>>>>>>>> o
>>>>>>>> iTWFpbCIsIldUIjoyfQ%3D%3D%7C0%7C%7C%7C&sdata=jyCicRQT%2FLtb1Yxi
>>>>>>>> FH
>>>>>>>> K
>>>>>>>> g
>>>>>>>> w5
>>>>>>>> B
>>>>>>>> nNktN4t9IciiyxtsEwO4%3D&reserved=0
>>>>>>>> The one I get looks like this:
>>>>>>>>
>>>>>>>> cat
>>>>>>>> build/workspace/recipes/kernel-module-hello-world/kernel-module
>>>>>>>> -h
>>>>>>>> e
>>>>>>>> l
>>>>>>>> lo
>>>>>>>> -
>>>>>>>> world_git.bb
>>>>>>>> # Recipe created by recipetool
>>>>>>>> # This is the basis of a recipe and may need further editing in order to be fully functional.
>>>>>>>> # (Feel free to remove these comments when editing.)
>>>>>>>>
>>>>>>>> # WARNING: the following LICENSE and LIC_FILES_CHKSUM values
>>>>>>>> are best guesses - it is # your responsibility to verify that the values are complete and correct.
>>>>>>>> #
>>>>>>>> # The following license files were not able to be identified
>>>>>>>> and are # represented as "Unknown" below, you will need to check them yourself:
>>>>>>>> #   LICENSE
>>>>>>>> LICENSE = "Unknown"
>>>>>>>> LIC_FILES_CHKSUM = "file://LICENSE;md5=7b785be6433a2e4af6a641cc7f0330d1"
>>>>>>>>
>>>>>>>> SRC_URI = "git://git.yoctoproject.org/git/kernel-module-hello-world;protocol=https;branch=master"
>>>>>>>>
>>>>>>>> # Modify these as desired
>>>>>>>> PV = "1.0+git"
>>>>>>>> SRCREV = "b66f0f2bd559a1df81fc1bda240eba07a2ecdd2b"
>>>>>>>>
>>>>>>>> S = "${WORKDIR}/git"
>>>>>>>>
>>>>>>>> inherit module
>>>>>>>>
>>>>>>>> EXTRA_OEMAKE:append:task-install = " -C ${STAGING_KERNEL_DIR} M=${S}"
>>>>>>>> EXTRA_OEMAKE += "KERNEL_DIR=${STAGING_KERNEL_DIR}"
>>>>>>>>
>>>>>>>> And it contains SRCREV = "b66f0f2bd559a1df81fc1bda240eba07a2ecdd2b"
>>>>>>>>
>>>>>>>> Tony
>>>>>>>>
>>>>>>>> -----Original Message-----
>>>>>>>> From: Mathieu Dubois-Briand <mathieu.dubois-briand@bootlin.com>
>>>>>>>> Sent: Friday, 15 August 2025 11:26
>>>>>>>> To: Tony Rex <tony.rex@ericsson.com>;
>>>>>>>> openembedded-core@lists.openembedded.org
>>>>>>>> Subject: Re: [OE-core] [PATCH 0/1] base.bbclass: Fix the recipe
>>>>>>>> parsing race issue
>>>>>>>>
>>>>>>>> [You don't often get email from mathieu.dubois-briand@bootlin.com.
>>>>>>>> Learn why this is important at
>>>>>>>> https://aka.ms/LearnAboutSenderIdentification ]
>>>>>>>>
>>>>>>>> On Thu Aug 14, 2025 at 7:23 PM CEST, Tony Rex via lists.openembedded.org wrote:
>>>>>>>>> It seems that the race condition happens when doing do_fetch
>>>>>>>>> on really small repos and it can be difficult to reproduce so
>>>>>>>>> below the other case with floating tag/branch that is easier to verify.
>>>>>>>>>
>>>>>>>>> To verify the floating tag/branch issue have s simple recipe
>>>>>>>>> hello_1.0.bb and set the SRC_URI to SRC_URI = "git://<repository>;protocol=https;user=${USER};branch=master;tag=v1.0"
>>>>>>>>>
>>>>>>>>> run command bitbake hello everything builds.
>>>>>>>>>
>>>>>>>>> Apply the patch
>>>>>>>>>
>>>>>>>>> run command bitbake hello
>>>>>>>>> ERROR: hello-1.0-r1 do_fetch: Bitbake Fetcher Error:
>>>>>>>>> FetchError("Recipe uses a floating tag/branch 'v1.0' for repo
>>>>>>>>> '<repository>' without a fixed SRCREV yet doesn't call
>>>>>>>>> bb.fetch2.get_srcrev() (use SRCPV in PV for OE).", None)
>>>>>>>>> ERROR: Logfile of failure stored in:
>>>>>>>>> hello/1.0/temp/log.do_fetch.3289932
>>>>>>>>> ERROR: Task (hello/hello_1.0.bb:do_fetch) failed with exit code '1'
>>>>>>>>>
>>>>>>>>> Tony Rex (1):
>>>>>>>>>   base.bbclass: Fix the recipe parsing race issue
>>>>>>>>>
>>>>>>>>>  meta/classes-global/base.bbclass | 2 +-
>>>>>>>>>  1 file changed, 1 insertion(+), 1 deletion(-)
>>>>>>>> Hi Tony,
>>>>>>>>
>>>>>>>> Thanks for your patch.
>>>>>>>>
>>>>>>>> It looks like this is breaking some builds with the following error:
>>>>>>>>
>>>>>>>> oeqa.utils.subprocesstweak.OETestCalledProcessError: Command '. /srv/pokybuild/yocto-worker/qemux86-64/build/build/tmp/work/qemux86_64-poky-linux/core-image-sato/1.0/testsdkext/environment-setup-core2-64-poky-linux > /dev/null; devtool add kernel-module-hello-world https://git.yoctoproject.org/git/kernel-module-hello-world;' returned non-zero exit status 1.
>>>>>>>> ...
>>>>>>>> ERROR: Bitbake Fetcher Error: FetchError("Recipe uses a
>>>>>>>> floating tag/branch 'master' for repo
>>>>>>>> 'git.yoctoproject.org/git/kernel-module-hello-world' without a
>>>>>>>> fixed SRCREV yet doesn't call bb.fetch2.get_srcrev() (use SRCPV
>>>>>>>> in PV for OE).", None)
>>>>>>>>
>>>>>>>> https://auto/
>>>>>>>> builder.yoctoproject.org%2Fvalkyrie%2F%23%2Fbuilders%2F68%2Fbui
>>>>>>>> ld
>>>>>>>> s
>>>>>>>> %
>>>>>>>> 2F
>>>>>>>> 2
>>>>>>>> 284&data=05%7C02%7Ctony.rex%40ericsson.com%7Cde1ae2aceeda470b38
>>>>>>>> 16
>>>>>>>> 0
>>>>>>>> 8
>>>>>>>> dd
>>>>>>>> d
>>>>>>>> efdd730%7C92e84cebfbfd47abbe52080c6b87953f%7C0%7C0%7C6389119040
>>>>>>>> 05
>>>>>>>> 2
>>>>>>>> 0
>>>>>>>> 90
>>>>>>>> 0
>>>>>>>> 3%7CUnknown%7CTWFpbGZsb3d8eyJFbXB0eU1hcGkiOnRydWUsIlYiOiIwLjAuM
>>>>>>>> DA
>>>>>>>> w
>>>>>>>> M
>>>>>>>> CI
>>>>>>>> s
>>>>>>>> IlAiOiJXaW4zMiIsIkFOIjoiTWFpbCIsIldUIjoyfQ%3D%3D%7C0%7C%7C%7C&s
>>>>>>>> da
>>>>>>>> t
>>>>>>>> a
>>>>>>>> =m
>>>>>>>> P
>>>>>>>> UXMdb%2FYOyGIm0rnncaWgcqtwkUIh8364IP3eYDPe4%3D&reserved=0
>>>>>>>> https://auto/
>>>>>>>> builder.yoctoproject.org%2Fvalkyrie%2F%23%2Fbuilders%2F30%2Fbui
>>>>>>>> ld
>>>>>>>> s
>>>>>>>> %
>>>>>>>> 2F
>>>>>>>> 2
>>>>>>>> 199&data=05%7C02%7Ctony.rex%40ericsson.com%7Cde1ae2aceeda470b38
>>>>>>>> 16
>>>>>>>> 0
>>>>>>>> 8
>>>>>>>> dd
>>>>>>>> d
>>>>>>>> efdd730%7C92e84cebfbfd47abbe52080c6b87953f%7C0%7C0%7C6389119040
>>>>>>>> 05
>>>>>>>> 2
>>>>>>>> 2
>>>>>>>> 74
>>>>>>>> 3
>>>>>>>> 2%7CUnknown%7CTWFpbGZsb3d8eyJFbXB0eU1hcGkiOnRydWUsIlYiOiIwLjAuM
>>>>>>>> DA
>>>>>>>> w
>>>>>>>> M
>>>>>>>> CI
>>>>>>>> s
>>>>>>>> IlAiOiJXaW4zMiIsIkFOIjoiTWFpbCIsIldUIjoyfQ%3D%3D%7C0%7C%7C%7C&s
>>>>>>>> da
>>>>>>>> t
>>>>>>>> a
>>>>>>>> =R
>>>>>>>> 0
>>>>>>>> HFameGl2LmIVMT9ml%2BDMnj%2FMbG6SW0%2F3yNZHQu2js%3D&reserved=0
>>>>>>>> https://auto/
>>>>>>>> builder.yoctoproject.org%2Fvalkyrie%2F%23%2Fbuilders%2F16%2Fbui
>>>>>>>> ld
>>>>>>>> s
>>>>>>>> %
>>>>>>>> 2F
>>>>>>>> 2
>>>>>>>> 247&data=05%7C02%7Ctony.rex%40ericsson.com%7Cde1ae2aceeda470b38
>>>>>>>> 16
>>>>>>>> 0
>>>>>>>> 8
>>>>>>>> dd
>>>>>>>> d
>>>>>>>> efdd730%7C92e84cebfbfd47abbe52080c6b87953f%7C0%7C0%7C6389119040
>>>>>>>> 05
>>>>>>>> 2
>>>>>>>> 4
>>>>>>>> 53
>>>>>>>> 8
>>>>>>>> 7%7CUnknown%7CTWFpbGZsb3d8eyJFbXB0eU1hcGkiOnRydWUsIlYiOiIwLjAuM
>>>>>>>> DA
>>>>>>>> w
>>>>>>>> M
>>>>>>>> CI
>>>>>>>> s
>>>>>>>> IlAiOiJXaW4zMiIsIkFOIjoiTWFpbCIsIldUIjoyfQ%3D%3D%7C0%7C%7C%7C&s
>>>>>>>> da
>>>>>>>> t
>>>>>>>> a
>>>>>>>> =N
>>>>>>>> S
>>>>>>>> FhEupwMkgqUAhpRgFbKnyFl%2FsY%2B4UJMhVQHhqOaNM%3D&reserved=0
>>>>>>>>
>>>>>>>> Can you have a look at this error please?
>>>>>>>>
>>>>>>>> Best regards,
>>>>>>>> Mathieu
>>>>>>>>
>>>>>>>> --
>>>>>>>> Mathieu Dubois-Briand, Bootlin
>>>>>>>> Embedded Linux and Kernel engineering https://boot/
>>>>>>>> lin.com%2F&data=05%7C02%7Ctony.rex%40ericsson.com%7Cde1ae2aceed
>>>>>>>> a4
>>>>>>>> 7
>>>>>>>> 0
>>>>>>>> b3
>>>>>>>> 8
>>>>>>>> 1608dddefdd730%7C92e84cebfbfd47abbe52080c6b87953f%7C0%7C0%7C638
>>>>>>>> 91
>>>>>>>> 1
>>>>>>>> 9
>>>>>>>> 04
>>>>>>>> 0
>>>>>>>> 05263327%7CUnknown%7CTWFpbGZsb3d8eyJFbXB0eU1hcGkiOnRydWUsIlYiOi
>>>>>>>> Iw
>>>>>>>> L
>>>>>>>> j
>>>>>>>> Au
>>>>>>>> M
>>>>>>>> DAwMCIsIlAiOiJXaW4zMiIsIkFOIjoiTWFpbCIsIldUIjoyfQ%3D%3D%7C0%7C%
>>>>>>>> 7C
>>>>>>>> %
>>>>>>>> 7
>>>>>>>> C&
>>>>>>>> s
>>>>>>>> data=oyi7YWsXDVHn7QB4fDdG1sBp7gQsfGWcJTWtX1CjAw4%3D&reserved=0
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>> 
>>>>>