Modifying SRC_URI before it is parsed by the fetcher

Modifying SRC_URI before it is parsed by the fetcher

[EXT-Antti Gärding]

[-- Attachment #1: Type: text/plain, Size: 1529 bytes --]

Hi,

I am developing a .bbclass file that modifies SRC_URI of the inheriting recipe before bitbake (in this case, the git fetcher) initializes its internal data structures with that data. Prepended to any task or as a dedicated task before any other tasks is too late, but a Python function declared __anonymous seems to be invoked right after parsing the recipe, which is early enough.

My log prints, however, revealed that such function is invoked not only at that point but also multiple times when building the recipe. I didn't inspect it thoroughly enough to be sure, but I got the impression that it might be invoked once during each task. That is not a critical issue right now, but it makes the logs look weird and requires extra care to make sure that running the function multiple times doesn't have any unwanted side-effects.

Is there any way to have code executed only right after parsing the recipe? In this case, the initialization of the internal data structures which it must precede seems to happen in urldata_init() of class Git in poky/bitbake/lib/bb/fetch2/git.py.

I discovered that there is variable BB_CURRENTTASK which is None on the very first run and contains the name of the current task on other runs. I tried making my function check it and act only when it is None, but that resulted in errors about basehash value having changed and bitbake thus considering the metadata non-deterministic.

Could BB_CURRENTTASK be somehow utilized to achieve what I am trying?


BR,
Antti Gärding

[-- Attachment #2: Type: text/html, Size: 4427 bytes --]

Re: [yocto] Modifying SRC_URI before it is parsed by the fetcher

[Alexander Kanavin]

On Tue, 12 May 2026 at 16:47, EXT-Antti Gärding via
lists.yoctoproject.org
<ext-antti.garding=kempower.com@lists.yoctoproject.org> wrote:
> I am developing a .bbclass file that modifies SRC_URI of the inheriting recipe before bitbake (in this case, the git fetcher) initializes its internal data structures with that data. Prepended to any task or as a dedicated task before any other tasks is too late, but a Python function declared __anonymous seems to be invoked right after parsing the recipe, which is early enough.

Please explain the problem first, before presenting the various
solution attempts. What do you need to modify in SRC_URI and why?
Maybe there's a completely different approach to the actual issue.

Alex

Re: [yocto] Modifying SRC_URI before it is parsed by the fetcher

[EXT-Antti Gärding]

[-- Attachment #1: Type: text/plain, Size: 429 bytes --]

> Please explain the problem first, before presenting the various
solution attempts. What do you need to modify in SRC_URI and why?
Maybe there's a completely different approach to the actual issue.

I want to add a tag parameter to one of the URIs under certain conditions so that bitbake can check that the commit matching the rev parameter has that tag. The tag check is already there in bitbake.


BR,
Antti Gärding

[-- Attachment #2: Type: text/html, Size: 1560 bytes --]

Re: [yocto] Modifying SRC_URI before it is parsed by the fetcher

[Quentin Schulz]

Hi Antti,

On 5/12/26 6:05 PM, EXT-Antti Gärding via lists.yoctoproject.org wrote:
> You don't often get email from ext-antti.garding=kempower.com@lists.yoctoproject.org. Learn why this is important<https://aka.ms/LearnAboutSenderIdentification>
>> Please explain the problem first, before presenting the various
> solution attempts. What do you need to modify in SRC_URI and why?
> Maybe there's a completely different approach to the actual issue.
> 
> I want to add a tag parameter to one of the URIs under certain conditions so that bitbake can check that the commit matching the rev parameter has that tag. The tag check is already there in bitbake.
> 

Which conditions?

The following pseudocode could help (untested).

def src_uri_tag(d):
     if conditions:
         return ';tag=' + mytag
     return ''

SRC_URI_TAG = "${@src_uri_tag(d)}"

SRC_URI = "git://someforge.co/myrepo.git;branch=main${SRC_URI_TAG}"

Cheers,
Quentin

Re: [yocto] Modifying SRC_URI before it is parsed by the fetcher

[EXT-Antti Gärding]

[-- Attachment #1: Type: text/plain, Size: 163 bytes --]

》Which conditions?

The tag is derived from PV and if there are multiple repositories in SRC_URI, only one is appended with a tag.


BR,
Antti Gärding


[-- Attachment #2: Type: text/html, Size: 1380 bytes --]

Re: [yocto] Modifying SRC_URI before it is parsed by the fetcher

[ChenQi]

[-- Attachment #1: Type: text/plain, Size: 917 bytes --]

On 5/13/26 14:26, EXT-Antti Gärding via lists.yoctoproject.org wrote:
> 》Which conditions?
>
> The tag is derived from PV and if there are multiple repositories in 
> SRC_URI, only one is appended with a tag.
>
>
> BR,
> Antti Gärding
>
>
> -=-=-=-=-=-=-=-=-=-=-=-
> Links: You receive all messages sent to this group.
> View/Reply Online (#66546):https://lists.yoctoproject.org/g/yocto/message/66546
> Mute This Topic:https://lists.yoctoproject.org/mt/119275823/3618072
> Group Owner:yocto+owner@lists.yoctoproject.org
> Unsubscribe:https://lists.yoctoproject.org/g/yocto/unsub [Qi.Chen@windriver.com]
> -=-=-=-=-=-=-=-=-=-=-=-
>
You can use a bbclass + event handler to achieve this.

e.g.,
modify-src-uri.bbclass

In that bbclass, write a function (e.g., modify_src_uri) to modify SRC_URIs.
Then:
addhandler modify_src_uri
modify_src_uri[eventmask] = "bb.event.XXX"

Regards,
Qi

[-- Attachment #2: Type: text/html, Size: 3136 bytes --]

Re: [yocto] Modifying SRC_URI before it is parsed by the fetcher

[EXT-Antti Gärding]

[-- Attachment #1: Type: text/plain, Size: 324 bytes --]

> You can use a bbclass + event handler to achieve this.

That sounds promising.

How can I find out which event to use? Grepping bitbake's code in my clone of Yocto didn't give me any easy hints. Yocto's manual gives impression that the list there is not a complete list but rather an example.


BR,
Antti Gärding

[-- Attachment #2: Type: text/html, Size: 2128 bytes --]

Re: [yocto] Modifying SRC_URI before it is parsed by the fetcher

[Alexander Kanavin]

On Wed, 13 May 2026 at 14:35, EXT-Antti Gärding via
lists.yoctoproject.org
<ext-antti.garding=kempower.com@lists.yoctoproject.org> wrote:
> > You can use a bbclass + event handler to achieve this.
>
> That sounds promising.
>
> How can I find out which event to use? Grepping bitbake's code in my clone of Yocto didn't give me any easy hints. Yocto's manual gives impression that the list there is not a complete list but rather an example.

I think you should still show us the body of the function that sets
the tag from the value PV. What is it doing that can not be done with
the approach that Quentin suggested? It's rather baffling that you
simply skipped his response without explaining that.

Alex

Re: [yocto] Modifying SRC_URI before it is parsed by the fetcher

[EXT-Antti Gärding]

[-- Attachment #1: Type: text/plain, Size: 665 bytes --]


> I think you should still show us the body of the function that sets
the tag from the value PV. What is it doing that can not be done with
the approach that Quentin suggested? It's rather baffling that you
simply skipped his response without explaining that.

Sorry for apparently ignoring Quentin's suggestion and thanks for bringing it up again. I first ignored it because applying it requires touching the SRC_URI in the recipe. Now that I had another look at it, it might work since it is not hard-coded but comes from the function. I will try it and also discuss with my team if they see any reason not to solve this that way.


BR,
Antti Gärding

[-- Attachment #2: Type: text/html, Size: 2105 bytes --]

Re: [yocto] Modifying SRC_URI before it is parsed by the fetcher

[EXT-Antti Gärding]

[-- Attachment #1: Type: text/plain, Size: 1244 bytes --]

> Which conditions?

> The following pseudocode could help (untested).

> def src_uri_tag(d):
>      if conditions:
>         return ';tag=' + mytag
>     return ''

> SRC_URI_TAG = "${@src_uri_tag(d)}"

> SRC_URI = "git://someforge.co/myrepo.git;branch=main${SRC_URI_TAG}"

I am getting back to this almost immediately after saying that I would try this...

I think this would work if the only goal was to append a tag parameter into an SRC_URI entry, but I need something more. I'd like the function to be in a class so that I can add the tag matching into a recipe by just inheriting that class regardless of what is in SRC_URI and also disable tag matching by just not inheriting that class anymore.

I don't know what to say about the conditions. Maybe they are not conditions at all but rather the logic that decides which SRC_URI entry to append the tag parameter to. That logic depends on PV and a variable that, if defined, contains the name of the repository for which the tag is to be appended. It is defined only when there are multiple repositories in SRC_URI and in that case, there is also a name parameter in each repository URI and that variable matches one of those repository names.


BR,
Antti Gärding

[-- Attachment #2: Type: text/html, Size: 3400 bytes --]

Re: [yocto] Modifying SRC_URI before it is parsed by the fetcher

[Alexander Kanavin]

On Wed, 13 May 2026 at 15:11, EXT-Antti Gärding via
lists.yoctoproject.org
<ext-antti.garding=kempower.com@lists.yoctoproject.org> wrote:
> I think this would work if the only goal was to append a tag parameter into an SRC_URI entry, but I need something more. I'd like the function to be in a class so that I can add the tag matching into a recipe by just inheriting that class regardless of what is in SRC_URI and also disable tag matching by just not inheriting that class anymore.

You still aren't saying what the condition for adding, or not adding
the tag is. What are the two scenarios here, why would one or the
other be selected? Please: if you want help, you should be forward and
clear with the use case, even if you have to disclose 'something'
about your development process.

You can probably just always inherit the class, but the class would
add the tag (or not), depending on whether some variable is set. And
then you either set it or not in your build/conf/local.conf, or some
other place that affects bitbake configuration for the build.

Alex

Re: [yocto] Modifying SRC_URI before it is parsed by the fetcher

[EXT-Antti Gärding]

[-- Attachment #1: Type: text/plain, Size: 1563 bytes --]

> You still aren't saying what the condition for adding, or not adding
the tag is. What are the two scenarios here, why would one or the
other be selected? Please: if you want help, you should be forward and
clear with the use case, even if you have to disclose 'something'
about your development process.

If there is only one repository in SRC_URI, then that is the one to add the tag to. If there are more than one of them, then one of them is the main or root repository and other repositories are cloned into subdirectories of that one. That one is identified based on a name parameter with help of the variable defined in the recipe that I mentioned earlier. Each SRC_URI entry is checked to see if the tag should be added to that entry and if yes, it is added. If it for some reason ends up being added to several entries or to none, that will either cause the build to fail or the tag-commit-hash-matching in bitbake to give false positive which are both acceptable outcomes in that situation.

> You can probably just always inherit the class, but the class would
add the tag (or not), depending on whether some variable is set. And
then you either set it or not in your build/conf/local.conf, or some
other place that affects bitbake configuration for the build.

For debugging purposes, the appending must be easy to toggle on and off for any single recipe. I would accomplish that by having it packed into a class which would make its decisions based on the meta data that is already in the recipe inheriting it.


BR,
Antti Gärding

[-- Attachment #2: Type: text/html, Size: 3793 bytes --]

Re: [yocto] Modifying SRC_URI before it is parsed by the fetcher

[Alexander Kanavin]

On Wed, 13 May 2026 at 16:22, EXT-Antti Gärding via
lists.yoctoproject.org
<ext-antti.garding=kempower.com@lists.yoctoproject.org> wrote:
> For debugging purposes, the appending must be easy to toggle on and off for any single recipe. I would accomplish that by having it packed into a class which would make its decisions based on the meta data that is already in the recipe inheriting it.

Then what in Quentin's suggestion isn't going to work? You put the
function and the SRC_URI_TAG assignment into the class. You modify the
SRC_URI in the recipe to refer to SRC_URI_TAG variable, and add a
class inherit into the recipe. Which part of this isn't appropriate or
possible and why?

Alex

Re: [yocto] Modifying SRC_URI before it is parsed by the fetcher

[EXT-Antti Gärding]

[-- Attachment #1: Type: text/plain, Size: 676 bytes --]

> Then what in Quentin's suggestion isn't going to work? You put the
function and the SRC_URI_TAG assignment into the class. You modify the
SRC_URI in the recipe to refer to SRC_URI_TAG variable, and add a
class inherit into the recipe. Which part of this isn't appropriate or
possible and why?

As far as I see it, it requires you not only to inherit a class but also to modify the SRC_URI and possibly make it dependent of the class (I am not sure what would happen if I added SRC_URI_TAG but didn't inherit the class).

In other words, you can't turn the tag verification on and off by simply adding or removing line "inherit src-uri-tag".


BR,
Antti Gärding

[-- Attachment #2: Type: text/html, Size: 2293 bytes --]

Re: [yocto] Modifying SRC_URI before it is parsed by the fetcher

[Quentin Schulz]

Hi Antti,

On 5/13/26 5:07 PM, EXT-Antti Gärding via lists.yoctoproject.org wrote:
>> Then what in Quentin's suggestion isn't going to work? You put the
> function and the SRC_URI_TAG assignment into the class. You modify the
> SRC_URI in the recipe to refer to SRC_URI_TAG variable, and add a
> class inherit into the recipe. Which part of this isn't appropriate or
> possible and why?
> 
> As far as I see it, it requires you not only to inherit a class but also to modify the SRC_URI and possibly make it dependent of the class (I am not sure what would happen if I added SRC_URI_TAG but didn't inherit the class).
> 

But the *only* thing you want to do is modifying SRC_URI but not 
actually modify it, I don't understand why this is an absolute 
impossibility and why you must find hacks within BitBake instead. This 
all sounds like a horrific XY problem.

> In other words, you can't turn the tag verification on and off by simply adding or removing line "inherit src-uri-tag".
> 

classes/src_uri_tag.bbclass

def src_uri_tag(d):
     if conditions or not 
bb.utils.to_boolean(d.getVar('PLEASE_ADD_TAG'), True):
         return ';tag=' + mytag
     return ''

SRC_URI_TAG = "${@src_uri_tag(d)}"

recipes-whatever/some/recipe.bb

inherit src_uri_tag

SRC_URI = "git://someforge.co/myrepo.git;branch=main${SRC_URI_TAG}"

 From your local.conf:

PLEASE_ADD_TAG = "0"

to disable tags globally.

PLEASE_ADD_TAG:pn-recipe = "0"

to disable tags for a specific recipe.

Untested of course.

Cheers,
Quentin