Re: [bitbake-devel] [PATCH] doc: document dependency tracking associated with addpylib

Re: [bitbake-devel] [PATCH] doc: document dependency tracking associated with addpylib

[Antonin Godard]

Hi,

On Mon Oct 12, 2026 at 10:25 AM CEST, Yann Dirson via lists.openembedded.org wrote:
> Reported-by: Claude Opus 5.5
> Signed-off-by: Yann Dirson <yann.dirson@vates.tech>
> ---
>  doc/bitbake-user-manual/bitbake-user-manual-metadata.rst | 8 ++++++++
>  1 file changed, 8 insertions(+)
>
> diff --git a/doc/bitbake-user-manual/bitbake-user-manual-metadata.rst b/doc/bitbake-user-manual/bitbake-user-manual-metadata.rst
> index 621aab253..eef1f9624 100644
> --- a/doc/bitbake-user-manual/bitbake-user-manual-metadata.rst
> +++ b/doc/bitbake-user-manual/bitbake-user-manual-metadata.rst
> @@ -2264,6 +2264,14 @@ The specified <namespace> is imported automatically, and if the imported
>  module specifies an attribute named ``BBIMPORTS``, that list of
>  sub-modules is iterated and imported too.
>  
> +Additionally, Bitbake will detect usage of functions from such a
> +library and include them in task hashes, so changes in those functions
> +trigger task rebuilds.  Note that only functions inside modules
> +declared using ``BBIMPORTS`` benefit from this dependency tracking,
> +any name defined directly in the specified namespace (e.g. in
> +`<namespace>/__init__.py`, or in a library hat would be defined as

s/hat/that/

> +`<namespace>.py`) would *not* benefit from this tracking.

Otherwise, this is clear to me. I did not know about this limitation/feature,
and this is definitely important to document, thanks!

Antonin
text:
Return-Path: <bounce+67608+20320+8963659+13752387@lists.openembedded.org>
Delivered-To: antonin.godard@bootlin.com
Received: from mail.galae.net ([172.22.1.253])
	by 9e7fea892943 with LMTP
	id 8CW/IMPbzGq5RBAAtaWaaQ
	(envelope-from <bounce+67608+20320+8963659+13752387@lists.openembedded.org>)
	for <antonin.godard@bootlin.com>; Mon, 12 Oct 2026 15:08:19 +0200
X-Original-To: antonin.godard@bootlin.com
Received: from smtpin-01.galae.net (smtpin-01.galae.net [51.159.2.148])
	(using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
	 key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256)
	(No client certificate requested)
	by mail.galae.net (Postcow) with ESMTPS id 0C14411D70CFA
	for <antonin.godard@bootlin.com>; Mon, 12 Oct 2026 15:08:14 +0200 (CEST)
Authentication-Results: mail.galae.net;
	dkim=pass header.d=lists.openembedded.org header.s=groups1 header.b="UGeNK/3F";
	dmarc=pass (policy=none) header.from=lists.openembedded.org;
	spf=pass (mail.galae.net: domain of bounce+67608+20320+8963659+13752387@lists.openembedded.org designates 66.175.222.12 as permitted sender) smtp.mailfrom=bounce+67608+20320+8963659+13752387@lists.openembedded.org
ARC-Seal: i=1; s=dkim; d=bootlin.com; t\x1791810495; a=rsa-sha256; cv=none;
	b=cNxEOJIttT5A4d73TX5EgewhO1UDsovk6JqMnVlJ/yDeswZLX3Zje5HS1C0H63666OYRQ+
	vB4qRoYtHGa8AY8VXlIYAvCd3scSSy2+4zEwA9Aow0cDN3u2XgkLx39MixSr1SsKQDlgP6
	jYtEN88EuCA/RIT5d22BMLn/yqm5KHEw9thz5IN9nSzr0sZ88wZ+u3xIRNPZ5NNQH6q6J/
	1YLfurJbqZzRJi3zOk1rGPXDvNzqUjDkVeIB6nstih1OCxii707kInq7CBQzujsGgwoKmI
	mOxVMFWe9dgnDhvhfu7Bole30vAuZhMmGqFepnogWNJg0HS1ZIoDApP0rCBaBg=ARC-Authentication-Results: i=1;
	mail.galae.net;
	dkim=pass header.d=lists.openembedded.org header.s=groups1 header.b="UGeNK/3F";
	dmarc=pass (policy=none) header.from=lists.openembedded.org;
	spf=pass (mail.galae.net: domain of bounce+67608+20320+8963659+13752387@lists.openembedded.org designates 66.175.222.12 as permitted sender) smtp.mailfrom=bounce+67608+20320+8963659+13752387@lists.openembedded.org
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=bootlin.com;
	s=dkim; t\x1791810495;
	h=from:from:sender:sender:reply-to:reply-to:subject:subject:date:date:
	 message-id:message-id:to:to:cc:cc:mime-version:mime-version:
	 content-type:content-type:resent-from:in-reply-to:in-reply-to:
	 references:references:list-id:list-help:list-unsubscribe:
	 list-unsubscribe-post:list-subscribe:list-post:dkim-signature;
	bh=znVcl4BpIH0xSR9Lwdoy3cSidbN6NYCx8EemJ2QhD7k=;
	b=KWOpEfbJDTzEccAY9Vf29WhbQ5Rcr/5nmPJ4pZ+Y5YIbRHGGU8iJgTOX5NQorIZi9o76wn
	fHehM/XctzkBcgSvUcWU58s0ZdVZGie/1jsGK5zffO1aGy4AcgGQ3Q/nGC9pKFVVs80yp7
	hBnQMIn6Q101NSiM0CzO/eBF7i4GCDm7CUMjKmBHmJu1MI8uyMsgkj8lGNGguhRkw7gnLs
	wLyf1+NFZcoNUP7YL5j4/RQCG/F+RpRmRYiBeBPPcdbt9niTGk9G4FX1sZtRUxLT1Mr+T0
	vXsb/1EK0hAFoCMXvgNSrrjIX104VeYT7u8bYJ7VNRDIFn0KrM6aKLMsGBODRg=Received: from web01.groups.io (web01.groups.io [66.175.222.12])
	by smtpin-01.galae.net (Postfix) with ESMTPS id 708A7343DD1
	for <antonin.godard@bootlin.com>; Mon, 12 Oct 2026 13:08:14 +0000 (UTC)
DKIM-Signature: a=rsa-sha256; bh=znVcl4BpIH0xSR9Lwdoy3cSidbN6NYCx8EemJ2QhD7k=;
 c=relaxed/relaxed; d=lists.openembedded.org;
 h=From:Subject:Date:Message-ID:To:Cc:In-Reply-To:References:Reply-To:Sender:List-Id:List-Post:List-Subscribe:List-Unsubscribe:List-Help:MIME-Version:Content-Type:Content-Transfer-Encoding;
 s=groups1; t\x1791810493; v=1; x\x1792069693;
 b=UGeNK/3Fbs1URhjutP5dGyKw9H89d9yyEoqsLuGP1zosiO/hll0uB9DoZ1mxbKsQLfvMflcI
 BoANhWpVMdM9MZYJAPeTh7Tvd88D12KKTwfrrZKrXDv92LBxBJXreua190vkhgTq5OUV8lgjbWi
 j3qljUEwp6DgG3wj4kqKv1EOCz+rMfG+1Ysb0t7Vz3bEWwk0LpdaXMS0hs47mZos8tNiVo+sMZg
 8RJzZrpnfwelSUvETX+Ge/Lv6OZd/BIc7/qMo5GEO1d2B8vKhwrHAFCJeWwZWDizfWvzaLpSGV+
 U50Cm4KjM5Q6bQsQiuKaxGj/9NnHtReK21+0al/P2niRA=X-Received: by 127.0.0.2 with SMTP id U7y0YY8964994x5XNlFmIQoq; Mon, 12 Oct 2026 06:08:13 -0700
X-Received: from smtpout-02.galae.net (smtpout-02.galae.net [185.246.84.56])
 by mx.groups.io with SMTP id smtpd.msgproc02-g2.12864.1791810489459952278
 for <bitbake-devel@lists.openembedded.org>;
 Mon, 12 Oct 2026 06:08:10 -0700
X-Received: from smtpout-01.galae.net (smtpout-01.galae.net [212.83.139.233])
	by smtpout-02.galae.net (Postfix) with ESMTPS id 224091A1044
	for <bitbake-devel@lists.openembedded.org>; Mon, 12 Oct 2026 13:08:07 +0000 (UTC)
X-Received: from mail.galae.net (mail.galae.net [212.83.136.155])
	by smtpout-01.galae.net (Postfix) with ESMTPS id EA0EB601A6;
	Mon, 12 Oct 2026 13:08:06 +0000 (UTC)
X-Received: from [127.0.0.1] (localhost [127.0.0.1]) by localhost (Mailerdaemon) with ESMTPSA id 77CBC11D70CE8;
	Mon, 12 Oct 2026 15:08:01 +0200 (CEST)
Mime-Version: 1.0
Date: Mon, 12 Oct 2026 15:08:00 +0200
Message-Id: <DM2VSZ91PNOJ.YXQAWC84B6NR@bootlin.com>
Cc: <docs@lists.yoctoproject.org>
From: "Antonin Godard via lists.openembedded.org" <antonin.godard=bootlin.com@lists.openembedded.org>
To: <yann.dirson@vates.tech>, <bitbake-devel@lists.openembedded.org>
Subject: Re: [bitbake-devel] [PATCH] doc: document dependency tracking associated with addpylib
References: <1791793541.8631fc262581453bbf619ec5b2062170.1a12f37b16e00080d7@vates.tech>
In-Reply-To: <1791793541.8631fc262581453bbf619ec5b2062170.1a12f37b16e00080d7@vates.tech>
X-Last-TLS-Session-Version: TLSv1.3
Precedence: Bulk
List-Subscribe: <mailto:bitbake-devel+subscribe@lists.openembedded.org>
List-Help: <mailto:bitbake-devel+help@lists.openembedded.org>
Sender: bitbake-devel@lists.openembedded.org
List-Id: <bitbake-devel.lists.openembedded.org>
Mailing-List: list bitbake-devel@lists.openembedded.org; contact bitbake-devel+owner@lists.openembedded.org
Delivered-To: mailing list bitbake-devel@lists.openembedded.org
Resent-Date: Mon, 12 Oct 2026 06:08:10 -0700
Resent-From: antonin.godard@bootlin.com
Reply-To: antonin.godard@bootlin.com
List-Post: <mailto:bitbake-devel@lists.openembedded.org>
List-Unsubscribe-Post: List-Unsubscribe=One-Click
List-Unsubscribe: <https://lists.openembedded.org/g/bitbake-devel/leave/13752387/8963659/1916163865/plugh>
X-Gm-Message-State: bb7qyzuGHZFqYnVPetBgRxu9x8963659AAContent-Type: multipart/mixed; boundary="unYFBYknsXXaikQDvLIZ"
X-Last-TLS-Session-Version: TLSv1.3
X-Rspamd-Queue-Id: 0C14411D70CFA
X-Spamd-Result: default: False [-7.38 / 30.00];
	BAYES_HAM(-5.50)[99.99%];
	IP_REPUTATION_HAM(-1.41)[asn: 63949(-0.40), country: SG(-0.01), ip: 66.175.222.12(-1.00)];
	MV_CASE(0.50)[];
	DMARC_POLICY_ALLOW(-0.50)[lists.openembedded.org,none];
	ONCE_RECEIVED(0.20)[];
	R_DKIM_ALLOW(-0.20)[lists.openembedded.org:s=groups1];
	R_SPF_ALLOW(-0.20)[+ip4:66.175.222.12:c];
	MAILLIST(-0.15)[generic];
	MIME_GOOD(-0.10)[multipart/mixed,text/plain];
	HAS_LIST_UNSUB(-0.01)[];
	MX_GOOD(-0.01)[];
	TAGGED_FROM(0.00)[67608,20320,8963659,13752387];
	URIBL_BLOCKED(0.00)[openembedded.org:email,openembedded.org:url,lists.openembedded.org:dkim,vates.tech:email,bootlin.com:mid,bootlin.com:email,bootlin.com:replyto,web01.groups.io:helo,web01.groups.io:rdns];
	FORGED_SENDER_MAILLIST(0.00)[];
	RCVD_TLS_ALL(0.00)[];
	MIME_TRACE(0.00)[0:+,1:+,2:+];
	ARC_SIGNED(0.00)[bootlin.com:s=dkim:i=1];
	RCPT_MAILCOW_DOMAIN(0.00)[bootlin.com];
	ARC_NA(0.00)[];
	BCC(0.00)[];
	FORGED_RECIPIENTS_MAILLIST(0.00)[];
	REPLYTO_DOM_NEQ_FROM_DOM(0.00)[];
	RCPT_COUNT_THREE(0.00)[3];
	DKIM_TRACE(0.00)[lists.openembedded.org:+];
	TO_DN_NONE(0.00)[];
	DNSWL_BLOCKED(0.00)[66.175.222.12:from];
	DWL_DNSWL_BLOCKED(0.00)[openembedded.org:dkim];
	PRECEDENCE_BULK(0.00)[];
	FROM_HAS_DN(0.00)[];
	FROM_NEQ_ENVFROM(0.00)[antonin.godard=bootlin.com@lists.openembedded.org,bounce@lists.openembedded.org];
	REPLYTO_DOM_NEQ_TO_DOM(0.00)[];
	NEURAL_HAM(-0.00)[-1.000];
	ASN(0.00)[asn:63949, ipnet:66.175.216.0/21, country:SG];
	HAS_REPLYTO(0.00)[antonin.godard@bootlin.com];
	EXTERNAL_RELAY_AUTHENTICATED(0.00)[];
	CTYPE_MIXED_BOGUS(0.00)[];
	RCVD_COUNT_ONE(0.00)[1];
	MISSING_XM_UA(0.00)[];
	RBL_SENDERSCORE_REPUT_BLOCKED(0.00)[66.175.222.12:from]

--unYFBYknsXXaikQDvLIZ
Content-Transfer-Encoding: quoted-printable
Content-Type: text/plain; charset=UTF-8

Hi,

On Mon Oct 12, 2026 at 10:25 AM CEST, Yann Dirson via lists.openembedded.org wrote:
> Reported-by: Claude Opus 5.5
> Signed-off-by: Yann Dirson <yann.dirson@vates.tech>
> ---
>  doc/bitbake-user-manual/bitbake-user-manual-metadata.rst | 8 ++++++++
>  1 file changed, 8 insertions(+)
>
> diff --git a/doc/bitbake-user-manual/bitbake-user-manual-metadata.rst b/doc/bitbake-user-manual/bitbake-user-manual-metadata.rst
> index 621aab253..eef1f9624 100644
> --- a/doc/bitbake-user-manual/bitbake-user-manual-metadata.rst
> +++ b/doc/bitbake-user-manual/bitbake-user-manual-metadata.rst
> @@ -2264,6 +2264,14 @@ The specified <namespace> is imported automatically, and if the imported
>  module specifies an attribute named ``BBIMPORTS``, that list of
>  sub-modules is iterated and imported too.
>  
> +Additionally, Bitbake will detect usage of functions from such a
> +library and include them in task hashes, so changes in those functions
> +trigger task rebuilds.  Note that only functions inside modules
> +declared using ``BBIMPORTS`` benefit from this dependency tracking,
> +any name defined directly in the specified namespace (e.g. in
> +`<namespace>/__init__.py`, or in a library hat would be defined as

s/hat/that/

> +`<namespace>.py`) would *not* benefit from this tracking.

Otherwise, this is clear to me. I did not know about this limitation/feature,
and this is definitely important to document, thanks!

Antonin

--unYFBYknsXXaikQDvLIZ
Content-Type: text/plain; charset="utf-8"
Content-Transfer-Encoding: quoted-printable
Content-Disposition: inline


-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#20320): https://lists.openembedded.org/g/bitbake-devel/message/20320
Mute This Topic: https://lists.openembedded.org/mt/121711853/8963659
Group Owner: bitbake-devel+owner@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/bitbake-devel/unsub [antonin.godard@bootlin.com]
-=-=-=-=-=-=-=-=-=-=-=-


--unYFBYknsXXaikQDvLIZ--

Re: [bitbake-devel] [PATCH] doc: document dependency tracking associated with addpylib

[Antonin Godard]

On Mon Oct 12, 2026 at 3:08 PM CEST, Antonin Godard via lists.openembedded.org wrote:
> Hi,
>
> On Mon Oct 12, 2026 at 10:25 AM CEST, Yann Dirson via lists.openembedded.org wrote:
>> Reported-by: Claude Opus 5.5
>> Signed-off-by: Yann Dirson <yann.dirson@vates.tech>
>> ---
>>  doc/bitbake-user-manual/bitbake-user-manual-metadata.rst | 8 ++++++++
>>  1 file changed, 8 insertions(+)
>>
>> diff --git a/doc/bitbake-user-manual/bitbake-user-manual-metadata.rst b/doc/bitbake-user-manual/bitbake-user-manual-metadata.rst
>> index 621aab253..eef1f9624 100644
>> --- a/doc/bitbake-user-manual/bitbake-user-manual-metadata.rst
>> +++ b/doc/bitbake-user-manual/bitbake-user-manual-metadata.rst
>> @@ -2264,6 +2264,14 @@ The specified <namespace> is imported automatically, and if the imported
>>  module specifies an attribute named ``BBIMPORTS``, that list of
>>  sub-modules is iterated and imported too.
>>  
>> +Additionally, Bitbake will detect usage of functions from such a
>> +library and include them in task hashes, so changes in those functions
>> +trigger task rebuilds.  Note that only functions inside modules
>> +declared using ``BBIMPORTS`` benefit from this dependency tracking,
>> +any name defined directly in the specified namespace (e.g. in
>> +`<namespace>/__init__.py`, or in a library hat would be defined as
>
> s/hat/that/
>
>> +`<namespace>.py`) would *not* benefit from this tracking.
>
> Otherwise, this is clear to me. I did not know about this limitation/feature,
> and this is definitely important to document, thanks!

Reading the code[1], it seems that functions prefixed with _ are not included
in the task checksums - if I understood correctly. I think it would be also
important to mention that. Could you add it in a v2?

[1]: https://git.openembedded.org/bitbake/tree/lib/bb/parse/ast.py#n319

Antonin

Re: [bitbake-devel] [PATCH] doc: document dependency tracking associated with addpylib

[Antonin Godard]

On Tue Oct 13, 2026 at 11:56 AM CEST, Yann Dirson via lists.openembedded.org wrote:
>
>
> On 10/12/26 3:20 PM, Antonin Godard wrote:
>> On Mon Oct 12, 2026 at 3:08 PM CEST, Antonin Godard via lists.openembedded.org wrote:
>>> Hi,
>>>
>>> On Mon Oct 12, 2026 at 10:25 AM CEST, Yann Dirson via lists.openembedded.org wrote:
>>>> Reported-by: Claude Opus 5.5
>>>> Signed-off-by: Yann Dirson <yann.dirson@vates.tech>
>>>> ---
>>>>   doc/bitbake-user-manual/bitbake-user-manual-metadata.rst | 8 ++++++++
>>>>   1 file changed, 8 insertions(+)
>>>>
>>>> diff --git a/doc/bitbake-user-manual/bitbake-user-manual-metadata.rst b/doc/bitbake-user-manual/bitbake-user-manual-metadata.rst
>>>> index 621aab253..eef1f9624 100644
>>>> --- a/doc/bitbake-user-manual/bitbake-user-manual-metadata.rst
>>>> +++ b/doc/bitbake-user-manual/bitbake-user-manual-metadata.rst
>>>> @@ -2264,6 +2264,14 @@ The specified <namespace> is imported automatically, and if the imported
>>>>   module specifies an attribute named ``BBIMPORTS``, that list of
>>>>   sub-modules is iterated and imported too.
>>>>   
>>>> +Additionally, Bitbake will detect usage of functions from such a
>>>> +library and include them in task hashes, so changes in those functions
>>>> +trigger task rebuilds.  Note that only functions inside modules
>>>> +declared using ``BBIMPORTS`` benefit from this dependency tracking,
>>>> +any name defined directly in the specified namespace (e.g. in
>>>> +`<namespace>/__init__.py`, or in a library hat would be defined as
>>>
>>> s/hat/that/
>>>
>>>> +`<namespace>.py`) would *not* benefit from this tracking.
>>>
>>> Otherwise, this is clear to me. I did not know about this limitation/feature,
>>> and this is definitely important to document, thanks!
>> 
>> Reading the code[1], it seems that functions prefixed with _ are not included
>> in the task checksums - if I understood correctly. I think it would be also
>> important to mention that. Could you add it in a v2?
>
> It could be asked why we would want to exclude them.  Usually that would 
> be internal functions, to be called by the exported ones, so they would 
> equally matter - but then they are actually not included in the task 
> hash, so it rather looks like a bug rather than something to document?

You're right, it seems strange to exclude them entirely. You may want to send a
patch to remove this then? At least it will kick-off the discussion on this
list.

Thanks,
Antonin

Re: [bitbake-devel] [PATCH] doc: document dependency tracking associated with addpylib

[Richard Purdie]

On Tue, 2026-10-13 at 14:09 +0200, Antonin Godard wrote:
> On Tue Oct 13, 2026 at 11:56 AM CEST, Yann Dirson via lists.openembedded.org wrote:
> > 
> > 
> > On 10/12/26 3:20 PM, Antonin Godard wrote:
> > > On Mon Oct 12, 2026 at 3:08 PM CEST, Antonin Godard via lists.openembedded.org wrote:
> > > > Hi,
> > > > 
> > > > On Mon Oct 12, 2026 at 10:25 AM CEST, Yann Dirson via lists.openembedded.org wrote:
> > > > > Reported-by: Claude Opus 5.5
> > > > > Signed-off-by: Yann Dirson <yann.dirson@vates.tech>
> > > > > ---
> > > > >   doc/bitbake-user-manual/bitbake-user-manual-metadata.rst | 8 ++++++++
> > > > >   1 file changed, 8 insertions(+)
> > > > > 
> > > > > diff --git a/doc/bitbake-user-manual/bitbake-user-manual-metadata.rst b/doc/bitbake-user-manual/bitbake-user-manual-metadata.rst
> > > > > index 621aab253..eef1f9624 100644
> > > > > --- a/doc/bitbake-user-manual/bitbake-user-manual-metadata.rst
> > > > > +++ b/doc/bitbake-user-manual/bitbake-user-manual-metadata.rst
> > > > > @@ -2264,6 +2264,14 @@ The specified <namespace> is imported automatically, and if the imported
> > > > >   module specifies an attribute named ``BBIMPORTS``, that list of
> > > > >   sub-modules is iterated and imported too.
> > > > >   
> > > > > +Additionally, Bitbake will detect usage of functions from such a
> > > > > +library and include them in task hashes, so changes in those functions
> > > > > +trigger task rebuilds.  Note that only functions inside modules
> > > > > +declared using ``BBIMPORTS`` benefit from this dependency tracking,
> > > > > +any name defined directly in the specified namespace (e.g. in
> > > > > +`<namespace>/__init__.py`, or in a library hat would be defined as
> > > > 
> > > > s/hat/that/
> > > > 
> > > > > +`<namespace>.py`) would *not* benefit from this tracking.
> > > > 
> > > > Otherwise, this is clear to me. I did not know about this limitation/feature,
> > > > and this is definitely important to document, thanks!
> > > 
> > > Reading the code[1], it seems that functions prefixed with _ are not included
> > > in the task checksums - if I understood correctly. I think it would be also
> > > important to mention that. Could you add it in a v2?
> > 
> > It could be asked why we would want to exclude them.  Usually that would 
> > be internal functions, to be called by the exported ones, so they would 
> > equally matter - but then they are actually not included in the task 
> > hash, so it rather looks like a bug rather than something to document?
> 
> You're right, it seems strange to exclude them entirely. You may want to send a
> patch to remove this then? At least it will kick-off the discussion on this
> list.

I suspect there are two levels of issues here. Funcitons with a single
"_" prefix are internal functions and those were the ones we likely
wanted to exclude. __init__ is probably an oversight. I do think people
are getting too focused on the detail of the taskhashes though. There
comes a point in our code when something becomes API rather than "data"
and the taskhashes really are meant to reflect more data than absolute
every detail. Before function library hashing that boundary was clearer
but it still exists (e.g. bitbake itself isn't hashed, nor should it
be, nor the host system python version).

Cheers,

Richard