[PATCH 1/1] icecc: calculate pn and bpn from FILE

[PATCH 1/1] icecc: calculate pn and bpn from FILE

[Peter Suti]

Starting with Yocto Mickledore the PN and BPN variables are set to "no-pn"
at this point in the `use_icecc()` function, so instead we need to
re-parse them from the file again as a workaround otherwise icecc is broken.

Fixes: 3be00ad9052de ("bitbake.conf: Add BB_HASH_CODEPARSER_VALS")
Where "PN=nopn" is set to optimise the codeparser cache's size.

[0] https://lists.yoctoproject.org/g/yocto/topic/icecc_support_broken/103429714

Signed-off-by: Peter Suti <peter.suti@streamunlimited.com>
---
 meta/classes/icecc.bbclass | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/meta/classes/icecc.bbclass b/meta/classes/icecc.bbclass
index 8a48f2ad63..e3b07028b8 100644
--- a/meta/classes/icecc.bbclass
+++ b/meta/classes/icecc.bbclass
@@ -143,8 +143,8 @@ def use_icecc(bb,d):
     if icecc_is_cross_canadian(bb, d):
         return "no"
 
-    pn = d.getVar('PN')
-    bpn = d.getVar('BPN')
+    pn = bb.parse.vars_from_file(d.getVar('FILE', False),d)[0] or 'defaultpkgname'
+    bpn = oe.utils.prune_suffix(pn, d.getVar('SPECIAL_PKGSUFFIX').split(), d)
 
     # Enable/disable checks are made against BPN, because there is a good
     # chance that if icecc should be skipped for a recipe, it should be skipped
-- 
2.43.0

Re: [OE-core] [PATCH 1/1] icecc: calculate pn and bpn from FILE

[Richard Purdie]

On Tue, 2025-06-03 at 10:58 +0200, Peter Suti via lists.openembedded.org wrote:
> Starting with Yocto Mickledore the PN and BPN variables are set to "no-pn"
> at this point in the `use_icecc()` function, so instead we need to
> re-parse them from the file again as a workaround otherwise icecc is broken.
> 
> Fixes: 3be00ad9052de ("bitbake.conf: Add BB_HASH_CODEPARSER_VALS")
> Where "PN=nopn" is set to optimise the codeparser cache's size.
> 
> [0] https://lists.yoctoproject.org/g/yocto/topic/icecc_support_broken/103429714
> 
> Signed-off-by: Peter Suti <peter.suti@streamunlimited.com>
> ---
>  meta/classes/icecc.bbclass | 4 ++--
>  1 file changed, 2 insertions(+), 2 deletions(-)
> 
> diff --git a/meta/classes/icecc.bbclass b/meta/classes/icecc.bbclass
> index 8a48f2ad63..e3b07028b8 100644
> --- a/meta/classes/icecc.bbclass
> +++ b/meta/classes/icecc.bbclass
> @@ -143,8 +143,8 @@ def use_icecc(bb,d):
>      if icecc_is_cross_canadian(bb, d):
>          return "no"
>  
> -    pn = d.getVar('PN')
> -    bpn = d.getVar('BPN')
> +    pn = bb.parse.vars_from_file(d.getVar('FILE', False),d)[0] or 'defaultpkgname'
> +    bpn = oe.utils.prune_suffix(pn, d.getVar('SPECIAL_PKGSUFFIX').split(), d)
>  
>      # Enable/disable checks are made against BPN, because there is a good
>      # chance that if icecc should be skipped for a recipe, it should be skipped

Unfortunately PN could be set differently to that, or set through class
extensions so I don't think this is a good way to fix the issue. It
does highlight that icecc probably just worked through luck anyway. It
would get it approximately right most of the time but I'm not sure we
want to rely on that.

To be honest, I'm more tempted to simply remove icecc since it would
seem not a lot of people are using it if it takes this long for fixes
to get sent :/.

Cheers,

Richard

Re: [OE-core] [PATCH 1/1] icecc: calculate pn and bpn from FILE

[Richard Purdie]

On Tue, 2025-06-03 at 10:40 +0100, Richard Purdie via lists.openembedded.org wrote:
> On Tue, 2025-06-03 at 10:58 +0200, Peter Suti via lists.openembedded.org wrote:
> > Starting with Yocto Mickledore the PN and BPN variables are set to "no-pn"
> > at this point in the `use_icecc()` function, so instead we need to
> > re-parse them from the file again as a workaround otherwise icecc is broken.
> > 
> > Fixes: 3be00ad9052de ("bitbake.conf: Add BB_HASH_CODEPARSER_VALS")
> > Where "PN=nopn" is set to optimise the codeparser cache's size.
> > 
> > [0] https://lists.yoctoproject.org/g/yocto/topic/icecc_support_broken/103429714
> > 
> > Signed-off-by: Peter Suti <peter.suti@streamunlimited.com>
> > ---
> >  meta/classes/icecc.bbclass | 4 ++--
> >  1 file changed, 2 insertions(+), 2 deletions(-)
> > 
> > diff --git a/meta/classes/icecc.bbclass b/meta/classes/icecc.bbclass
> > index 8a48f2ad63..e3b07028b8 100644
> > --- a/meta/classes/icecc.bbclass
> > +++ b/meta/classes/icecc.bbclass
> > @@ -143,8 +143,8 @@ def use_icecc(bb,d):
> >      if icecc_is_cross_canadian(bb, d):
> >          return "no"
> >  
> > -    pn = d.getVar('PN')
> > -    bpn = d.getVar('BPN')
> > +    pn = bb.parse.vars_from_file(d.getVar('FILE', False),d)[0] or 'defaultpkgname'
> > +    bpn = oe.utils.prune_suffix(pn, d.getVar('SPECIAL_PKGSUFFIX').split(), d)
> >  
> >      # Enable/disable checks are made against BPN, because there is a good
> >      # chance that if icecc should be skipped for a recipe, it should be skipped
> 
> Unfortunately PN could be set differently to that, or set through class
> extensions so I don't think this is a good way to fix the issue. It
> does highlight that icecc probably just worked through luck anyway. It
> would get it approximately right most of the time but I'm not sure we
> want to rely on that.
> 
> To be honest, I'm more tempted to simply remove icecc since it would
> seem not a lot of people are using it if it takes this long for fixes
> to get sent :/.

Just to clarify, this would then let someone maintain this in a
separate layer as it is self contained. I appreciate we'd need to find
that maintainer someone needs to fill that role to keep this working...

Cheers,

Richard

Re: [OE-core] [PATCH 1/1] icecc: calculate pn and bpn from FILE

[Peter Suti]

On Tue, Jun 3, 2025 at 11:59 AM Richard Purdie
<richard.purdie@linuxfoundation.org> wrote:
>
> On Tue, 2025-06-03 at 10:40 +0100, Richard Purdie via lists.openembedded.org wrote:
> > Unfortunately PN could be set differently to that, or set through class
> > extensions so I don't think this is a good way to fix the issue. It
> > does highlight that icecc probably just worked through luck anyway. It
> > would get it approximately right most of the time but I'm not sure we
> > want to rely on that.

Maybe that is why we have to disable a bunch of recipes with
ICECC_RECIPE_DISABLE.

> > To be honest, I'm more tempted to simply remove icecc since it would
> > seem not a lot of people are using it if it takes this long for fixes
> > to get sent :/.

I understand. We will keep using it internally with the above patch
because it works for our use cases and
provides noticeable speedups still.

> Just to clarify, this would then let someone maintain this in a
> separate layer as it is self contained. I appreciate we'd need to find
> that maintainer someone needs to fill that role to keep this working...
>
> Cheers,
>
> Richard

Thank you for taking a look and the quick response!

Best Regards,
Peter

Re: [OE-core] [PATCH 1/1] icecc: calculate pn and bpn from FILE

[Douglas Royds]

From:  Peter Suti via lists.openembedded.org <peter.suti=streamunlimited.com@lists.openembedded.org>
> On Tue, Jun 3, 2025 at 11:59 AM Richard Purdie <richard.purdie@linuxfoundation.org> wrote:
> >
> > On Tue, 2025-06-03 at 10:40 +0100, Richard Purdie via lists.openembedded.org wrote:
> > > Unfortunately PN could be set differently to that, or set through class
> > > extensions so I don't think this is a good way to fix the issue. It
> > > does highlight that icecc probably just worked through luck anyway. It
> > > would get it approximately right most of the time but I'm not sure we
> > > want to rely on that.
> 
> Maybe that is why we have to disable a bunch of recipes with
> ICECC_RECIPE_DISABLE.
> 
> > > To be honest, I'm more tempted to simply remove icecc since it would
> > > seem not a lot of people are using it if it takes this long for fixes
> > > to get sent :/.
> 
> I understand. We will keep using it internally with the above patch
> because it works for our use cases and
> provides noticeable speedups still.
> 
> > Just to clarify, this would then let someone maintain this in a
> > separate layer as it is self contained. I appreciate we'd need to find
> > that maintainer someone needs to fill that role to keep this working...

I suspect the reason no one has fixed this might be that no one else has hit it
yet — we haven't. Fortunately in the meantime, Peter has found a work-around
that works for him.

I'd argue that icecc is well supported in oe-core. These are the top 6
contributors to icecc.bbclass, making 97 commits over 10 years, almost 10 per
year:

   21 Joshua Watt
   18 Richard Purdie
   14 Martin Jansa
   11 Douglas Royds
   9 Tobias Henkel
   6 Dmitry Eremin-Solenikov

I'm sure one of these contributors will bump into this same problem at some
point, and that they'll fix it then. Please don't drop it from oe-core.

Re: [OE-core] [PATCH 1/1] icecc: calculate pn and bpn from FILE

[Richard Purdie]

On Tue, 2025-06-03 at 21:55 +0000, Douglas Royds wrote:
> From:  Peter Suti via lists.openembedded.org
> <peter.suti=streamunlimited.com@lists.openembedded.org>
> > On Tue, Jun 3, 2025 at 11:59 AM Richard Purdie
> > <richard.purdie@linuxfoundation.org> wrote:
> > > 
> > > On Tue, 2025-06-03 at 10:40 +0100, Richard Purdie via
> > > lists.openembedded.org wrote:
> > > > Unfortunately PN could be set differently to that, or set
> > > > through class
> > > > extensions so I don't think this is a good way to fix the
> > > > issue. It
> > > > does highlight that icecc probably just worked through luck
> > > > anyway. It
> > > > would get it approximately right most of the time but I'm not
> > > > sure we
> > > > want to rely on that.
> > 
> > Maybe that is why we have to disable a bunch of recipes with
> > ICECC_RECIPE_DISABLE.
> > 
> > > > To be honest, I'm more tempted to simply remove icecc since it
> > > > would
> > > > seem not a lot of people are using it if it takes this long for
> > > > fixes
> > > > to get sent :/.
> > 
> > I understand. We will keep using it internally with the above patch
> > because it works for our use cases and
> > provides noticeable speedups still.
> > 
> > > Just to clarify, this would then let someone maintain this in a
> > > separate layer as it is self contained. I appreciate we'd need to
> > > find
> > > that maintainer someone needs to fill that role to keep this
> > > working...
> 
> I suspect the reason no one has fixed this might be that no one else
> has hit it yet — we haven't. Fortunately in the meantime, Peter has
> found a work-around that works for him.

Which branches are you using this with?

> I'd argue that icecc is well supported in oe-core. These are the top
> 6 contributors to icecc.bbclass, making 97 commits over 10 years,
> almost 10 per year:
> 
>    21 Joshua Watt
>    18 Richard Purdie
>    14 Martin Jansa
>    11 Douglas Royds
>    9 Tobias Henkel
>    6 Dmitry Eremin-Solenikov
> 
> I'm sure one of these contributors will bump into this same problem
> at some point, and that they'll fix it then. Please don't drop it
> from oe-core.

I can safely say I simply don't use it and never have, which hints at
the kinds of commits I'm responsible for. Joshua used to but doesn't
now and hasn't for a few years. As far as I know, Martin doesn't any
more either. I'm therefore not sure this data makes a good case for it
being well supported.

At this point I think it belongs in its own layer. I appreciate people
don't want to hear that and yes, that means someone will have to
actually step up maintain it rather than relying on me and others
somehow keeping it working mostly by luck.

Cheers,

Richard

Re: [OE-core] [PATCH 1/1] icecc: calculate pn and bpn from FILE

[Joshua Watt]

On Tue, Jun 3, 2025 at 4:15 PM Richard Purdie via
lists.openembedded.org
<richard.purdie=linuxfoundation.org@lists.openembedded.org> wrote:
>
> On Tue, 2025-06-03 at 21:55 +0000, Douglas Royds wrote:
> > From:  Peter Suti via lists.openembedded.org
> > <peter.suti=streamunlimited.com@lists.openembedded.org>
> > > On Tue, Jun 3, 2025 at 11:59 AM Richard Purdie
> > > <richard.purdie@linuxfoundation.org> wrote:
> > > >
> > > > On Tue, 2025-06-03 at 10:40 +0100, Richard Purdie via
> > > > lists.openembedded.org wrote:
> > > > > Unfortunately PN could be set differently to that, or set
> > > > > through class
> > > > > extensions so I don't think this is a good way to fix the
> > > > > issue. It
> > > > > does highlight that icecc probably just worked through luck
> > > > > anyway. It
> > > > > would get it approximately right most of the time but I'm not
> > > > > sure we
> > > > > want to rely on that.
> > >
> > > Maybe that is why we have to disable a bunch of recipes with
> > > ICECC_RECIPE_DISABLE.
> > >
> > > > > To be honest, I'm more tempted to simply remove icecc since it
> > > > > would
> > > > > seem not a lot of people are using it if it takes this long for
> > > > > fixes
> > > > > to get sent :/.
> > >
> > > I understand. We will keep using it internally with the above patch
> > > because it works for our use cases and
> > > provides noticeable speedups still.
> > >
> > > > Just to clarify, this would then let someone maintain this in a
> > > > separate layer as it is self contained. I appreciate we'd need to
> > > > find
> > > > that maintainer someone needs to fill that role to keep this
> > > > working...
> >
> > I suspect the reason no one has fixed this might be that no one else
> > has hit it yet — we haven't. Fortunately in the meantime, Peter has
> > found a work-around that works for him.
>
> Which branches are you using this with?
>
> > I'd argue that icecc is well supported in oe-core. These are the top
> > 6 contributors to icecc.bbclass, making 97 commits over 10 years,
> > almost 10 per year:
> >
> >    21 Joshua Watt
> >    18 Richard Purdie
> >    14 Martin Jansa
> >    11 Douglas Royds
> >    9 Tobias Henkel
> >    6 Dmitry Eremin-Solenikov
> >
> > I'm sure one of these contributors will bump into this same problem
> > at some point, and that they'll fix it then. Please don't drop it
> > from oe-core.
>
> I can safely say I simply don't use it and never have, which hints at
> the kinds of commits I'm responsible for. Joshua used to but doesn't
> now and hasn't for a few years. As far as I know, Martin doesn't any
> more either. I'm therefore not sure this data makes a good case for it
> being well supported.
>
> At this point I think it belongs in its own layer. I appreciate people
> don't want to hear that and yes, that means someone will have to
> actually step up maintain it rather than relying on me and others
> somehow keeping it working mostly by luck.

I agree. icecc can be quite useful, but it is one of those things that
is basically impossible to test unless you are actively using it (e.g.
it can't be easily tested on the autobuilder). I don't use it anymore,
and I don't even have a setup to test it on, so I can't really provide
much help on keeping it functional.

>
> Cheers,
>
> Richard
>
> -=-=-=-=-=-=-=-=-=-=-=-
> Links: You receive all messages sent to this group.
> View/Reply Online (#217828): https://lists.openembedded.org/g/openembedded-core/message/217828
> Mute This Topic: https://lists.openembedded.org/mt/113444145/3616693
> Group Owner: openembedded-core+owner@lists.openembedded.org
> Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [JPEWhacker@gmail.com]
> -=-=-=-=-=-=-=-=-=-=-=-
>