RaspberryPi Kernel - sometimes it works, sometimes it doesn't

RaspberryPi Kernel - sometimes it works, sometimes it doesn't

[Chris Tapp]

I've been building the 3.1.9 Raspberry Pi kernel under Denzil using the meta layer at https://github.com/djwillis/meta-raspberrypi. This uses a kernel recipe based on the git repository at https://github.com/raspberrypi/linux/commits/rpi-patches.

Some of the kernel commit IDs (e.g. 94fbbc4e3988075abad0d3b32842b82c590324fc) seem to build ok, but they don't always run. As in, if I -c clean and build then sometimes I end up with a bootable image, sometimes I don't. I'm not changing anything else in the build environment.

Any ideas what can cause this?

Chris Tapp

opensource@keylevel.com
www.keylevel.com

Re: RaspberryPi Kernel - sometimes it works, sometimes it doesn't

[Tomas Frydrych]

Hi,

On 10/06/12 12:30, Chris Tapp wrote:
> I've been building the 3.1.9 Raspberry Pi kernel under Denzil using
> the meta layer at https://github.com/djwillis/meta-raspberrypi. This
> uses a kernel recipe based on the git repository at
> https://github.com/raspberrypi/linux/commits/rpi-patches.
> 
> Some of the kernel commit IDs (e.g.
> 94fbbc4e3988075abad0d3b32842b82c590324fc) seem to build ok, but they
> don't always run. As in, if I -c clean and build then sometimes I end
> up with a bootable image, sometimes I don't. I'm not changing
> anything else in the build environment.
> 
> Any ideas what can cause this?

I find that -c clean does not work very well, afterward the package gets
recompiled but instead of the actual package packages being rebuilt, an
earlier version of the packages gets pulled out of sstate into the
image. I definitely saw this behaviour with Yocto kernels, but I think
happens with other packages as well; I always do -c cleansstate now to
avoid this.

Tomas

Re: RaspberryPi Kernel - sometimes it works, sometimes it doesn't

[Chris Tapp]

On 11 Jun 2012, at 08:29, Tomas Frydrych wrote:

> Hi,
> 
> On 10/06/12 12:30, Chris Tapp wrote:
>> I've been building the 3.1.9 Raspberry Pi kernel under Denzil using
>> the meta layer at https://github.com/djwillis/meta-raspberrypi. This
>> uses a kernel recipe based on the git repository at
>> https://github.com/raspberrypi/linux/commits/rpi-patches.
>> 
>> Some of the kernel commit IDs (e.g.
>> 94fbbc4e3988075abad0d3b32842b82c590324fc) seem to build ok, but they
>> don't always run. As in, if I -c clean and build then sometimes I end
>> up with a bootable image, sometimes I don't. I'm not changing
>> anything else in the build environment.
>> 
>> Any ideas what can cause this?
> 
> I find that -c clean does not work very well, afterward the package gets
> recompiled but instead of the actual package packages being rebuilt, an
> earlier version of the packages gets pulled out of sstate into the
> image. I definitely saw this behaviour with Yocto kernels, but I think
> happens with other packages as well; I always do -c cleansstate now to
> avoid this.


Yes, I've seen that in the past as well. I should probably have mentioned that I've also tried cleanstate and/or deleting any residual sstate-cache/sstate- files for linux-raspberrypi, etc.

Chris Tapp

opensource@keylevel.com
www.keylevel.com

Re: RaspberryPi Kernel - sometimes it works, sometimes it doesn't

[Khem Raj]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On 6/11/2012 12:29 AM, Tomas Frydrych wrote:
> Hi,
> 
> On 10/06/12 12:30, Chris Tapp wrote:
>> I've been building the 3.1.9 Raspberry Pi kernel under Denzil
>> using the meta layer at
>> https://github.com/djwillis/meta-raspberrypi. This uses a kernel
>> recipe based on the git repository at 
>> https://github.com/raspberrypi/linux/commits/rpi-patches.
>> 
>> Some of the kernel commit IDs (e.g. 
>> 94fbbc4e3988075abad0d3b32842b82c590324fc) seem to build ok, but
>> they don't always run. As in, if I -c clean and build then
>> sometimes I end up with a bootable image, sometimes I don't. I'm
>> not changing anything else in the build environment.
>> 
>> Any ideas what can cause this?
> 
> I find that -c clean does not work very well, afterward the package
> gets recompiled but instead of the actual package packages being
> rebuilt, an earlier version of the packages gets pulled out of
> sstate into the image. I definitely saw this behaviour with Yocto
> kernels, but I think happens with other packages as well; I always
> do -c cleansstate now to avoid this.

yes thats the intended behavior if nothing changed that ensues a
recompile then it will use precompiled sstate for the package

> 
> Tomas _______________________________________________ yocto mailing
> list yocto@yoctoproject.org 
> https://lists.yoctoproject.org/listinfo/yocto

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.12 (MingW32)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/

iEYEARECAAYFAk/V+FwACgkQuwUzVZGdMxRlnwCfaIy1wo/G0gEIVbQQe0ImsKPa
2csAoJGhol1N0xrudHfT/c7T97Hoizhf
=l1iC
-----END PGP SIGNATURE-----

Re: RaspberryPi Kernel - sometimes it works, sometimes it doesn't

[Paul Eggleton]

On Monday 11 June 2012 06:53:32 Khem Raj wrote:
> On 6/11/2012 12:29 AM, Tomas Frydrych wrote:
> > I find that -c clean does not work very well, afterward the package
> > gets recompiled but instead of the actual package packages being
> > rebuilt, an earlier version of the packages gets pulled out of
> > sstate into the image. I definitely saw this behaviour with Yocto
> > kernels, but I think happens with other packages as well; I always
> > do -c cleansstate now to avoid this.
> 
> yes thats the intended behavior if nothing changed that ensues a
> recompile then it will use precompiled sstate for the package

To clarify, it's designed to be able to determine when the recipe has changed 
in such a way that it needs to be rebuilt (by building up dependencies between 
variables and computing a checksum of the value of each one, which ends up in 
a checksum for each task); if it sees no change then the previous task output 
will be used from the sstate cache. It's worth noting however that until 
recently (post-1.2) we did not handle when local files referred to in SRC_URI 
changed. There also still may be other circumstances under which changes to 
the recipe or variables which it refers to will not change the sstate 
checksums; if you come across a situation where you made a change and sstate 
was re-used when you expected it to be rebuilt, please raise it.

Cheers,
Paul

-- 

Paul Eggleton
Intel Open Source Technology Centre

Re: RaspberryPi Kernel - sometimes it works, sometimes it doesn't

[Tomas Frydrych]

On 11/06/12 17:29, Paul Eggleton wrote:
> On Monday 11 June 2012 06:53:32 Khem Raj wrote:
>> On 6/11/2012 12:29 AM, Tomas Frydrych wrote:
>>> I find that -c clean does not work very well, afterward the package
>>> gets recompiled but instead of the actual package packages being
>>> rebuilt, an earlier version of the packages gets pulled out of
>>> sstate into the image. I definitely saw this behaviour with Yocto
>>> kernels, but I think happens with other packages as well; I always
>>> do -c cleansstate now to avoid this.
>>
>> yes thats the intended behavior if nothing changed that ensues a
>> recompile then it will use precompiled sstate for the package
> 
> To clarify, it's designed to be able to determine when the recipe has changed 
> in such a way that it needs to be rebuilt (by building up dependencies between 
> variables and computing a checksum of the value of each one, which ends up in 
> a checksum for each task); if it sees no change then the previous task output 
> will be used from the sstate cache. It's worth noting however that until 
> recently (post-1.2) we did not handle when local files referred to in SRC_URI 
> changed. There also still may be other circumstances under which changes to 
> the recipe or variables which it refers to will not change the sstate 
> checksums; if you come across a situation where you made a change and sstate 
> was re-used when you expected it to be rebuilt, please raise it.

Over years of working with Poky I have developed this sort of a normal
work flow:

  bitbake -c devshell <package>
  < do some tweaking >
  bitbake -c compile -f <package>
  bitbake <package>          <---- this pulls package from sstate!!!
  scp ...
  < test, repeat>

This no longer works, even after a forced recompile, Poky just pulls a
package out of sstate. It seems the only reliable way to force a package
rebuild is either to cleansstate or bump the PR, neither of which is
viable an option in the above scenario. What am I missing? Is this
really intended?

Tomas

Re: RaspberryPi Kernel - sometimes it works, sometimes it doesn't

[Paul Eggleton]

On Tuesday 12 June 2012 08:23:54 Tomas Frydrych wrote:
> Over years of working with Poky I have developed this sort of a normal
> work flow:
> 
>   bitbake -c devshell <package>
>   < do some tweaking >
>   bitbake -c compile -f <package>
>   bitbake <package>          <---- this pulls package from sstate!!!
>   scp ...
>   < test, repeat>
> 
> This no longer works, even after a forced recompile, Poky just pulls a
> package out of sstate. It seems the only reliable way to force a package
> rebuild is either to cleansstate or bump the PR, neither of which is
> viable an option in the above scenario. What am I missing? Is this
> really intended?

I was surprised because this was not behaviour I would expect either, however 
that is indeed what it does here when I try that sequence. I'm not sure why it 
is behaving this way but I think it is a bug.

FWIW, we will be looking at fixing this exact workflow pretty soon although it 
may involve an extra explicit step to invalidate the stamps.

Cheers,
Paul

-- 

Paul Eggleton
Intel Open Source Technology Centre

Re: RaspberryPi Kernel - sometimes it works, sometimes it doesn't

[Gary Thomas]

On 2012-06-12 05:26, Paul Eggleton wrote:
> On Tuesday 12 June 2012 08:23:54 Tomas Frydrych wrote:
>> Over years of working with Poky I have developed this sort of a normal
>> work flow:
>>
>>    bitbake -c devshell<package>
>>    <  do some tweaking>
>>    bitbake -c compile -f<package>
>>    bitbake<package>           <---- this pulls package from sstate!!!
>>    scp ...
>>    <  test, repeat>
>>
>> This no longer works, even after a forced recompile, Poky just pulls a
>> package out of sstate. It seems the only reliable way to force a package
>> rebuild is either to cleansstate or bump the PR, neither of which is
>> viable an option in the above scenario. What am I missing? Is this
>> really intended?
>
> I was surprised because this was not behaviour I would expect either, however
> that is indeed what it does here when I try that sequence. I'm not sure why it
> is behaving this way but I think it is a bug.
>
> FWIW, we will be looking at fixing this exact workflow pretty soon although it
> may involve an extra explicit step to invalidate the stamps.

IMO, if you run a specific step like "-c compile -f", this should automatically
invalidate any stamps and sstate info for the package that depend on that step.
In this case, it should invalidate "install, package, ..." - everything that
normally happens after "compile".  This would fix the observed weirdness above.

-- 
------------------------------------------------------------
Gary Thomas                 |  Consulting for the
MLB Associates              |    Embedded world
------------------------------------------------------------

Re: RaspberryPi Kernel - sometimes it works, sometimes it doesn't

[Paul Eggleton]

On Tuesday 12 June 2012 05:30:34 Gary Thomas wrote:
> On 2012-06-12 05:26, Paul Eggleton wrote:
> > FWIW, we will be looking at fixing this exact workflow pretty soon
> > although it may involve an extra explicit step to invalidate the stamps.
> 
> IMO, if you run a specific step like "-c compile -f", this should
> automatically invalidate any stamps and sstate info for the package that
> depend on that step. In this case, it should invalidate "install, package,
> ..." - everything that normally happens after "compile".  This would fix
> the observed weirdness above.

FYI I'm going to track/fix this under bug 2256 since that is pretty much the 
same thing.

Cheers,
Paul

-- 

Paul Eggleton
Intel Open Source Technology Centre