Using pigz-native from sstate-cache while zlib-native is not there yet

Using pigz-native from sstate-cache while zlib-native is not there yet

[Riku Voipio]

Hi,

I've started observing build failures with pigz-native fails due
ttrying to use too old libz (from host), instead of the one in
sysroot. I take most don't see this bug because they have new enough
zlib on their host system.

| DEBUG: Executing shell function sstate_create_package
| gzip: /lib/x86_64-linux-gnu/libz.so.1: version `ZLIB_1.2.5.1' not
found (required by gzip)
| tar: Child returned status 1
| tar: Error is not recoverable: exiting now
| WARNING: exit code 2 from a shell command.
| DEBUG: Python function sstate_task_postfunc finished
| ERROR: Function failed: sstate_create_package

This seems to happen, because

1) pigz-native is extracted from sstate cache
2) zlib-native is outdated, and it's compilation is being started
3) another task trying to use gzip/pigz meanwhile

Simply setting DEPENDS_class-native = "zlib-native" in pigz.bb didn't
seem to help. These errors started around September 3rd, and it
unclear to me what exactly changed that exposed thise bug.

Full build log at:

https://ci.linaro.org/jenkins/job/openembedded-armv8-rootfs/label=oe_persistent_cloud,rootfs=lamp/484/consoleText

Any hints howto fix this would be appreciated,
Riku

Re: Using pigz-native from sstate-cache while zlib-native is not there yet

[Richard Purdie]

On Tue, 2013-09-10 at 17:55 +0300, Riku Voipio wrote:
> I've started observing build failures with pigz-native fails due
> ttrying to use too old libz (from host), instead of the one in
> sysroot. I take most don't see this bug because they have new enough
> zlib on their host system.
> 
> | DEBUG: Executing shell function sstate_create_package
> | gzip: /lib/x86_64-linux-gnu/libz.so.1: version `ZLIB_1.2.5.1' not
> found (required by gzip)
> | tar: Child returned status 1
> | tar: Error is not recoverable: exiting now
> | WARNING: exit code 2 from a shell command.
> | DEBUG: Python function sstate_task_postfunc finished
> | ERROR: Function failed: sstate_create_package
> 
> This seems to happen, because
> 
> 1) pigz-native is extracted from sstate cache
> 2) zlib-native is outdated, and it's compilation is being started
> 3) another task trying to use gzip/pigz meanwhile
> 
> Simply setting DEPENDS_class-native = "zlib-native" in pigz.bb didn't
> seem to help. These errors started around September 3rd, and it
> unclear to me what exactly changed that exposed thise bug.
> 
> Full build log at:
> 
> https://ci.linaro.org/jenkins/job/openembedded-armv8-rootfs/label=oe_persistent_cloud,rootfs=lamp/484/consoleText
> 
> Any hints howto fix this would be appreciated,

We also just saw this on the autobuilder which reminded me of this
email. I did look at the issue at the time but didn't understand it.
Combining what I just looked at in a build and what you reported, I
think I might have spotted a pattern now though. What happens is as you
describe:

pigz-native is used from sstate.
zlib-native is getting rebuilt.

pigz-native has some special handling to ensure its not used until the
system is ready. This is through a class and installing into a subdir of
PATH which only gets added in when we believe its available. We use
pigz-native in the image generation code and its in DEPENDS.

DEPENDS are guaranteed available for do_configure. do_populate_lic can
run before do_configure so the DEPENDS isn't met and I think this is our
corner case.

I suspect ways of fixing this are to either:

a) force do_populate_lic after do_configure everywhere
b) statically link pigz-native
c) add in an explicit dependency to gzipnative.bbclass forcing
do_populate_lic after do_configure. If do_unpack handled a tar file in
an image, it would also be at risk of course.

Looking at each, a) is overkill and our dependency tree is nasty enough
already. b) sounds nice but is also risky since what happens if the gzip
binary is half copied when we run it. Our hardlink copying should deal
with that but I'm still nervous. This leaves us with c) so we could do:

do_unpack[depends] += "gzip-native:do_populate_sysroot"

which I think should address the problem?

If anyone is interested, the reproducer is:

bitbake pigz-native
bitbake zlib-native -c clean
bitbake core-image-minimal -c populate_lic --no-setscene -f

however your system needs to have an old version of zlib on it which
pigz-native can't run against. The line above fixes it.

Cheers,

Richard

Re: Using pigz-native from sstate-cache while zlib-native is not there yet

[Riku Voipio]

Hi Richard,

On 21 September 2013 12:54, Richard Purdie
> do_unpack[depends] += "gzip-native:do_populate_sysroot"
>
> which I think should address the problem?
>
> If anyone is interested, the reproducer is:
>
> bitbake pigz-native
> bitbake zlib-native -c clean
> bitbake core-image-minimal -c populate_lic --no-setscene -f

> however your system needs to have an old version of zlib on it which
> pigz-native can't run against. The line above fixes it.

I verified on ubuntu precise that the above steps do not cause the
zlib relocation error anymore. I see the fix is alreadt in OE-core,
thanks!

Riku