Re: [OE-core] [PATCH v11] util-linux: split uuid in separate recipe to allow bootstrapping

["Scott Branden" <scott.branden@broadcom.com>]

[-- Attachment #1: Type: text/plain, Size: 5176 bytes --]

I have not debugged yet, but I suspect this change is causing the following failure if I merge the latest poky into our builds:

ERROR: mc:host:util-linux-uuid-2.36.2-r0 do_package: QA Issue: util-linux-uuid: Files/directories were installed but not shipped in any package:
  /usr/lib/debug
  /usr/lib/debug/usr
  /usr/lib/debug/usr/lib
  /usr/lib/debug/usr/lib/libuuid.so.1.3.0.debug
Please set FILES such that these items are packaged. Alternatively if they are unneeded, avoid installing them or delete them within do_install.
util-linux-uuid: 4 installed and not shipped files. [installed-vs-shipped]
ERROR: mc:host:util-linux-uuid-2.36.2-r0 do_package: Fatal QA errors found, failing task.
ERROR: Logfile of failure stored in: /hdd/yocto/genx/poky/build/tmp/work/core2-64-poky-linux/util-linux-uuid/2.36.2-r0/temp/log.do_package.31753
ERROR: Task (mc:host:/hdd/yocto/genx/poky/build/../meta/recipes-core/util-linux/util-linux-uuid_2.36.2.bb:do_package) failed with exit code '1'

On 2021-03-15 2:51 p.m., Richard Purdie wrote:
> On Mon, 2021-03-15 at 13:57 +0000, Richard Purdie via lists.openembedded.org wrote:
>> On Mon, 2021-03-15 at 14:55 +0100, Martin Jansa wrote:
>>> On Mon, Mar 15, 2021 at 12:21:37PM +0000, Richard Purdie wrote:
>>>> On Mon, 2021-03-15 at 11:50 +0000, Luca Boccassi wrote:
>>>>> On Mon, 2021-03-15 at 10:49 +0000, Richard Purdie wrote:
>>>>>> On Mon, 2021-03-15 at 10:44 +0000, Luca Boccassi wrote:
>>>>>>> On Sun, 2021-03-14 at 22:10 +0000, Richard Purdie wrote:
>>>>>>>> On Thu, 2021-03-11 at 15:09 +0000, luca.boccassi@gmail.com wrote:
>>>>>>>>> From: Luca Boccassi <luca.boccassi@microsoft.com>
>>>>>>>>>
>>>>>>>>> Recently util-linux gained an (optional) build dependency on libcryptsetup.
>>>>>>>>> But libcryptsetup build-depends on util-linux for blkid (optional, can be disabled)
>>>>>>>>> and uuid (mandatory).
>>>>>>>>> Split out util-linux-uuid in a different recipe to break the cycle.
>>>>>>>>>
>>>>>>>>> https://github.com/karelzak/util-linux/pull/898
>>>>>>>>>
>>>>>>>>> Signed-off-by: Luca Boccassi <luca.boccassi@microsoft.com>
>>>>>>>>
>>>>>>>> Unfortunately I noticed we had a performance regression in buildtimes in 
>>>>>>>> recent changes. The closest I have this narrowed down to so far:
>>>>>>>>
>>>>>>>> https://autobuilder.yocto.io/pub/non-release/20210314-14/testresults/buildperf-ubuntu1604/perf-ubuntu1604_master_20210314181831_d42487bf52.html
>>>>>>>>
>>>>>>>> suggests it may be this change. I have more tests queued to confirm
>>>>>>>> that definitively, if so we'll have to figure out why as this shouldn't
>>>>>>>> really happen, its an 8% regression :(.
>>>>>>>
>>>>>>> Very strange that a single recipe could do that - is there something
>>>>>>> wrong in the new .bb that I missed and could cause this?
>>>>>>
>>>>>> I'm wondering if it is because we're building util-linux twice now and
>>>>>> there is some key choke point in the dependency chain. I have no evidence
>>>>>> for that yet, it is just speculation though.
>>>>>
>>>>> With the autoconf options I've set, on my laptop it takes 32s to do
>>>>> configure + make -j2. Most of that is autoconf - make -j2 takes 8s.
>>>>>
>>>>> Only 3 libraries are built with this combination: libcommon.a,
>>>>> libtcolors.a, and libuuid.a/so. No executables or anything else is
>>>>> built. It doesn't look like libtcolors is actually needed, I'll see if
>>>>> I can prepare a patch to skip it, but I don't think it will buy more
>>>>> than 1s, it's just two object files.
>>>>>
>>>>> The good news is that meson support is about to land upstream, which
>>>>> should be significantly faster than autoconf + make:
>>>>>
>>>>> https://github.com/karelzak/util-linux/commits/topic/meson
>>>>
>>>> Meson definitely improves the speed! I was wondering if it was from
>>>> configure for example.
>>>>
>>>> I now have more performance test results in (takes time to interleave 
>>>> them with testing of master):
>>>>
>>>> https://autobuilder.yocto.io/pub/non-release/20210315-1/testresults/buildperf-ubuntu1604/perf-ubuntu1604_master_20210315005048_6bb1621815.html
>>>>
>>>> and I think this means it isn't from the util-linux change but one of 
>>>> another three. I'm not entirely convinced those changes could do this
>>>> but it is what the data says. 
>>>>
>>>> I've queued more bisection to narrow it down from there...
>>>
>>> BTW: this split also needs manual cleanup in the TMPDIR, right?
>>
>> It shouldn't. The system should spot that util-linux has changed and uninstall
>> it from the sysroots as it goes. There is something not working right there :(
> 
> I was wrong about that, the system doesn't have code for this, it has code for
> the sysroots but not for other sstate tasks.
> 
> I think this is an oversight and we need to simplify things and make this cleanup
> happen pre-build, much like the "unreachable" tasks cleanup happens today. If we
> do make this happen, we probably need to add parallelism as the number of stale
> sstate tasks being cleaned could be substantial.
> 
> Cheers,
> 
> Richard
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 


[-- Attachment #2: S/MIME Cryptographic Signature --]
[-- Type: application/pkcs7-signature, Size: 4212 bytes --]