Re: [OE-core] [PATCH v3] util-linux: split uuid in separate recipe to allow bootstrapping

["Luca Boccassi" <luca.boccassi@microsoft.com>]

On Thu, 2020-12-10 at 20:04 +0000, Richard Purdie wrote:
> On Thu, 2020-12-10 at 18:47 +0000, Luca Boccassi wrote:
> > On Thu, 2020-12-10 at 15:52 +0000, Richard Purdie wrote:
> > > On Mon, 2020-11-23 at 13:28 +0000, Luca Bocassi wrote:
> > > > From: Luca Boccassi <luca.boccassi@microsoft.com>
> > > > 
> > > > In v2.35 util-linux  gained an (optional) build
> > > > dependency on libcryptsetup. But libcryptsetup build-depends on
> > > > util-linux for blkid (optional, can be disabled) and uuid
> > > > (mandatory).
> > > > Split out util-linux-uuid in a different recipe to break the
> > > > cycle.
> > > > 
> > > > Add a packageconfig switch (disabled by default) to allow using
> > > > the
> > > > new dependency.
> > > > 
> > > > https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fkarelzak%2Futil-linux%2Fpull%2F898&amp;data=04%7C01%7CLuca.Boccassi%40microsoft.com%7Cf1cc5d5a36b94793f16708d89d46dfac%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637432275624402419%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C2000&amp;sdata=k69BQ6hOpusjBlcC%2FBo%2BFRLsAc%2FiYd5pLPfh65lcRdQ%3D&amp;reserved=0
> > > > 
> > > > Signed-off-by: Luca Boccassi <luca.boccassi@microsoft.com>
> > > > ---
> > > > v1: util-linux 2.35 is not out yet, but I'd like to get the
> > > > preparatory work
> > > >     underway as I'm not sure if this is the best approach or if
> > > > there are
> > > >     alternatives. Suggestions and comments very welcome. Thanks!
> > > > v2: changed packages names to reflect old ones (eg: libuuid1 ->
> > > > util-linux-libuuid)
> > > >     and leave uuid build enable in main recipe to allow for
> > > > uuidgen build to happen,
> > > >     as it does not have its own autoconf switch. Delete the
> > > > library manualy from
> > > >     the main recipe after build instead, and add dependency.
> > > >     Might help to break loop python3 -> util-linux -> libselinux
> > > > -> python3, as it's
> > > >     only libuuid that is needed, see 
> > > > https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Flists.yoctoproject.org%2Fg%2Fyocto%2Fmessage%2F47570&amp;data=04%7C01%7CLuca.Boccassi%40microsoft.com%7Cf1cc5d5a36b94793f16708d89d46dfac%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637432275624402419%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C2000&amp;sdata=poH8DFxARF1Myy%2BRRirbOE3bJzu5uSO%2F%2BLfsalK%2F4ro%3D&amp;reserved=0
> > > > v3: rebased and refactored to have a common util-linux.inc file
> > > > 
> > > 
> > > I'm afraid this causes do_package_qa errors in basic testing:
> > > 
> > > https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fautobuilder.yoctoproject.org%2Ftyphoon%2F%23%2Fbuilders%2F83%2Fbuilds%2F1668&amp;data=04%7C01%7CLuca.Boccassi%40microsoft.com%7Cf1cc5d5a36b94793f16708d89d46dfac%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637432275624402419%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C2000&amp;sdata=uiLItrvBcoNjjjCFZaUlQfC1%2B26N%2BnV72BUiy%2FuBq0M%3D&amp;reserved=0
> > 
> > Thanks, added RDEPENDS in v4. Strange that those didn't pop up when
> > building poky locally, I usually get QA warnings as expected.
> > Anything
> > in local.conf to enable to get them?
> 
> No, you should see it in a standard build. It did make me wonder how
> this was tested :/.

Very weird. Maybe some packageconfig difference? I did not go through
all the possible permutations manually.

We have been using this recipe change in production for a year now, so
it has plenty of builds behind.

> > > I am worried about this change as we're starting to see a number of
> > > circular dependencies in util-linux (there is a new bug about the
> > > pylibmount PACKAGECONFIG option too), maybe we should flag this
> > > upstream?
> > > 
> > > Multiple recipes like this usually turn into a maintenance
> > > nightmare
> > > unfortunately which is part of my reluctance to go in this
> > > direction,
> > > not sure we have any choice though.
> > 
> > Well I've added the feature, and both the maintainer and myself were
> > aware of the implications. It's optional, so on distros with multi-
> > stage bootstrapping functionality like Debian/Ubuntu/RHEL/Suse/etc it
> > can be automatically disabled for the first stage build. At runtime
> > it can also be optional via dlopen, if desired (via --configure
> > flag).
> 
> I have to ask why libuuid couldn't be done in a separate repository and
> avoid the need to do a multi-stage build of a component? To me at
> least, it would seem to make sense to logically split the library code
> out, then it avoids all the complexity. Yes, that means a different
> component to release but that isn't unusual.

Because there's no need for the extra complications - again, it's all
optional features, so bootstrapping is not an issue when the tooling is
there to support it.
I'm not a util-linux maintainer so my opinion on the subject counts for
precisely nothing, but as a contributor and user I'd not be very happy
if it was stuck to the lowest common denominator.

> > Yocto could really use multi stage support - this isn't the first and
> > won't be the last occurrence. Just my 2c...
> 
> Well, we can do it as you're proving, its just ugly and hard to
> maintain. I don't think the other distros will be particularly happy
> about needing to do it either. Outside of libgcc, we've not really
> found that we need to do this often at all and the compiler/libc
> interface is a lot more "special" than uuid.

But that's what I'm saying: it doesn't have to be ugly, if the
infrastructure is there to support it.

On Debian and derivatives, you just mark the dependency with <!stage1>
- and that's it. When bootstrapping you start from stage1 and the
resolver skips those. If the package configure/make scripts are done
well, by default optional dependencies are skipped if not available and
if not explicitly set - and util-linux does that.
In the RPM world, the spec has conditional macros and you set the
appropriate one at the build config level (eg: in the lower ring
project on OBS).
It's not perfect of course, and requires attention, and there are
complications and gotchas, and things do go wrong at times - but such
is life in the software world.


> Thanks for updating the patch. I'll put it back into the queue and test
> the new version.

Thank you - does the approach of adding RDEPENDS look right? The
interaction between those variables and the native/nativesdk builds
still confuses me a lot, and I get it wrong all the time.

-- 
Kind regards,
Luca Boccassi