[Buildroot] Proposed util-linux split

[Buildroot] Proposed util-linux split

[Gustavo Zacarias]

Hi all.
The last couple of days i've been thinking and working towards a
solution that would split out libblkid/libuuid out of the util-linux
package.
It's a remake of my previous "util-linux don't install binaries" patch.
The objectives are:

* Upgrading to newer versions of the libraries - it's not required by
any package, but it's useful for libblkid to recognize newer superblock
types.

* We can't upgrade util-linux past 2.20.1 unless we want to drop the
basic loginutils functionality since it requires PAM starting from 2.21.
By splitting the libraries out i can use just the libraries from newer
util-linux versions. Caveat is you won't get the newer libraries if
you're installing util-linux with the library options on.

* Reduce bloat! The most important point for me, one packages needs
libuuid and we get a dozen util-linux binaries around as a free gift.

I've got a working not-quite-ready (needs cleanup) patchset that adds
the libblkid and libuuid packages. These are based on a slightly patched
(configure.ac & Makefile.am) util-linux-2.22.2 that basically reduces
build time and keeps the original install target (you can't make -C
libblkid install with newer util-linux versions, see
http://karelzak.blogspot.com/2013/02/non-recursive-automake.html)

When a package needs libuuid it would now change to:
select BR2_PACKAGE_LIBUUID if !BR2_PACKAGE_UTIL_LINUX_LIBUUID
to avoid duplication.
And on libuuid Config.in:
depends on !BR2_PACKAGE_UTIL_LINUX_LIBUUID

This lets us keep util-linux around (which doesn't work with external
libblkid/libuuid because it doesn't expect them to be so) and slimmer
versions of the libraries when people don't want util-linux.

Another added bonus is libuuid doesn't require wchar this way.

What do you think?
Regards.

[Buildroot] Proposed util-linux split

[Thomas Petazzoni]

Dear Gustavo Zacarias,

On Tue, 26 Feb 2013 07:53:14 -0300, Gustavo Zacarias wrote:

> The last couple of days i've been thinking and working towards a
> solution that would split out libblkid/libuuid out of the util-linux
> package.
> It's a remake of my previous "util-linux don't install binaries" patch.
> The objectives are:
> 
> * Upgrading to newer versions of the libraries - it's not required by
> any package, but it's useful for libblkid to recognize newer superblock
> types.
> 
> * We can't upgrade util-linux past 2.20.1 unless we want to drop the
> basic loginutils functionality since it requires PAM starting from 2.21.
> By splitting the libraries out i can use just the libraries from newer
> util-linux versions. Caveat is you won't get the newer libraries if
> you're installing util-linux with the library options on.

I would suggest that we upgrade util-linux and add the necessary PAM
dependency when needed. We already have package/linux-pam/, so it
shouldn't be a big effort.

Of course, only the util-linux programs that actually need PAM support
should make this library become a dependency of util-linux.

> * Reduce bloat! The most important point for me, one packages needs
> libuuid and we get a dozen util-linux binaries around as a free gift.
> 
> I've got a working not-quite-ready (needs cleanup) patchset that adds
> the libblkid and libuuid packages. These are based on a slightly patched
> (configure.ac & Makefile.am) util-linux-2.22.2 that basically reduces
> build time and keeps the original install target (you can't make -C
> libblkid install with newer util-linux versions, see
> http://karelzak.blogspot.com/2013/02/non-recursive-automake.html)
> 
> When a package needs libuuid it would now change to:
> select BR2_PACKAGE_LIBUUID if !BR2_PACKAGE_UTIL_LINUX_LIBUUID
> to avoid duplication.
> And on libuuid Config.in:
> depends on !BR2_PACKAGE_UTIL_LINUX_LIBUUID

Would it be possible instead that util-linux always uses the libuuid
from the libuuid package?

> This lets us keep util-linux around (which doesn't work with external
> libblkid/libuuid because it doesn't expect them to be so) and slimmer
> versions of the libraries when people don't want util-linux.
> 
> Another added bonus is libuuid doesn't require wchar this way.
> 
> What do you think?

I must admit I don't really like having multiple packages for the same
upstream tarball. Another option is to make something like:

config BR2_PACKAGE_UTIL_LINUX
	bool "util-linux"
	select BR2_PACKAGE_UTIL_LINUX_LIBUUID if !BR2_PACKAGE_UTIL_LINUX_AT_LEAST_SOMETHING

config BR2_PACKAGE_UTIL_LINUX_AT_LEAST_SOMETHING
	bool

config BR2_PACKAGE_UTIL_LINUX_LIBUUID
	bool "libuuid"

config BR2_PACKAGE_UTIL_LINUX_LIBBLKDID
	bool "libblkid"
	select BR2_PACKAGE_UTIL_LINUX_AT_LEAST_SOMETHING

config BR2_PACKAGE_UTIL_LINUX_LIBMOUNT
	bool "libmount"
	select BR2_PACKAGE_UTIL_LINUX_AT_LEAST_SOMETHING

config BR2_PACKAGE_UTIL_LINUX_SOMEPROGA
	bool "program a"
	select BR2_PACKAGE_UTIL_LINUX_AT_LEAST_SOMETHING

This way, by default util-linux only installs libuuid.

No?

Thomas
-- 
Thomas Petazzoni, Free Electrons
Kernel, drivers, real-time and embedded Linux
development, consulting, training and support.
http://free-electrons.com

[Buildroot] Proposed util-linux split

[Thomas Petazzoni]

On Tue, 26 Feb 2013 19:33:30 +0100, Thomas Petazzoni wrote:

> I would suggest that we upgrade util-linux and add the necessary PAM
> dependency when needed. We already have package/linux-pam/, so it
> shouldn't be a big effort.
> 
> Of course, only the util-linux programs that actually need PAM support
> should make this library become a dependency of util-linux.

I just checked, and PAM is only needed when login-utils is enabled.

Also, 2.22 has apparently moved to a non-recursive usage of automake,
which should make the build much, much faster.

Best regards,

Thomas
-- 
Thomas Petazzoni, Free Electrons
Kernel, drivers, real-time and embedded Linux
development, consulting, training and support.
http://free-electrons.com

[Buildroot] Proposed util-linux split

[Gustavo Zacarias]

On 02/26/2013 03:40 PM, Thomas Petazzoni wrote:
> I just checked, and PAM is only needed when login-utils is enabled.
> 
> Also, 2.22 has apparently moved to a non-recursive usage of automake,
> which should make the build much, much faster.

Yes, we loose util-linux login for non-pam installations.
It's the upstream policy, everyone should have PAM.
On a personal level i don't care really, for the smallest login there's
busybox.
Yes, the build is faster too, however excluding components isn't good on
the non-recursive usage of automake, we'd just have to use good'ol rm in
a POST_BUILD hook unfortunately.
Regards.

[Buildroot] Proposed util-linux split

[Gustavo Zacarias]

On 02/26/2013 03:33 PM, Thomas Petazzoni wrote:

> I would suggest that we upgrade util-linux and add the necessary PAM
> dependency when needed. We already have package/linux-pam/, so it
> shouldn't be a big effort.
> 
> Of course, only the util-linux programs that actually need PAM support
> should make this library become a dependency of util-linux.

That wouldn't be hard, howerver we'll loose the busybox login
alternative for cases without PAM.

> Would it be possible instead that util-linux always uses the libuuid
> from the libuuid package?

Not without heavy patching which doesn't look so simple for 2.22+ that
did a big rewrite in the autotools fu.

> I must admit I don't really like having multiple packages for the same
> upstream tarball. Another option is to make something like:
> 
> config BR2_PACKAGE_UTIL_LINUX
> 	bool "util-linux"
> 	select BR2_PACKAGE_UTIL_LINUX_LIBUUID if !BR2_PACKAGE_UTIL_LINUX_AT_LEAST_SOMETHING
> 
> config BR2_PACKAGE_UTIL_LINUX_AT_LEAST_SOMETHING
> 	bool
> 
> config BR2_PACKAGE_UTIL_LINUX_LIBUUID
> 	bool "libuuid"
> 
> config BR2_PACKAGE_UTIL_LINUX_LIBBLKDID
> 	bool "libblkid"
> 	select BR2_PACKAGE_UTIL_LINUX_AT_LEAST_SOMETHING
> 
> config BR2_PACKAGE_UTIL_LINUX_LIBMOUNT
> 	bool "libmount"
> 	select BR2_PACKAGE_UTIL_LINUX_AT_LEAST_SOMETHING
> 
> config BR2_PACKAGE_UTIL_LINUX_SOMEPROGA
> 	bool "program a"
> 	select BR2_PACKAGE_UTIL_LINUX_AT_LEAST_SOMETHING
> 
> This way, by default util-linux only installs libuuid.
> 
> No?

That's what my previous util-linux patch did, however if we upgrade to
newer (2.22+) util-linux the old trick (make -C libxxx) doesn't work any
longer and needs patching anyway (it uses "automodules", hence no
regular Makefile for the subdirs, those are included from the top Makefile).
The patch isn't too straightforward since i remove the automodules
includes, hence not very nice for dynamic patching via sed.
The alternative would be to remove stuff we always build since
util-linux always installs a basic set of utilities that can't be
disabled (and remember, no install-libuuid or anything similar as
targets, they're all pushed into a big variable with no useful naming by
the automodules mechanism).
Regards.

[Buildroot] Proposed util-linux split

[Thomas Petazzoni]

Dear Gustavo Zacarias,

On Tue, 26 Feb 2013 15:41:15 -0300, Gustavo Zacarias wrote:

> > Of course, only the util-linux programs that actually need PAM support
> > should make this library become a dependency of util-linux.
> 
> That wouldn't be hard, howerver we'll loose the busybox login
> alternative for cases without PAM.

People wanting lightweight stuff will use Busybox, so I think it's
fine. And as said on IRC, if someone wants to use login utils from
util-linux without PAM, then that someone should go upstream and fix
this.

> > Would it be possible instead that util-linux always uses the libuuid
> > from the libuuid package?
> 
> Not without heavy patching which doesn't look so simple for 2.22+ that
> did a big rewrite in the autotools fu.

Ok, then I think I would really prefer to keep a single package.

> That's what my previous util-linux patch did, however if we upgrade to
> newer (2.22+) util-linux the old trick (make -C libxxx) doesn't work any
> longer and needs patching anyway (it uses "automodules", hence no
> regular Makefile for the subdirs, those are included from the top Makefile).
> The patch isn't too straightforward since i remove the automodules
> includes, hence not very nice for dynamic patching via sed.
> The alternative would be to remove stuff we always build since
> util-linux always installs a basic set of utilities that can't be
> disabled (and remember, no install-libuuid or anything similar as
> targets, they're all pushed into a big variable with no useful naming by
> the automodules mechanism).

After:

./configure --disable-libblkid --disable-libmount
--disable-deprecated-mount --disable-mount --disable-losetup
--disable-fsck --disable-partx --disable-uuidd --disable-mountpoint
--disable-fallocate --disable-unshare --disable-arch --disable-ddate
--disable-eject --disable-agetty --disable-cramfs --disable-switch_root
--disable-pivot_root --disable-kill --disable-last --disable-utmpdump
--disable-line --disable-mesg --disable-raw --disable-rename
--disable-reset --disable-vipw --disable-newgrp --disable-chfn-chsh
--disable-sulogin --disable-su --disable-schedutils --disable-wall

The build takes:

real    0m11.406s
user    0m35.654s
sys     0m4.528s

on my laptop. It certainly builds and installs a bunch more crap than
just libuuid, but it's not too bad.

Another solution is to apply the attached patch, and use the
--disable-allprogs option. Then, it builds and installs only libuuid.
It then takes:

real	0m3.209s
user	0m8.333s
sys	0m1.196s

Best regards,

Thomas
-- 
Thomas Petazzoni, Free Electrons
Kernel, drivers, real-time and embedded Linux
development, consulting, training and support.
http://free-electrons.com
-------------- next part --------------
A non-text attachment was scrubbed...
Name: util-linux-01-dont-build-anything.patch
Type: text/x-patch
Size: 3360 bytes
Desc: not available
URL: <http://lists.busybox.net/pipermail/buildroot/attachments/20130226/6cf0518d/attachment.bin>

[Buildroot] Proposed util-linux split

[Gustavo Zacarias]

On 02/26/2013 04:03 PM, Thomas Petazzoni wrote:

> Another solution is to apply the attached patch, and use the
> --disable-allprogs option. Then, it builds and installs only libuuid.
> It then takes:
> 
> real	0m3.209s
> user	0m8.333s
> sys	0m1.196s
> 
> Best regards,

I just kill the includes from Makefile.am, it could be a conditional
patch and likely to change less in future versions of util-linux.
Building all at once kills the libuuid without wchar posibility, how
useful that is is up in the air, i didn't try removing WCHAR depends
from apps caused by this and seeing which ones make it.
Regards.