[PATCH] util-linux: Remove static libraries from -dev packages

[PATCH] util-linux: Remove static libraries from -dev packages

[Phil Blundell]

Fixes the QA warnings:

WARNING: QA Issue: non -staticdev package contains static .a library: util-linux-libblkid-dev path '/work/mips32el-oe-linux/util-linux/2.21.2-r3micro3/packages-split/util-linux-libblkid-dev/lib/libblkid.a'
WARNING: QA Issue: non -staticdev package contains static .a library: util-linux-libuuid-dev path '/work/mips32el-oe-linux/util-linux/2.21.2-r3micro3/packages-split/util-linux-libuuid-dev/lib/libuuid.a'

Signed-off-by: Phil Blundell <pb@pbcl.net>
---
 meta/recipes-core/util-linux/util-linux.inc       |    4 ++--
 meta/recipes-core/util-linux/util-linux_2.21.2.bb |    2 +-
 2 files changed, 3 insertions(+), 3 deletions(-)

diff --git a/meta/recipes-core/util-linux/util-linux.inc b/meta/recipes-core/util-linux/util-linux.inc
index 2c5267f..b2a1552 100644
--- a/meta/recipes-core/util-linux/util-linux.inc
+++ b/meta/recipes-core/util-linux/util-linux.inc
@@ -57,9 +57,9 @@ FILES_util-linux-readprofile = "${base_sbindir}/readprofile.${BPN}"
 FILES_util-linux-uuidgen = "${bindir}/uuidgen"
 
 FILES_util-linux-libblkid = "${base_libdir}/libblkid.so.*"
-FILES_util-linux-libblkid-dev = "${base_libdir}/libblkid.so ${base_libdir}/libblkid.a ${base_libdir}/libblkid.la ${includedir}/blkid ${libdir}/pkgconfig/blkid.pc"
+FILES_util-linux-libblkid-dev = "${base_libdir}/libblkid.so ${base_libdir}/libblkid.la ${includedir}/blkid ${libdir}/pkgconfig/blkid.pc"
 FILES_util-linux-libuuid = "${base_libdir}/libuuid.so.*"
-FILES_util-linux-libuuid-dev = "${base_libdir}/libuuid.so ${base_libdir}/libuuid.a ${base_libdir}/libuuid.la ${includedir}/uuid ${libdir}/pkgconfig/uuid.pc"
+FILES_util-linux-libuuid-dev = "${base_libdir}/libuuid.so ${base_libdir}/libuuid.la ${includedir}/uuid ${libdir}/pkgconfig/uuid.pc"
 FILES_util-linux-lscpu = "${bindir}/lscpu"
 
 FILES_util-linux-fsck = "${base_sbindir}/fsck*"
diff --git a/meta/recipes-core/util-linux/util-linux_2.21.2.bb b/meta/recipes-core/util-linux/util-linux_2.21.2.bb
index e8f00fc..792730c 100644
--- a/meta/recipes-core/util-linux/util-linux_2.21.2.bb
+++ b/meta/recipes-core/util-linux/util-linux_2.21.2.bb
@@ -1,5 +1,5 @@
 MAJOR_VERSION = "2.21"
-PR = "r3"
+PR = "r4"
 require util-linux.inc
 
 # note that `lscpu' is under GPLv3+
-- 
1.7.10.4

Re: [PATCH] util-linux: Remove static libraries from -dev packages

[Colin Walters]

On Tue, 2012-09-25 at 13:54 +0100, Phil Blundell wrote:
> Fixes the QA warnings:
> 
> WARNING: QA Issue: non -staticdev package contains static .a library: util-linux-libblkid-dev path '/work/mips32el-oe-linux/util-linux/2.21.2-r3micro3/packages-split/util-linux-libblkid-dev/lib/libblkid.a'
> WARNING: QA Issue: non -staticdev package contains static .a library: util-linux-libuuid-dev path '/work/mips32el-oe-linux/util-linux/2.21.2-r3micro3/packages-split/util-linux-libuuid-dev/lib/libuuid.a'

> -FILES_util-linux-libblkid-dev = "${base_libdir}/libblkid.so ${base_libdir}/libblkid.a ${base_libdir}/libblkid.la ${includedir}/blkid ${libdir}/pkgconfig/blkid.pc"
> +FILES_util-linux-libblkid-dev = "${base_libdir}/libblkid.so ${base_libdir}/libblkid.la ${includedir}/blkid ${libdir}/pkgconfig/blkid.pc"

Wouldn't EXTRA_OECONF += " --disable-static" be better?

Re: [PATCH] util-linux: Remove static libraries from -dev packages

[Phil Blundell]

On Tue, 2012-09-25 at 17:59 -0400, Colin Walters wrote:
> On Tue, 2012-09-25 at 13:54 +0100, Phil Blundell wrote:
> > Fixes the QA warnings:
> > 
> > WARNING: QA Issue: non -staticdev package contains static .a library: util-linux-libblkid-dev path '/work/mips32el-oe-linux/util-linux/2.21.2-r3micro3/packages-split/util-linux-libblkid-dev/lib/libblkid.a'
> > WARNING: QA Issue: non -staticdev package contains static .a library: util-linux-libuuid-dev path '/work/mips32el-oe-linux/util-linux/2.21.2-r3micro3/packages-split/util-linux-libuuid-dev/lib/libuuid.a'
> 
> > -FILES_util-linux-libblkid-dev = "${base_libdir}/libblkid.so ${base_libdir}/libblkid.a ${base_libdir}/libblkid.la ${includedir}/blkid ${libdir}/pkgconfig/blkid.pc"
> > +FILES_util-linux-libblkid-dev = "${base_libdir}/libblkid.so ${base_libdir}/libblkid.la ${includedir}/blkid ${libdir}/pkgconfig/blkid.pc"
> 
> Wouldn't EXTRA_OECONF += " --disable-static" be better?

That'd be inconsistent with other packages, since we do generally build
and ship the static libraries.  Having a big switch to turn off static
libraries globally seems like a fine plan, but I can't see any obvious
reason why the util-linux ones are any more useless than the rest.

p.

Re: [PATCH] util-linux: Remove static libraries from -dev packages

[Colin Walters]

On Tue, 2012-09-25 at 23:00 +0100, Phil Blundell wrote:

> That'd be inconsistent with other packages, since we do generally build
> and ship the static libraries.  Having a big switch to turn off static
> libraries globally seems like a fine plan, but I can't see any obvious
> reason why the util-linux ones are any more useless than the rest.

Makes sense.  I wonder if there are actually any users of the static
libraries.

For what it's worth in gnome-ostree I do just globally pass
--disable-static by default.

Re: [PATCH] util-linux: Remove static libraries from -dev packages

[Richard Purdie]

On Tue, 2012-09-25 at 18:11 -0400, Colin Walters wrote:
> On Tue, 2012-09-25 at 23:00 +0100, Phil Blundell wrote:
> 
> > That'd be inconsistent with other packages, since we do generally build
> > and ship the static libraries.  Having a big switch to turn off static
> > libraries globally seems like a fine plan, but I can't see any obvious
> > reason why the util-linux ones are any more useless than the rest.
> 
> Makes sense.  I wonder if there are actually any users of the static
> libraries.
> 
> For what it's worth in gnome-ostree I do just globally pass
> --disable-static by default.

I tested this a while back to see what performance difference it made.
The answer was "nothing too significant", I don't have the exact timings
handy. I do remember having to exclude sqlite-native from the list since
pseudo static links against it.

Cheers,

Richard

Re: [PATCH] util-linux: Remove static libraries from -dev packages

[Phil Blundell]

On Wed, 2012-09-26 at 09:49 +0100, Richard Purdie wrote:
> On Tue, 2012-09-25 at 18:11 -0400, Colin Walters wrote:
> > On Tue, 2012-09-25 at 23:00 +0100, Phil Blundell wrote:
> > 
> > > That'd be inconsistent with other packages, since we do generally build
> > > and ship the static libraries.  Having a big switch to turn off static
> > > libraries globally seems like a fine plan, but I can't see any obvious
> > > reason why the util-linux ones are any more useless than the rest.
> > 
> > Makes sense.  I wonder if there are actually any users of the static
> > libraries.
> > 
> > For what it's worth in gnome-ostree I do just globally pass
> > --disable-static by default.
> 
> I tested this a while back to see what performance difference it made.
> The answer was "nothing too significant", I don't have the exact timings
> handy. I do remember having to exclude sqlite-native from the list since
> pseudo static links against it.

It's slightly surprising that it doesn't make that much of a difference,
given that building static libraries does essentially double the number
of compilations for library code.  Though, of course, glibc doesn't
support --disable-static nowadays and there might be a few other big
packages that have the same issue.

I guess that if you have enough cores, compilation count becomes
something of a non-issue since it's one of the few things that does
parallelize very well.  It might be interesting to repeat the
measurements of --disable-static on a machine with only a few CPUs and
see whether it makes more of a difference there.

Out of interest, why does pseudo static-link against sqlite anyway?

p.

Re: [PATCH] util-linux: Remove static libraries from -dev packages

[Richard Purdie]

On Wed, 2012-09-26 at 10:28 +0100, Phil Blundell wrote:
> On Wed, 2012-09-26 at 09:49 +0100, Richard Purdie wrote:
> > On Tue, 2012-09-25 at 18:11 -0400, Colin Walters wrote:
> > > On Tue, 2012-09-25 at 23:00 +0100, Phil Blundell wrote:
> > > 
> > > > That'd be inconsistent with other packages, since we do generally build
> > > > and ship the static libraries.  Having a big switch to turn off static
> > > > libraries globally seems like a fine plan, but I can't see any obvious
> > > > reason why the util-linux ones are any more useless than the rest.
> > > 
> > > Makes sense.  I wonder if there are actually any users of the static
> > > libraries.
> > > 
> > > For what it's worth in gnome-ostree I do just globally pass
> > > --disable-static by default.
> > 
> > I tested this a while back to see what performance difference it made.
> > The answer was "nothing too significant", I don't have the exact timings
> > handy. I do remember having to exclude sqlite-native from the list since
> > pseudo static links against it.
> 
> It's slightly surprising that it doesn't make that much of a difference,
> given that building static libraries does essentially double the number
> of compilations for library code.  Though, of course, glibc doesn't
> support --disable-static nowadays and there might be a few other big
> packages that have the same issue.
> 
> I guess that if you have enough cores, compilation count becomes
> something of a non-issue since it's one of the few things that does
> parallelize very well.  It might be interesting to repeat the
> measurements of --disable-static on a machine with only a few CPUs and
> see whether it makes more of a difference there.

Yes, admittedly I probably did test that in a highly parallel
environment and was focusing on the critical path timings as a result.

> Out of interest, why does pseudo static-link against sqlite anyway?

Given the crazy things that pseudo does and the way it works as an ld
preload that keeps itself preloaded, I'm guessing it stops itself having
to worry about finding its own libraries correctly. I just added a
package override for sqlite3-native which wasn't a big deal.

Cheers,

Richard

Re: [PATCH] util-linux: Remove static libraries from -dev packages

[Mark Hatle]

On 9/25/12 5:11 PM, Colin Walters wrote:
> On Tue, 2012-09-25 at 23:00 +0100, Phil Blundell wrote:
>
>> That'd be inconsistent with other packages, since we do generally build
>> and ship the static libraries.  Having a big switch to turn off static
>> libraries globally seems like a fine plan, but I can't see any obvious
>> reason why the util-linux ones are any more useless than the rest.
>
> Makes sense.  I wonder if there are actually any users of the static
> libraries.

Users are primarily small systems, where static linking may result in a slightly 
smaller system.  (When I say small systems, I'm talking about single application 
embedded systems..)

> For what it's worth in gnome-ostree I do just globally pass
> --disable-static by default.
>
>
>
> _______________________________________________
> Openembedded-core mailing list
> Openembedded-core@lists.openembedded.org
> http://lists.linuxtogo.org/cgi-bin/mailman/listinfo/openembedded-core
>

Re: [PATCH] util-linux: Remove static libraries from -dev packages

[Saul Wold]

On 09/25/2012 05:54 AM, Phil Blundell wrote:
> Fixes the QA warnings:
>
> WARNING: QA Issue: non -staticdev package contains static .a library: util-linux-libblkid-dev path '/work/mips32el-oe-linux/util-linux/2.21.2-r3micro3/packages-split/util-linux-libblkid-dev/lib/libblkid.a'
> WARNING: QA Issue: non -staticdev package contains static .a library: util-linux-libuuid-dev path '/work/mips32el-oe-linux/util-linux/2.21.2-r3micro3/packages-split/util-linux-libuuid-dev/lib/libuuid.a'
>
> Signed-off-by: Phil Blundell <pb@pbcl.net>
> ---
>   meta/recipes-core/util-linux/util-linux.inc       |    4 ++--
>   meta/recipes-core/util-linux/util-linux_2.21.2.bb |    2 +-
>   2 files changed, 3 insertions(+), 3 deletions(-)
>
> diff --git a/meta/recipes-core/util-linux/util-linux.inc b/meta/recipes-core/util-linux/util-linux.inc
> index 2c5267f..b2a1552 100644
> --- a/meta/recipes-core/util-linux/util-linux.inc
> +++ b/meta/recipes-core/util-linux/util-linux.inc
> @@ -57,9 +57,9 @@ FILES_util-linux-readprofile = "${base_sbindir}/readprofile.${BPN}"
>   FILES_util-linux-uuidgen = "${bindir}/uuidgen"
>
>   FILES_util-linux-libblkid = "${base_libdir}/libblkid.so.*"
> -FILES_util-linux-libblkid-dev = "${base_libdir}/libblkid.so ${base_libdir}/libblkid.a ${base_libdir}/libblkid.la ${includedir}/blkid ${libdir}/pkgconfig/blkid.pc"
> +FILES_util-linux-libblkid-dev = "${base_libdir}/libblkid.so ${base_libdir}/libblkid.la ${includedir}/blkid ${libdir}/pkgconfig/blkid.pc"
>   FILES_util-linux-libuuid = "${base_libdir}/libuuid.so.*"
> -FILES_util-linux-libuuid-dev = "${base_libdir}/libuuid.so ${base_libdir}/libuuid.a ${base_libdir}/libuuid.la ${includedir}/uuid ${libdir}/pkgconfig/uuid.pc"
> +FILES_util-linux-libuuid-dev = "${base_libdir}/libuuid.so ${base_libdir}/libuuid.la ${includedir}/uuid ${libdir}/pkgconfig/uuid.pc"
>   FILES_util-linux-lscpu = "${bindir}/lscpu"
>
>   FILES_util-linux-fsck = "${base_sbindir}/fsck*"
> diff --git a/meta/recipes-core/util-linux/util-linux_2.21.2.bb b/meta/recipes-core/util-linux/util-linux_2.21.2.bb
> index e8f00fc..792730c 100644
> --- a/meta/recipes-core/util-linux/util-linux_2.21.2.bb
> +++ b/meta/recipes-core/util-linux/util-linux_2.21.2.bb
> @@ -1,5 +1,5 @@
>   MAJOR_VERSION = "2.21"
> -PR = "r3"
> +PR = "r4"
>   require util-linux.inc
>
>   # note that `lscpu' is under GPLv3+
>

Merged into OE-Core

Thanks
	Sau!

Re: [PATCH] util-linux: Remove static libraries from -dev packages

[Phil Blundell]

On Wed, 2012-09-26 at 15:05 +0100, Richard Purdie wrote:
> Yes, admittedly I probably did test that in a highly parallel
> environment and was focusing on the critical path timings as a result.

Since one of the machines at the office had nothing better to do on a
Sunday I thought I'd rerun a test with and without static libs on a
smaller system than yours.  I used:

disable_static = "--disable-static"
disable_static_pn-sqlite3-native = ""
disable_static_pn-openssl = ""
disable_static_pn-openssl-native = ""
disable_static_pn-qemu-native = ""
EXTRA_OECONF_append = " ${disable_static}"
BB_NUMBER_THREADS = "4"
PARALLEL_MAKE = "-j4"

and I had to hack the openjade recipe to remove "-a" from a couple of
oe_libinstalls, but apart from that I used an unmodified tree with the
default local.conf from oe-init-build-env.

The timings for "bitbake core-minimal-native" with and without static
libraries were:

13988.74user 2043.25system 1:48:41elapsed 245%CPU (0avgtext+0avgdata
1789392maxresident)k
vs
12876.66user 1931.70system 1:42:43elapsed 240%CPU (0avgtext+0avgdata
1800288maxresident)k

which equates to about an 8% saving in user cycles and about 5.5% in
wall time.  So, not vast, but not totally insignificant either.  You
also get about a 5% saving in TMPDIR size (equivalent to ~ 850MB in this
test).

Obviously, there are some packages which don't use autotools (or
deliberately disregard --disable-static) and hence didn't get their
static libraries disabled by this simplistic configuration.
Specifically, I still have:

tmp-eglibc/deploy/ipk/i586/bison-staticdev_2.5.1-r0_i586.ipk
tmp-eglibc/deploy/ipk/i586/e2fsprogs-staticdev_1.42.1-r3_i586.ipk
tmp-eglibc/deploy/ipk/i586/eglibc-staticdev_2.16-r14+svnr20393_i586.ipk
tmp-eglibc/deploy/ipk/i586/elfutils-staticdev_0.148-r9_i586.ipk
tmp-eglibc/deploy/ipk/i586/flex-staticdev_2.5.35-r3_i586.ipk
tmp-eglibc/deploy/ipk/i586/libacl-staticdev_2.2.51-r3_i586.ipk
tmp-eglibc/deploy/ipk/i586/libattr-staticdev_2.4.46-r4_i586.ipk
tmp-eglibc/deploy/ipk/i586/libcap-staticdev_2.22-r5_i586.ipk
tmp-eglibc/deploy/ipk/i586/libstdc++-staticdev_4.7.2-r13_i586.ipk
tmp-eglibc/deploy/ipk/i586/libwrap-staticdev_7.6-r9_i586.ipk
tmp-eglibc/deploy/ipk/i586/libz-staticdev_1.2.7-r0_i586.ipk
tmp-eglibc/deploy/ipk/i586/lighttpd-module-staticfile_1.4.31-r0_i586.ipk
tmp-eglibc/deploy/ipk/i586/ncurses-staticdev_5.9-r10.1_i586.ipk
tmp-eglibc/deploy/ipk/i586/ocf-linux-staticdev_20100325-r3.0_i586.ipk
tmp-eglibc/deploy/ipk/i586/openssl-staticdev_1.0.0j-r15.3_i586.ipk
tmp-eglibc/deploy/ipk/i586/psmisc-staticdev_22.19-r0_i586.ipk
tmp-eglibc/deploy/ipk/i586/python-distutils-staticdev_2.7.3-r0.2_i586.ipk

p.