[RFC] granular library packaging

[RFC] granular library packaging

[Koen Kooi]

Hi,

I tried to come up with a way to make splitting up library packages a 
bit easier an used this in the opencv recipe:

python populate_packages_prepend () {
	cv_libdir = bb.data.expand('${libdir}', d)
	cv_libdir_dbg = bb.data.expand('${libdir}/.debug', d)
	do_split_packages(d, cv_libdir, '^lib(.*)\.so$', 'lib%s-dev', 'OpenCV 
%s development package', extra_depends='${PN}-dev', allow_links=True)
	do_split_packages(d, cv_libdir, '^lib(.*)\.la$', 'lib%s-dev', 'OpenCV 
%s development package', extra_depends='${PN}-dev')
	do_split_packages(d, cv_libdir, '^lib(.*)\.a$', 'lib%s-dev', 'OpenCV %s 
development package', extra_depends='${PN}-dev')
	do_split_packages(d, cv_libdir, '^lib(.*)\.so\.*', 'lib%s', 'OpenCV %s 
library', extra_depends='', allow_links=True)
}

FILES_${PN} = "${bindir}"
FILES_${PN}-dbg += "${libdir}/.debug"
FILES_${PN}-dev = "${includedir} ${libdir}/pkgconfig"

The advantages of this:

* all libs get their own package
* .so, .a and .la files end up in the right package
* lib<foo>-dev depends on the headers in ${PN}-dev keeping the depchains 
intact

The downsides:

* populate_packages_prepend runs *before* do_strip, so there's on big 
-dbg package to rule them all
* doesn't handle granular pkgconfig files.

I had a chat with Richard about it and we agreed that this needs some 
more thought, especially wrt depencies and granular packaging. My 
motivation for splitting things is to make using gdb less space-heavy 
due to pulling in a huge -dbg package that depends on extra, unneeded 
libs. As I pointed out above, that is sadly the case here since do_strip 
runs too late.

So, what are your opinions on splitting -dev and -dbg packages and would 
something like this be acceptable to put in lib_package.bbclass?

Note that a regexp based FILES is a related, but different discussion.


regards,

Koen

Re: [RFC] granular library packaging

[Richard Purdie]

On Mon, 2008-10-06 at 16:00 +0200, Koen Kooi wrote:
> I had a chat with Richard about it and we agreed that this needs some 
> more thought, especially wrt depencies and granular packaging. My 
> motivation for splitting things is to make using gdb less space-heavy 
> due to pulling in a huge -dbg package that depends on extra, unneeded 
> libs. As I pointed out above, that is sadly the case here since do_strip 
> runs too late.

There are various subtle problems with having multiple -dev and -dbg
packages. There are assumptions made in parts of the code that of foo
has bar as a build time dependency, foo-dev should RDEPEND on bar-dev.
If we split into multiple -dev packages, foo-dev needs to RDEPEND on all
the *-dev packages provided by bar-dev which gets a little more
complicated (think about trying to implement that when the list of -dev
packages is dynamic ;-).

For debugging or profiling, usually when you install foo-dbg you want
symbols for every library that package might use and the dependencies of
those libraries to ensure you get a meaningful stacktrace.

From what Koen's described he wants individual libs so he only has to
install one lib and not the dependencies. In that case I'd suggesting
telling the package manager to ignore the dependencies instead of trying
to neuter the package dependencies which are actually useful. If we go
with multiple libs, the depchain -dev and -dbg code will then have to
catch up with the change somehow and we'll be back to square one.

In machines where you're going to extensively use -dev and -dbg packages
you usually have enough space to install dependencies so my reasoning
has always been this isn't as much of a problem as the general grangular
packaging. Having tried to implement multiple -dev and -dbg in depchains
I'm very nervous of that idea (I think the code may now partially handle
it but there were problems, I don't recall exactly what offhand).

Cheers,

Richard