Gstreamer packaging

Gstreamer packaging

[Koen Kooi]

Saul said I broke gst-meta with my patch, so let's view the gstreamer situation:

given:

1) plugins move between -good, -bad and -ugly
2) libraries move between -good, -bad and -ugly
3) applications require specific plugins
4) upgrade paths must work

When combining 1 and 4 the current 'gst-plugin-{good,bad,ugly}-foo' naming does not work, since the same file will get provided by 2 packages after the move. Attentive readers will have guessed that the current gst-meta tries, and ultimately fails to hide 1) and 2).

So the new systems does the following:

* split out each plugin as gst-plugin-<foo>
* split out each lib as lib<foo>

So both plugins and libraries have a stable package name (barring plugin renames, e.g. flvdemux -> flv). Package feeds and upgrades finally work as expected

There are some downsides with this approach:

a) no way to know a priori where a plugin lives at a given time
b) by extension you build too much or too little.
c) scary multiple provider messages during parsing

OE .dev has a slightly different approach where you manually go through deploy and see what got generated by who and plug that into PROVIDES. I'm not a big fan of that, but it eliminates those scary messages.

I would very much like to have package feeds and upgrades to work, which currently is impossible for gstreamer packages.

regards,

Koen

Re: Gstreamer packaging

[Phil Blundell]

On Wed, 2011-06-29 at 11:33 +0200, Koen Kooi wrote:
> So the new systems does the following:
> 
> * split out each plugin as gst-plugin-<foo>
> * split out each lib as lib<foo>
> 
> So both plugins and libraries have a stable package name (barring plugin renames, e.g. flvdemux -> flv). Package feeds and upgrades finally work as expected

Agreed, I think this is about the only reasonable thing to do.  The way
that the gstreamer folks bundle up their plugins for distribution, and
particularly the semi-arbitrary split between -base, -good and -bad, is
not especially helpful for consumers of those packages.

In the past I have been strongly tempted to just stick all the plugins
(with the possible exception of -ugly, which might require a bit of
ENTERPRISE_DISTRO care) into a single recipe so that at least you always
know which recipe needs building to get a given plugin.  That would
obviously lead to more build time but I think it is probably a good
tradeoff in this situation.  In an ideal world it would be nice for all
the plugins to be packaged independently a la Xorg, but I have no idea
whether the gstreamer folks would be receptive to that idea.

> OE .dev has a slightly different approach where you manually go
> through deploy and see what got generated by who and plug that into
> PROVIDES. I'm not a big fan of that, but it eliminates those scary
> messages.

I guess that does also work, but I didn't like the patch when it first
went into .dev and I am still not very fond of it.

p.

Re: Gstreamer packaging

[Richard Purdie]

On Wed, 2011-06-29 at 10:55 +0100, Phil Blundell wrote:
> On Wed, 2011-06-29 at 11:33 +0200, Koen Kooi wrote:
> > So the new systems does the following:
> > 
> > * split out each plugin as gst-plugin-<foo>
> > * split out each lib as lib<foo>
> > 
> > So both plugins and libraries have a stable package name (barring
> plugin renames, e.g. flvdemux -> flv). Package feeds and upgrades
> finally work as expected
> 
> Agreed, I think this is about the only reasonable thing to do.  The way
> that the gstreamer folks bundle up their plugins for distribution, and
> particularly the semi-arbitrary split between -base, -good and -bad, is
> not especially helpful for consumers of those packages.
>
> In the past I have been strongly tempted to just stick all the plugins
> (with the possible exception of -ugly, which might require a bit of
> ENTERPRISE_DISTRO care) into a single recipe so that at least you always
> know which recipe needs building to get a given plugin.  That would
> obviously lead to more build time but I think it is probably a good
> tradeoff in this situation.  In an ideal world it would be nice for all
> the plugins to be packaged independently a la Xorg, but I have no idea
> whether the gstreamer folks would be receptive to that idea.

Let me quickly recap the problem. Its perfectly reasonable for a recipe
to want to depend on "gst-plugin-<foo>".

The trouble is that bitbake is left pretty much totally clueless when
something says it would like to have "gst-plugin-<foo>" and multiple
things provide it.

Obviously you can make the recipe depend on good+bad+ugly but its less
than ideal for build time reasons (esp. when considering dependencies)
but also the reason that good/bad/ugly exist in the first place which is
licensing. If the recipe always has to depend on good+bad+ugly, it
becomes rather tricky to disable ugly and work out whether the resulting
configuration can build. Companies interested in license compliance do
have a strong need to be able to do this.

Its for the latter reason that OE-Core has kept ${PN} in the plugin
names at present since deterministic builds are kind of nice.

> > OE .dev has a slightly different approach where you manually go
> > through deploy and see what got generated by who and plug that into
> > PROVIDES. I'm not a big fan of that, but it eliminates those scary
> > messages.
> 
> I guess that does also work, but I didn't like the patch when it first
> went into .dev and I am still not very fond of it.

It would at least ensure deterministic builds but I share your lack of
fondness.

A recipe per plugin would certainly start to look attractive and it
might be worth talking to the gstreamer people...

Cheers,

Richard

Re: Gstreamer packaging

[Phil Blundell]

On Wed, 2011-06-29 at 11:53 +0100, Richard Purdie wrote:
> Obviously you can make the recipe depend on good+bad+ugly but its less
> than ideal for build time reasons (esp. when considering dependencies)
> but also the reason that good/bad/ugly exist in the first place which is
> licensing. 

It's only -ugly which has licensing issues, and those same licensing
issues also mean that plugins don't tend to migrate to or from -ugly in
the same way as they do between the other packages.  So, if -good, -bad
and -base were combined in a single recipe, leaving -ugly separate, that
would solve about 90% of the current nuisance.

Dependency-wise, gst-plugins-good is the only one with a large stack of
DEPENDS.  If you assume that most people are probably going to need at
least one plugin from that package anyway, which I think is probably the
case in reality, then the dependency impact of combining -good, -bad and
-base in a single recipe would be fairly negligible.  (The only extra
dependencies you'd get from -bad would be libmusicbrainz, tremor and
librsvg.)

Of course, another way to attack the dependency stack problem would be
to combine the recipes and then be more aggressive about allowing
DISTROs to select only the plugins (and hence the dependencies) that
they want.  For example, I have long found it slightly annoying that
gst-plugins-good requires gtk+ and x11 at build time since I don't
actually use either of those libraries in my deployed system.

p.

Re: Gstreamer packaging

[Phil Blundell]

On Wed, 2011-06-29 at 11:53 +0100, Richard Purdie wrote:
> Obviously you can make the recipe depend on good+bad+ugly but its less
> than ideal for build time reasons (esp. when considering dependencies)
> but also the reason that good/bad/ugly exist in the first place which is
> licensing. If the recipe always has to depend on good+bad+ugly, it
> becomes rather tricky to disable ugly and work out whether the resulting
> configuration can build. Companies interested in license compliance do
> have a strong need to be able to do this.

Incidentally, there isn't actually (as far as I can tell) anything in
the current -ugly recipe which would indicate to an ENTERPRISE_DISTRO
that this recipe is potentially problematic.  Its LICENSE field just
reads "GPLv2+ & LGPLv2.1+ & LGPLv2+", which might or might not be
accurate, and it doesn't appear to have the self-skipping behaviour
which the corresponding recipe in oe.dev does.

p.

Re: Gstreamer packaging

[Richard Purdie]

On Wed, 2011-06-29 at 12:08 +0100, Phil Blundell wrote:
> On Wed, 2011-06-29 at 11:53 +0100, Richard Purdie wrote:
> > Obviously you can make the recipe depend on good+bad+ugly but its less
> > than ideal for build time reasons (esp. when considering dependencies)
> > but also the reason that good/bad/ugly exist in the first place which is
> > licensing. If the recipe always has to depend on good+bad+ugly, it
> > becomes rather tricky to disable ugly and work out whether the resulting
> > configuration can build. Companies interested in license compliance do
> > have a strong need to be able to do this.
> 
> Incidentally, there isn't actually (as far as I can tell) anything in
> the current -ugly recipe which would indicate to an ENTERPRISE_DISTRO
> that this recipe is potentially problematic.  Its LICENSE field just
> reads "GPLv2+ & LGPLv2.1+ & LGPLv2+", which might or might not be
> accurate, and it doesn't appear to have the self-skipping behaviour
> which the corresponding recipe in oe.dev does.

See conf/distro/include/default-distrovars.inc:

# This is a list of packages that require a commercial license to ship
# product. If shipped as part of an image these packages may have 
# implications so they are disabled by default
COMMERCIAL_LICENSE ?= "lame gst-fluendo-mp3 libmad mpeg2dec ffmpeg qmmp"
COMMERCIAL_AUDIO_PLUGINS ?= ""
# COMMERCIAL_AUDIO_PLUGINS ?= "gst-plugins-ugly-mad gst-plugins-ugly-mpegaudioparse"
COMMERCIAL_VIDEO_PLUGINS ?= ""
# COMMERCIAL_VIDEO_PLUGINS ?= "gst-plugins-ugly-mpeg2dec gst-plugins-ugly-mpegstream gst-plugins-bad-mpegvideoparse"
COMMERCIAL_QT ?= ""
# COMMERCIAL_QT ?= "qmmp"

I agree we should like have this kind of information indicated at the
recipe level though.

Cheers,

Richard