proposal to move cogl, clutter and related recipes from oe-core to dedicated meta-clutter layer

proposal to move cogl, clutter and related recipes from oe-core to dedicated meta-clutter layer

[Tomas Frydrych]

I think it would makes sense to move clutter related packages from
oe-core into a dedicated layer:

* AFAIK nothing in oe-core requires cogl/clutter/mx,

* The packages in oe-core are effectively unmaintained, several upstream
releases behind, and pretty much unusable,

* The somewhat random nature of clutter and cogl releases makes it hard
to sensibly manage these packages within the oe-core release cycle, but
a dedicated layer could follow the upstream developments.


I have started work on new clutter and related packages for use by
meta-guacamayo at https://github.com/Guacamayo/meta-clutter, but I'd be
more than happy for the layer to live somewhere else and become the
canonical location for clutter-related bits and pieces.

Tomas

-- 
http://sleepfive.com

Re: proposal to move cogl, clutter and related recipes from oe-core to dedicated meta-clutter layer

[Phil Blundell]

On Wed, 2013-05-08 at 16:11 +0100, Tomas Frydrych wrote:
> I think it would makes sense to move clutter related packages from
> oe-core into a dedicated layer:
> 
> * AFAIK nothing in oe-core requires cogl/clutter/mx,
> 
> * The packages in oe-core are effectively unmaintained, several upstream
> releases behind, and pretty much unusable,

I think this would be a fine idea.  As you observe, the recipes in
oe-core today are old, slightly hokey, and not a whole lot of use for
much.  We are currently maintaining our own clutter recipes in a local
layer but it would be nice to have something else upstream to lean on.

One particular issue for us is that the clutter recipes as they stand in
oe-core are very X11-centric.  We don't have X and, instead, use the
eglnative backend.  It'd be handy for us if some putative refactoring of
the recipes made them more friendly to non-X11 deployments.

p.

Re: proposal to move cogl, clutter and related recipes from oe-core to dedicated meta-clutter layer

[Tomas Frydrych]

On 08/05/13 16:23, Phil Blundell wrote:
> One particular issue for us is that the clutter recipes as they stand in
> oe-core are very X11-centric.  We don't have X and, instead, use the
> eglnative backend.  It'd be handy for us if some putative refactoring of
> the recipes made them more friendly to non-X11 deployments.

I have a similar problem in Guacamayo, where I have more than one target
machine, and generally prefer to use the elg-native backend unless it
cannot be avoided because the elg driver is too broken. So the recipes
in Guacamayo were set up so as to make it easy to reconfigure the
packages based on machine and distro features using a mininal bbappend,
e.g.,
https://github.com/Guacamayo/meta-guacamayo/blob/master/meta-guacamayo/recipes-graphics/clutter/clutter-1.12_git.bbappend.
The original recipes are too much marked by the specifics of the
Guacamayo set up and their gradual evolution, but I am in the process of
cleaning this up, so it can be used by others.

Tomas


-- 
http://sleepfive.com

Re: proposal to move cogl, clutter and related recipes from oe-core to dedicated meta-clutter layer

[Richard Purdie]

On Wed, 2013-05-08 at 16:11 +0100, Tomas Frydrych wrote:
> I think it would makes sense to move clutter related packages from
> oe-core into a dedicated layer:
> 
> * AFAIK nothing in oe-core requires cogl/clutter/mx,
> 
> * The packages in oe-core are effectively unmaintained, several upstream
> releases behind, and pretty much unusable,
> 
> * The somewhat random nature of clutter and cogl releases makes it hard
> to sensibly manage these packages within the oe-core release cycle, but
> a dedicated layer could follow the upstream developments.
> 
> 
> I have started work on new clutter and related packages for use by
> meta-guacamayo at https://github.com/Guacamayo/meta-clutter, but I'd be
> more than happy for the layer to live somewhere else and become the
> canonical location for clutter-related bits and pieces.

I have no idea why you've always felt the need to maintain the clutter
pieces in your own layer rather than interacting with the ones in
OE-Core instead which I'd love to see better maintained. I'm not aware
of any barrier that has prevented that.

I can't help feeling this is an artificially created problem :/.

Cheers,

Richard

Re: proposal to move cogl, clutter and related recipes from oe-core to dedicated meta-clutter layer

[Tomas Frydrych]

On 08/05/13 16:23, Richard Purdie wrote:
> On Wed, 2013-05-08 at 16:11 +0100, Tomas Frydrych wrote:
>> I think it would makes sense to move clutter related packages from
>> oe-core into a dedicated layer:
>>
>> * AFAIK nothing in oe-core requires cogl/clutter/mx,
>>
>> * The packages in oe-core are effectively unmaintained, several upstream
>> releases behind, and pretty much unusable,
>>
>> * The somewhat random nature of clutter and cogl releases makes it hard
>> to sensibly manage these packages within the oe-core release cycle, but
>> a dedicated layer could follow the upstream developments.
>>
>>
>> I have started work on new clutter and related packages for use by
>> meta-guacamayo at https://github.com/Guacamayo/meta-clutter, but I'd be
>> more than happy for the layer to live somewhere else and become the
>> canonical location for clutter-related bits and pieces.
> 
> I have no idea why you've always felt the need to maintain the clutter
> pieces in your own layer rather than interacting with the ones in
> OE-Core instead which I'd love to see better maintained. I'm not aware
> of any barrier that has prevented that.

It's mostly a matter of timing. Clutter does not provide LTS releases,
it pretty much deprecates the previous stable branch as soon as new
stable branch is started, so tracking the upstream reasonably quickly
matters. The timing for the danny oe-core release and the arrival of
clutter 1.12 was such that it simply could not have made it into
oe-core. Needing 1.12 I had no option than to package it elsewhere.

Yes, I could have submitted clutter 1.12 recipes to oe-core in some form
and shape in the last 6 months, and we would have had a less outdated
package in oe-core; but nevertheless outdated, since again the clutter
1.14 release came too late to make it into dylan. I can see this
happening again and again.

If there is a good reason to maintain clutter, cogl and mx in oe-core,
then I'll make patches for 1.14, but I am not convinced there is a good
reason, and that everyone would be better served by a dedicated layer.

Tomas

Re: proposal to move cogl, clutter and related recipes from oe-core to dedicated meta-clutter layer

[Richard Purdie]

On Wed, 2013-05-08 at 17:20 +0100, Tomas Frydrych wrote:
> On 08/05/13 16:23, Richard Purdie wrote:
> > On Wed, 2013-05-08 at 16:11 +0100, Tomas Frydrych wrote:
> >> I think it would makes sense to move clutter related packages from
> >> oe-core into a dedicated layer:
> >>
> >> * AFAIK nothing in oe-core requires cogl/clutter/mx,
> >>
> >> * The packages in oe-core are effectively unmaintained, several upstream
> >> releases behind, and pretty much unusable,
> >>
> >> * The somewhat random nature of clutter and cogl releases makes it hard
> >> to sensibly manage these packages within the oe-core release cycle, but
> >> a dedicated layer could follow the upstream developments.
> >>
> >>
> >> I have started work on new clutter and related packages for use by
> >> meta-guacamayo at https://github.com/Guacamayo/meta-clutter, but I'd be
> >> more than happy for the layer to live somewhere else and become the
> >> canonical location for clutter-related bits and pieces.
> > 
> > I have no idea why you've always felt the need to maintain the clutter
> > pieces in your own layer rather than interacting with the ones in
> > OE-Core instead which I'd love to see better maintained. I'm not aware
> > of any barrier that has prevented that.
> 
> It's mostly a matter of timing. Clutter does not provide LTS releases,
> it pretty much deprecates the previous stable branch as soon as new
> stable branch is started, so tracking the upstream reasonably quickly
> matters. The timing for the danny oe-core release and the arrival of
> clutter 1.12 was such that it simply could not have made it into
> oe-core. Needing 1.12 I had no option than to package it elsewhere.
>
> Yes, I could have submitted clutter 1.12 recipes to oe-core in some form
> and shape in the last 6 months, and we would have had a less outdated
> package in oe-core; but nevertheless outdated, since again the clutter
> 1.14 release came too late to make it into dylan. I can see this
> happening again and again.

The trouble is you can make this argument for every single piece of
software in OE-Core. There was nothing stopping you pushing the 1.12
work back into what became dylan as soon as it opened up for changes.
There was also nothing you maintaining an a branch of danny with the
1.12 updates backported into it.

> If there is a good reason to maintain clutter, cogl and mx in oe-core,
> then I'll make patches for 1.14, but I am not convinced there is a good
> reason, and that everyone would be better served by a dedicated layer.

A dedicated layer will still have timing issues, it will just move onto
your personal schedule rather than the OE-Core one and whilst this will
obviously suit you, it likely won't suit all other users.

I suspect the bigger problem here is that clutter is hard to write
recipes for since it needs to suit a number of different targets and
configurations. Going to the effort of doing a generic implementation in
OE-Core is hard, whereas creating your own layer means you can customise
to your usecase and not worry about the other cases. I suspect your
reply to this will be that anyone wanting to add other cases can send
you patches. The implication is that the layer will become much more
specialised/focused than the core recipes currently are.

My preference would still be to fix up the recipes in the core, than
have some specific branches for danny/dylan with the 1.12/1.14
components in if/as needed. We can create the core recipes so they're
properly configurable to the different usecases.

From what I gather you're going ahead with meta-clutter anyway
though :/.

Cheers,

Richard

Re: proposal to move cogl, clutter and related recipes from oe-core to dedicated meta-clutter layer

[Tomas Frydrych]

Hi Richard,

On 10/05/13 10:05, Richard Purdie wrote:
>> Yes, I could have submitted clutter 1.12 recipes to oe-core in some form
>> and shape in the last 6 months, and we would have had a less outdated
>> package in oe-core; but nevertheless outdated, since again the clutter
>> 1.14 release came too late to make it into dylan. I can see this
>> happening again and again.
> 
> The trouble is you can make this argument for every single piece of
> software in OE-Core.

The clutter related packages are the only ones where I have run into
this problem. In the last 12 months clutter went through 7 stable
releases or so, of these there were 3 minor version changes, which
signal API changes, another minor version change, as well as a major
version change to 2.0, is on the horizon. If you are developing an
application using clutter; you need to keep up. If you are using
Yocto/OE to develop an application using clutter (and people do), then
everyone is left to maintain their own rolling recipes.


> A dedicated layer will still have timing issues, it will just move onto
> your personal schedule rather than the OE-Core one and whilst this will
> obviously suit you, it likely won't suit all other users.

For a small dedicated layer the schedule can closely track the upstream.
It might not suit all clutter users, but I think it could be made to
work for most of them; the current situation suits no one at all.


> I suspect the bigger problem here is that clutter is hard to write
> recipes for since it needs to suit a number of different targets and
> configurations. Going to the effort of doing a generic implementation in
> OE-Core is hard, whereas creating your own layer means you can customise
> to your usecase and not worry about the other cases. I suspect your
> reply to this will be that anyone wanting to add other cases can send
> you patches. The implication is that the layer will become much more
> specialised/focused than the core recipes currently are.

My reply is that it clutter is not that complex, there are only a finite
number of possible configurations that make sense and it should be
entirely possible to write a good base recipe that can be easily tweaked
using a bbappend based on machine and distro needs. But that's not the
real issue.


> My preference would still be to fix up the recipes in the core, than
> have some specific branches for danny/dylan with the 1.12/1.14
> components in if/as needed. We can create the core recipes so they're
> properly configurable to the different usecases.

Fixing up the recipes in oe-core only addresses one aspect of the
problem. The fast turnover of the clutter packages will remain, as will
the fact that nothing in oe-core uses clutter, so the oe-core packages
are untested. Then there is the fact that oe-core does not have any
machines that clutter could be really used with. Then there are also
other projects that are closely tied to clutter version, such as (the
recently removed) mutter, and, dare I say, GnomeShell, which should be
maintained together.

I am still to hear any reason why clutter should be in oe-core ... the
same logic that said mutter should be removed from oe-core applies to
clutter, I think.

Tomas

Re: proposal to move cogl, clutter and related recipes from oe-core to dedicated meta-clutter layer

[Richard Purdie]

On Fri, 2013-05-10 at 11:56 +0100, Tomas Frydrych wrote:
> Hi Richard,
> 
> On 10/05/13 10:05, Richard Purdie wrote:
> >> Yes, I could have submitted clutter 1.12 recipes to oe-core in some form
> >> and shape in the last 6 months, and we would have had a less outdated
> >> package in oe-core; but nevertheless outdated, since again the clutter
> >> 1.14 release came too late to make it into dylan. I can see this
> >> happening again and again.
> > 
> > The trouble is you can make this argument for every single piece of
> > software in OE-Core.
> 
> The clutter related packages are the only ones where I have run into
> this problem.

That is down to the particular development focus you have right now
though. To pick another example, if you're doing arm64 work, the
toolchain is horribly dated in the core. I dare say you could have this
issue for any of the software components (glib, gtk+) and so on.

>  In the last 12 months clutter went through 7 stable
> releases or so, of these there were 3 minor version changes, which
> signal API changes, another minor version change, as well as a major
> version change to 2.0, is on the horizon. If you are developing an
> application using clutter; you need to keep up. If you are using
> Yocto/OE to develop an application using clutter (and people do), then
> everyone is left to maintain their own rolling recipes.

I'd hope that with a decent set of core .inc files in OE-Core, this
wouldn't be a big issue as you could add those newer versions trivially,
then drop them as they make it into the core.

> > A dedicated layer will still have timing issues, it will just move onto
> > your personal schedule rather than the OE-Core one and whilst this will
> > obviously suit you, it likely won't suit all other users.
> 
> For a small dedicated layer the schedule can closely track the upstream.
> It might not suit all clutter users, but I think it could be made to
> work for most of them; the current situation suits no one at all.

The closely track upstream is the key part and I think the core can do
this, apart from the ~six week stabilisation window.


> > I suspect the bigger problem here is that clutter is hard to write
> > recipes for since it needs to suit a number of different targets and
> > configurations. Going to the effort of doing a generic implementation in
> > OE-Core is hard, whereas creating your own layer means you can customise
> > to your usecase and not worry about the other cases. I suspect your
> > reply to this will be that anyone wanting to add other cases can send
> > you patches. The implication is that the layer will become much more
> > specialised/focused than the core recipes currently are.
> 
> My reply is that it clutter is not that complex, there are only a finite
> number of possible configurations that make sense and it should be
> entirely possible to write a good base recipe that can be easily tweaked
> using a bbappend based on machine and distro needs. But that's not the
> real issue.

I agree it should be entirely possible.

> > My preference would still be to fix up the recipes in the core, than
> > have some specific branches for danny/dylan with the 1.12/1.14
> > components in if/as needed. We can create the core recipes so they're
> > properly configurable to the different usecases.
> 
> Fixing up the recipes in oe-core only addresses one aspect of the
> problem. The fast turnover of the clutter packages will remain, as will
> the fact that nothing in oe-core uses clutter, so the oe-core packages
> are untested. Then there is the fact that oe-core does not have any
> machines that clutter could be really used with. Then there are also
> other projects that are closely tied to clutter version, such as (the
> recently removed) mutter, and, dare I say, GnomeShell, which should be
> maintained together.

Well, I originally tried to insist we had some mechanism for testing
clutter in OE-Core however the pieces have either become obsoleted or
removed for other reasons. I do really want to see ptest packages for
clutter which the autobuilder would call into and run. I appreciate GL
support under qemu through software rendering is currently problematic
but I want to see that fixed in the 1.5 cycle.

> I am still to hear any reason why clutter should be in oe-core ... the
> same logic that said mutter should be removed from oe-core applies to
> clutter, I think.

My argument for this is that I really do want to stress out the graphics
stack we have, clutter provides a good way to test some of those
components, particularly some of the more unusual parts. Currently its
mostly build test however we do have plans to make that runtime too. I
do think that clutters unusual usage of the stack makes it particular
useful in this role. I'd appreciate help from anyone who can help make
this all work.

I would therefore like to see a decent set of clutter recipes in the
core.

I keep hearing people arguing for removing many different pieces of the
core. I continue to believe we do need a small set of diverse
technologies in the core, complete with tests (which I want to extend)
to ensure that the core does do something useful.

Cheers,

Richard

Re: proposal to move cogl, clutter and related recipes from oe-core to dedicated meta-clutter layer

[Tomas Frydrych]

On 10/05/13 12:32, Richard Purdie wrote:
> On Fri, 2013-05-10 at 11:56 +0100, Tomas Frydrych wrote:
>> On 10/05/13 10:05, Richard Purdie wrote:
> The closely track upstream is the key part and I think the core can do
> this, apart from the ~six week stabilisation window.

If you mean you are prepared to do frequent point releases to keep up
with clutter, that could work. But if you mean that those interested can
closely track oe-core master, then that is not that useful, there are
too many other changes happening at the same time. A small single
purpose layer means updates (and breakages) can be very contained.


> My argument for this is that I really do want to stress out the graphics
> stack we have, clutter provides a good way to test some of those
> components, particularly some of the more unusual parts. Currently its
> mostly build test however we do have plans to make that runtime too. I
> do think that clutters unusual usage of the stack makes it particular
> useful in this role. I'd appreciate help from anyone who can help make
> this all work.

I am all for this, but does using clutter for automated tests require
for the clutter packages to be in oe-core? The only thing you can stress
test in oe-core using clutter is mesa, which is only applicable to the
i915/i965 chip sets. I think it would be useful if any such tests could
be applied to other graphics stacks provided by BSPs; I think all of the
BSPs would benefit from this.

I'd really like for people to be able to just pick up uptodate and
working clutter packages for the major platforms the actively maintained
BSPs support. Every so often someone asks about clutter for XYZ (usually
the Beagleboard or RPi) on the clutter list: this should be readily
available somewhere.

have a good weekend everyone,

Tomas

-- 
http://sleepfive.com

Re: proposal to move cogl, clutter and related recipes from oe-core to dedicated meta-clutter layer

[Richard Purdie]

On Fri, 2013-05-10 at 17:39 +0100, Tomas Frydrych wrote:
> On 10/05/13 12:32, Richard Purdie wrote:
> > On Fri, 2013-05-10 at 11:56 +0100, Tomas Frydrych wrote:
> >> On 10/05/13 10:05, Richard Purdie wrote:
> > The closely track upstream is the key part and I think the core can do
> > this, apart from the ~six week stabilisation window.
> 
> If you mean you are prepared to do frequent point releases to keep up
> with clutter, that could work. But if you mean that those interested can
> closely track oe-core master, then that is not that useful, there are
> too many other changes happening at the same time. A small single
> purpose layer means updates (and breakages) can be very contained.

Point releases of what? I don't mind clutter in master changing quite a
lot up to our stabilisation point. Once we've released, I don't mind
someone else picking up a danny-clutter branch or something like that
which is backporting specific changes.

> > My argument for this is that I really do want to stress out the graphics
> > stack we have, clutter provides a good way to test some of those
> > components, particularly some of the more unusual parts. Currently its
> > mostly build test however we do have plans to make that runtime too. I
> > do think that clutters unusual usage of the stack makes it particular
> > useful in this role. I'd appreciate help from anyone who can help make
> > this all work.
> 
> I am all for this, but does using clutter for automated tests require
> for the clutter packages to be in oe-core? The only thing you can stress
> test in oe-core using clutter is mesa, which is only applicable to the
> i915/i965 chip sets. I think it would be useful if any such tests could
> be applied to other graphics stacks provided by BSPs; I think all of the
> BSPs would benefit from this.

We'd be able to test mesa and the software fallbacks under qemu which
would be a start. If we can get those working with a decent framework
for the tests, I'm hoping wider BSP testing would follow. I don't have
unlimited resources available but I can try and get the software
infrastructure to the point where doing the testing could be picked up
by someone else relatively easily.

It is a big help in trying to achieve this if we don't have many
different layers involved as that would complicate the problem, even
coordinating layer releases between different maintainers is tricky
right now and trying to develop something like this over layer
boundaries sounds like adding to the complexity to the point I'd rather
just scrap the idea :(.

> I'd really like for people to be able to just pick up uptodate and
> working clutter packages for the major platforms the actively maintained
> BSPs support.

Agreed.

>  Every so often someone asks about clutter for XYZ (usually
> the Beagleboard or RPi) on the clutter list: this should be readily
> available somewhere.

I'd hope the recipes in the core would be in a good state in this
regard.

Have a good weekend.

Cheers,

Richard

Re: proposal to move cogl, clutter and related recipes from oe-core to dedicated meta-clutter layer

[Otavio Salvador]

On Fri, May 10, 2013 at 2:19 PM, Richard Purdie
<richard.purdie@linuxfoundation.org> wrote:
> On Fri, 2013-05-10 at 17:39 +0100, Tomas Frydrych wrote:
>> On 10/05/13 12:32, Richard Purdie wrote:
>> > On Fri, 2013-05-10 at 11:56 +0100, Tomas Frydrych wrote:
>> >> On 10/05/13 10:05, Richard Purdie wrote:
>> > The closely track upstream is the key part and I think the core can do
>> > this, apart from the ~six week stabilisation window.
>>
>> If you mean you are prepared to do frequent point releases to keep up
>> with clutter, that could work. But if you mean that those interested can
>> closely track oe-core master, then that is not that useful, there are
>> too many other changes happening at the same time. A small single
>> purpose layer means updates (and breakages) can be very contained.
>
> Point releases of what? I don't mind clutter in master changing quite a
> lot up to our stabilisation point. Once we've released, I don't mind
> someone else picking up a danny-clutter branch or something like that
> which is backporting specific changes.

So we're going to have <branch>-<pet package> branches? I think this
does not scale and I am at Tomas side. The layer seems the right thing
 to do so it does not need to be done by branch and avoid confusing
users.

>> > My argument for this is that I really do want to stress out the graphics
>> > stack we have, clutter provides a good way to test some of those
>> > components, particularly some of the more unusual parts. Currently its
>> > mostly build test however we do have plans to make that runtime too. I
>> > do think that clutters unusual usage of the stack makes it particular
>> > useful in this role. I'd appreciate help from anyone who can help make
>> > this all work.
>>
>> I am all for this, but does using clutter for automated tests require
>> for the clutter packages to be in oe-core? The only thing you can stress
>> test in oe-core using clutter is mesa, which is only applicable to the
>> i915/i965 chip sets. I think it would be useful if any such tests could
>> be applied to other graphics stacks provided by BSPs; I think all of the
>> BSPs would benefit from this.
>
> We'd be able to test mesa and the software fallbacks under qemu which
> would be a start. If we can get those working with a decent framework
> for the tests, I'm hoping wider BSP testing would follow. I don't have
> unlimited resources available but I can try and get the software
> infrastructure to the point where doing the testing could be picked up
> by someone else relatively easily.

You could do just the same but adding meta-clutter to the AB setup for
testing. One thing does not conflict with the another.

> It is a big help in trying to achieve this if we don't have many
> different layers involved as that would complicate the problem, even
> coordinating layer releases between different maintainers is tricky
> right now and trying to develop something like this over layer
> boundaries sounds like adding to the complexity to the point I'd rather
> just scrap the idea :(.

Well if layers make life harder it invalidates one of points of Yocto
which I always liked and depends on heavily - modular design.

I am sure we all want a solid release so I am also sure Tomas wouldn't
put experimental version of clutter near of release. We are eating our
own dog food so we are all interested in make it work, not the
opposed. In this case I think it can be well coordinated.

One possible thing is you could require AB used layers to be at
git.yoctoproject.org so in case of broken recipes (bbappend or so) you
could  rename it.

>> I'd really like for people to be able to just pick up uptodate and
>> working clutter packages for the major platforms the actively maintained
>> BSPs support.
>
> Agreed.
>
>>  Every so often someone asks about clutter for XYZ (usually
>> the Beagleboard or RPi) on the clutter list: this should be readily
>> available somewhere.
>
> I'd hope the recipes in the core would be in a good state in this
> regard.

Good weekend for everyone :-)

Regards,

--
Otavio Salvador                             O.S. Systems
E-mail: otavio@ossystems.com.br  http://www.ossystems.com.br
Mobile: +55 53 9981-7854              http://projetos.ossystems.com.br

Re: proposal to move cogl, clutter and related recipes from oe-core to dedicated meta-clutter layer

[Mark Hatle]

On 5/10/13 3:22 PM, Otavio Salvador wrote:
> On Fri, May 10, 2013 at 2:19 PM, Richard Purdie
> <richard.purdie@linuxfoundation.org> wrote:
>> On Fri, 2013-05-10 at 17:39 +0100, Tomas Frydrych wrote:
>>> On 10/05/13 12:32, Richard Purdie wrote:
>>>> On Fri, 2013-05-10 at 11:56 +0100, Tomas Frydrych wrote:
>>>>> On 10/05/13 10:05, Richard Purdie wrote:
>>>> The closely track upstream is the key part and I think the core can do
>>>> this, apart from the ~six week stabilisation window.
>>>
>>> If you mean you are prepared to do frequent point releases to keep up
>>> with clutter, that could work. But if you mean that those interested can
>>> closely track oe-core master, then that is not that useful, there are
>>> too many other changes happening at the same time. A small single
>>> purpose layer means updates (and breakages) can be very contained.
>>
>> Point releases of what? I don't mind clutter in master changing quite a
>> lot up to our stabilisation point. Once we've released, I don't mind
>> someone else picking up a danny-clutter branch or something like that
>> which is backporting specific changes.
>
> So we're going to have <branch>-<pet package> branches? I think this
> does not scale and I am at Tomas side. The layer seems the right thing
>   to do so it does not need to be done by branch and avoid confusing
> users.
>
>>>> My argument for this is that I really do want to stress out the graphics
>>>> stack we have, clutter provides a good way to test some of those
>>>> components, particularly some of the more unusual parts. Currently its
>>>> mostly build test however we do have plans to make that runtime too. I
>>>> do think that clutters unusual usage of the stack makes it particular
>>>> useful in this role. I'd appreciate help from anyone who can help make
>>>> this all work.
>>>
>>> I am all for this, but does using clutter for automated tests require
>>> for the clutter packages to be in oe-core? The only thing you can stress
>>> test in oe-core using clutter is mesa, which is only applicable to the
>>> i915/i965 chip sets. I think it would be useful if any such tests could
>>> be applied to other graphics stacks provided by BSPs; I think all of the
>>> BSPs would benefit from this.
>>
>> We'd be able to test mesa and the software fallbacks under qemu which
>> would be a start. If we can get those working with a decent framework
>> for the tests, I'm hoping wider BSP testing would follow. I don't have
>> unlimited resources available but I can try and get the software
>> infrastructure to the point where doing the testing could be picked up
>> by someone else relatively easily.
>
> You could do just the same but adding meta-clutter to the AB setup for
> testing. One thing does not conflict with the another.
>
>> It is a big help in trying to achieve this if we don't have many
>> different layers involved as that would complicate the problem, even
>> coordinating layer releases between different maintainers is tricky
>> right now and trying to develop something like this over layer
>> boundaries sounds like adding to the complexity to the point I'd rather
>> just scrap the idea :(.
>
> Well if layers make life harder it invalidates one of points of Yocto
> which I always liked and depends on heavily - modular design.

One of the key pieces of the oe-core, it must be able to work without any 
additional layers.  It also much be able to be tested without any additional layers.

I personally don't have a preference for the clutter and related items on where 
they live... but I do want to make sure that oe-core can standalone as a 
starting point for people to develop devices.  Part of being standalone means 
that it is capable of being tested.

--Mark

> I am sure we all want a solid release so I am also sure Tomas wouldn't
> put experimental version of clutter near of release. We are eating our
> own dog food so we are all interested in make it work, not the
> opposed. In this case I think it can be well coordinated.
>
> One possible thing is you could require AB used layers to be at
> git.yoctoproject.org so in case of broken recipes (bbappend or so) you
> could  rename it.
>
>>> I'd really like for people to be able to just pick up uptodate and
>>> working clutter packages for the major platforms the actively maintained
>>> BSPs support.
>>
>> Agreed.
>>
>>>   Every so often someone asks about clutter for XYZ (usually
>>> the Beagleboard or RPi) on the clutter list: this should be readily
>>> available somewhere.
>>
>> I'd hope the recipes in the core would be in a good state in this
>> regard.
>
> Good weekend for everyone :-)
>
> Regards,
>
> --
> Otavio Salvador                             O.S. Systems
> E-mail: otavio@ossystems.com.br  http://www.ossystems.com.br
> Mobile: +55 53 9981-7854              http://projetos.ossystems.com.br
>
> _______________________________________________
> Openembedded-core mailing list
> Openembedded-core@lists.openembedded.org
> http://lists.linuxtogo.org/cgi-bin/mailman/listinfo/openembedded-core
>

Re: proposal to move cogl, clutter and related recipes from oe-core to dedicated meta-clutter layer

[Otavio Salvador]

On Fri, May 10, 2013 at 5:37 PM, Mark Hatle <mark.hatle@windriver.com> wrote:
> On 5/10/13 3:22 PM, Otavio Salvador wrote:
>> Well if layers make life harder it invalidates one of points of Yocto
>> which I always liked and depends on heavily - modular design.
>
> One of the key pieces of the oe-core, it must be able to work without any
> additional layers.  It also much be able to be tested without any additional
> layers.
>
> I personally don't have a preference for the clutter and related items on
> where they live... but I do want to make sure that oe-core can standalone as
> a starting point for people to develop devices.  Part of being standalone
> means that it is capable of being tested.

This wouldn't be a regression as it was not used widely for testing.
Clutter would be much more useful for users if it could profit of a
more active users base and avoid 'in-house' solutions and forks so I
only see good points in split it out from OE-Core.

--
Otavio Salvador                             O.S. Systems
E-mail: otavio@ossystems.com.br  http://www.ossystems.com.br
Mobile: +55 53 9981-7854              http://projetos.ossystems.com.br

Re: proposal to move cogl, clutter and related recipes from oe-core to dedicated meta-clutter layer

[Tomas Frydrych]

Hi Mark,

On 10/05/13 21:37, Mark Hatle wrote:
> One of the key pieces of the oe-core, it must be able to work without
> any additional layers.  It also much be able to be tested without any
> additional layers.
> 
> I personally don't have a preference for the clutter and related items
> on where they live... but I do want to make sure that oe-core can
> standalone as a starting point for people to develop devices.  Part of
> being standalone means that it is capable of being tested.

Yes, but the clutter packages are not being tested, and more to the
point, they cannot be meaningfully tested using oe-core, because such
testing has to be done against a real OGL/GLES implementation -- Clutter
can only be tested properly in conjunction with BSP layer(s). All you
can test with oe-core is the mesa software rasterizer, which is just
about the one thing nobody cares about because it is of no practical use.

There are two ways that tested clutter packages are achievable:

a) Generic clutter recipes in oe-core and BSP specific bbappends in the
BSP layers. There are a couple of problems with this: I don't think the
BSP maintainers have time/impetus for maintaining the bbappends and
testing Clutter, and the BSP support might need upgrading the base
package and dependencies (plus Clutter is a fairly high level package to
be part of a BSP anyway, IMO).

b) A dedicated layer that is not subject to the oe-core constraints and
can make reference to external BSP layers.

Tomas

-- 
http://sleepfive.com

Re: proposal to move cogl, clutter and related recipes from oe-core to dedicated meta-clutter layer

[Phil Blundell]

On Mon, 2013-05-13 at 10:30 +0100, Tomas Frydrych wrote:
> Yes, but the clutter packages are not being tested, and more to the
> point, they cannot be meaningfully tested using oe-core, because such
> testing has to be done against a real OGL/GLES implementation -- Clutter
> can only be tested properly in conjunction with BSP layer(s). All you
> can test with oe-core is the mesa software rasterizer, which is just
> about the one thing nobody cares about because it is of no practical use.

It seems a bit hyperbolic to claim that testing clutter is impossible
without GPU hardware (either real or emulated).  The majority of the
code even in cogl, and virtually all the code in clutter itself, is
mostly independent of the underlying GL implementation.  From the point
of view of testing whether clutter basically "works" and is correctly
built/packaged it seems as though this ought to be quite sufficient.

Indeed, testing against Mesa is in some sense a benefit because you can
exercise both the GL and GLES2 backends (and even GLES1 if you so
desired) as well as all the windowing systems.

p.

Re: proposal to move cogl, clutter and related recipes from oe-core to dedicated meta-clutter layer

[Burton, Ross]

On 13 May 2013 16:41, Phil Blundell <pb@pbcl.net> wrote:
> It seems a bit hyperbolic to claim that testing clutter is impossible
> without GPU hardware (either real or emulated).  The majority of the
> code even in cogl, and virtually all the code in clutter itself, is
> mostly independent of the underlying GL implementation.  From the point
> of view of testing whether clutter basically "works" and is correctly
> built/packaged it seems as though this ought to be quite sufficient.
>
> Indeed, testing against Mesa is in some sense a benefit because you can
> exercise both the GL and GLES2 backends (and even GLES1 if you so
> desired) as well as all the windowing systems.

Well, the software renderer in Mesa that we're shipping doesn't work
with Clutter.  If we integrate llvm into oe-core then we can enable
llvmpipe which does actually work with Clutter.

Ross

Re: proposal to move cogl, clutter and related recipes from oe-core to dedicated meta-clutter layer

[Tomas Frydrych]

Hi Phil,

On 13/05/13 16:41, Phil Blundell wrote:
> It seems a bit hyperbolic to claim that testing clutter is impossible
> without GPU hardware (either real or emulated).  The majority of the
> code even in cogl, and virtually all the code in clutter itself, is
> mostly independent of the underlying GL implementation.  From the point
> of view of testing whether clutter basically "works" and is correctly
> built/packaged it seems as though this ought to be quite sufficient.

There are too separate issues: testing clutter itself, and using clutter
to test other parts of the graphics stack.

Re the former, all you can say after testing cogl/clutter against mesa
software rasterizer is that a particular configuration and a particular
backend (in oe-core that would be the GLX backend) work with mesa
software rasterizer. This says nothing about any other configuration,
with any other backend or whether it works with any other GL/GLES
implementation.

In reality, cogl/clutter need patches to build against the likes of TI's
GLES (e.g., Beagleboard) or to work on the likes of RaspberryPi, and
these patches cannot be included in oe-core, of course, because oe-core
knows nothing of such machines. Then there are little annoyances, like
the fact that over the last year or so, clutter has tended to have a
dependency on a specific but mostly random versions of libxkbcommon
(from what I can gather, determined more then anything by whatever the
Fedora packager happened to package and what version of Fedora the
clutter maintainer was using), or Clutter's customary insistence on as
recent glib as you can get. This can all be nicely encapsulated in small
layer.

Regarding using clutter to test other parts of the graphics stack, which
is what Richard is wanting clutter in oe-core for, this amounts to
testing mesa sw rasterizer. There is limited value here, it's the one GL
implementation that nobody using Yocto to build images will be deploying
for real, so this is a weak argument for needing clutter in oe-core.

Tomas

-- 
http://sleepfive.com

Re: proposal to move cogl, clutter and related recipes from oe-core to dedicated meta-clutter layer

[Paul Eggleton]

On Tuesday 14 May 2013 10:14:20 Tomas Frydrych wrote:
> On 13/05/13 16:41, Phil Blundell wrote:
> > It seems a bit hyperbolic to claim that testing clutter is impossible
> > without GPU hardware (either real or emulated).  The majority of the
> > code even in cogl, and virtually all the code in clutter itself, is
> > mostly independent of the underlying GL implementation.  From the point
> > of view of testing whether clutter basically "works" and is correctly
> > built/packaged it seems as though this ought to be quite sufficient.
> 
> There are too separate issues: testing clutter itself, and using clutter
> to test other parts of the graphics stack.
> 
> Re the former, all you can say after testing cogl/clutter against mesa
> software rasterizer is that a particular configuration and a particular
> backend (in oe-core that would be the GLX backend) work with mesa
> software rasterizer. This says nothing about any other configuration,
> with any other backend or whether it works with any other GL/GLES
> implementation.
> 
> In reality, cogl/clutter need patches to build against the likes of TI's
> GLES (e.g., Beagleboard) or to work on the likes of RaspberryPi, and
> these patches cannot be included in oe-core, of course, because oe-core
> knows nothing of such machines. 

Having clutter in OE-Core does not preclude such testing with additional BSPs, 
and I'm unclear on how moving it out to another layer helps at all with this 
specific issue.

> Then there are little annoyances, like
> the fact that over the last year or so, clutter has tended to have a
> dependency on a specific but mostly random versions of libxkbcommon
> (from what I can gather, determined more then anything by whatever the
> Fedora packager happened to package and what version of Fedora the
> clutter maintainer was using), or Clutter's customary insistence on as
> recent glib as you can get. This can all be nicely encapsulated in small
> layer.

This could present a problem. What if I want Clutter but I don't want the 
latest version of glib, but instead the version that is being shipped with OE-
Core that is tested with the other pieces of the system that depend upon it 
(especially given glib has recent history of breaking other packages)? Surely 
the safest alternative is the last stable version of Clutter that works with 
that version of glib? That would make it difficult to depend upon an external 
layer that provides its own newer version of glib would it not?

There's no denying that the maintenance of the Clutter recipes in OE-Core has 
slipped. I don't think that is an argument in itself to split them out, that 
just means we need to recognise that and maintain those recipes more 
effectively. It's not a case where we've missed the release because it was too 
close, we've not updated it for longer than that.

Honestly I think if Clutter continues to be something that people are using to 
develop applications we're much better off with the "canonical" stable version 
being in OE-Core.

Cheers,
Paul

-- 

Paul Eggleton
Intel Open Source Technology Centre

Re: proposal to move cogl, clutter and related recipes from oe-core to dedicated meta-clutter layer

[Tomas Frydrych]

Hi Paul,

On 14/05/13 17:55, Paul Eggleton wrote:
> Having clutter in OE-Core does not preclude such testing with additional BSPs, 
> and I'm unclear on how moving it out to another layer helps at all with this 
> specific issue.

It prevents efficiently supporting clutter on any real machine that does
not use mesa's GL, which means all machines not in meta-intel, and some
machines in meta-intel. This is the main issue, real HW support.


> This could present a problem. What if I want Clutter but I don't want the 
> latest version of glib, but instead the version that is being shipped with OE-
> Core that is tested with the other pieces of the system that depend upon it 
> (especially given glib has recent history of breaking other packages)? Surely 
> the safest alternative is the last stable version of Clutter that works with 
> that version of glib? That would make it difficult to depend upon an external 
> layer that provides its own newer version of glib would it not?

Sometimes a 6 month old release is not enough, and having to provide the
updated packages yourself is the least desirable of all options. In a
small layer, such issues can be handled gracefully, and their impact
limited.


> There's no denying that the maintenance of the Clutter recipes in OE-Core has 
> slipped. I don't think that is an argument in itself to split them out, that 
> just means we need to recognise that and maintain those recipes more 
> effectively.

The lack of maintenance reflects the relative importance of Clutter for
oe-core, and is an orthogonal issue. I am not complaining that it is not
being maintained, I am arguing that it cannot be properly maintained
with just reference to mesa and qemu, hence the suggestion to split it out.


> Honestly I think if Clutter continues to be something that people are using to 
> develop applications we're much better off with the "canonical" stable version 
> being in OE-Core.

Where 'canonical' means 'unusable on non-Intel HW', but I am repeating
myself ...

Tomas

Re: proposal to move cogl, clutter and related recipes from oe-core to dedicated meta-clutter layer

[Paul Eggleton]

On Wednesday 15 May 2013 10:19:57 Tomas Frydrych wrote:
> On 14/05/13 17:55, Paul Eggleton wrote:
> > Having clutter in OE-Core does not preclude such testing with additional
> > BSPs, and I'm unclear on how moving it out to another layer helps at all
> > with this specific issue.
> 
> It prevents efficiently supporting clutter on any real machine that does
> not use mesa's GL, which means all machines not in meta-intel, and some
> machines in meta-intel. This is the main issue, real HW support.

How does it prevent that? Surely if machine-specific changes are required then 
they will be required on top of a separate layer as much as they are if the 
recipes remain in OE-Core.

> > This could present a problem. What if I want Clutter but I don't want the
> > latest version of glib, but instead the version that is being shipped with
> > OE- Core that is tested with the other pieces of the system that depend
> > upon it (especially given glib has recent history of breaking other
> > packages)? Surely the safest alternative is the last stable version of
> > Clutter that works with that version of glib? That would make it
> > difficult to depend upon an external layer that provides its own newer
> > version of glib would it not?
> 
> Sometimes a 6 month old release is not enough, and having to provide the
> updated packages yourself is the least desirable of all options. In a
> small layer, such issues can be handled gracefully, and their impact
> limited.

Yes, I agree, "sometimes"; the same could be said for almost any piece of 
software; new features and fixes come along and some of those will be too 
important to some users to ignore. The layer mechanism exists to allow specific 
recipes to be extended if needed. Having the recipes in OE-Core does not 
preclude their extension or replacement with newer versions elsewhere for 
those that need it.

> > There's no denying that the maintenance of the Clutter recipes in OE-Core
> > has slipped. I don't think that is an argument in itself to split them
> > out, that just means we need to recognise that and maintain those recipes
> > more effectively.
> 
> The lack of maintenance reflects the relative importance of Clutter for
> oe-core, and is an orthogonal issue. I am not complaining that it is not
> being maintained, I am arguing that it cannot be properly maintained
> with just reference to mesa and qemu, hence the suggestion to split it out.

You may well be right about the need to test on other GL implementations. That 
does not explain how moving them to a separate layer directly helps to address 
that need. You must also expect to make some changes to the recipes 
themselves, so what changes would you be making?
 
> > Honestly I think if Clutter continues to be something that people are
> > using to develop applications we're much better off with the "canonical"
> > stable version being in OE-Core.
> 
> Where 'canonical' means 'unusable on non-Intel HW', but I am repeating
> myself ...

See above.

Cheers,
Paul

-- 

Paul Eggleton
Intel Open Source Technology Centre

Re: proposal to move cogl, clutter and related recipes from oe-core to dedicated meta-clutter layer

[Tomas Frydrych]

Hi Paul,

On 15/05/13 10:49, Paul Eggleton wrote:
>> It prevents efficiently supporting clutter on any real machine that does
>> not use mesa's GL, which means all machines not in meta-intel, and some
>> machines in meta-intel. This is the main issue, real HW support.
> 
> How does it prevent that? Surely if machine-specific changes are required then 
> they will be required on top of a separate layer as much as they are if the 
> recipes remain in OE-Core.

It could be all pulled together into the meta-clutter layer, the
supported BSPs and machines documented, etc, so that common machines
just work out of the box. We could have a dedicated mailing list, a bug
tracker, build a community around it, pull resources.


> The layer mechanism exists to allow specific 
> recipes to be extended if needed. Having the recipes in OE-Core does not 
> preclude their extension or replacement with newer versions elsewhere for 
> those that need it.

I have followed the model you advocate for over a year with clutter, and
it is a PITA, so I am thinking that perhaps there are others who are
doing the same and we could do it in one well known place.


> You may well be right about the need to test on other GL implementations.
> That does not explain how moving them to a separate layer directly helps to address 
> that need. You must also expect to make some changes to the recipes 
> themselves, so what changes would you be making?

It's not just about testing, you have to build it first: I would like to
see a set of recipes that can support a whole bunch of machines in the
public OE BSP layers out of the box: configs that work and make sense,
patches where needed, documentation, including documentation of BSP
specific issues.

In the absence of a community-owned meta-clutter layer, if anyone is
stuck maintaining their own clutter recipes, I have a set at
https://github.com/Guacamayo/meta-clutter which can perhaps be of some use.

Tomas


-- 
http://sleepfive.com

Re: proposal to move cogl, clutter and related recipes from oe-core to dedicated meta-clutter layer

[Otavio Salvador]

On Wed, May 15, 2013 at 8:35 AM, Tomas Frydrych
<tf+lists.yocto@r-finger.com> wrote:
> Hi Paul,
>
> On 15/05/13 10:49, Paul Eggleton wrote:
>>> It prevents efficiently supporting clutter on any real machine that does
>>> not use mesa's GL, which means all machines not in meta-intel, and some
>>> machines in meta-intel. This is the main issue, real HW support.
>>
>> How does it prevent that? Surely if machine-specific changes are required then
>> they will be required on top of a separate layer as much as they are if the
>> recipes remain in OE-Core.
>
> It could be all pulled together into the meta-clutter layer, the
> supported BSPs and machines documented, etc, so that common machines
> just work out of the box. We could have a dedicated mailing list, a bug
> tracker, build a community around it, pull resources.

+1

>> The layer mechanism exists to allow specific
>> recipes to be extended if needed. Having the recipes in OE-Core does not
>> preclude their extension or replacement with newer versions elsewhere for
>> those that need it.
>
> I have followed the model you advocate for over a year with clutter, and
> it is a PITA, so I am thinking that perhaps there are others who are
> doing the same and we could do it in one well known place.

+1

>> You may well be right about the need to test on other GL implementations.
>> That does not explain how moving them to a separate layer directly helps to address
>> that need. You must also expect to make some changes to the recipes
>> themselves, so what changes would you be making?
>
> It's not just about testing, you have to build it first: I would like to
> see a set of recipes that can support a whole bunch of machines in the
> public OE BSP layers out of the box: configs that work and make sense,
> patches where needed, documentation, including documentation of BSP
> specific issues.
>
> In the absence of a community-owned meta-clutter layer, if anyone is
> stuck maintaining their own clutter recipes, I have a set at
> https://github.com/Guacamayo/meta-clutter which can perhaps be of some use.

+1

I share same feeling of Tomas and I agree that a new layer is the way
to go. Having it in a specific layer will allow for more shared work
and easy a community creation around it.

--
Otavio Salvador                             O.S. Systems
E-mail: otavio@ossystems.com.br  http://www.ossystems.com.br
Mobile: +55 53 9981-7854              http://projetos.ossystems.com.br

Re: proposal to move cogl, clutter and related recipes from oe-core to dedicated meta-clutter layer

[Andreas Oberritter]

On 15.05.2013 11:53, Otavio Salvador wrote:
> On Wed, May 15, 2013 at 8:35 AM, Tomas Frydrych
> <tf+lists.yocto@r-finger.com> wrote:
>> Hi Paul,
>>
>> On 15/05/13 10:49, Paul Eggleton wrote:
>>>> It prevents efficiently supporting clutter on any real machine that does
>>>> not use mesa's GL, which means all machines not in meta-intel, and some
>>>> machines in meta-intel. This is the main issue, real HW support.
>>>
>>> How does it prevent that? Surely if machine-specific changes are required then
>>> they will be required on top of a separate layer as much as they are if the
>>> recipes remain in OE-Core.
>>
>> It could be all pulled together into the meta-clutter layer, the
>> supported BSPs and machines documented, etc, so that common machines
>> just work out of the box. We could have a dedicated mailing list, a bug
>> tracker, build a community around it, pull resources.
> 
> +1
> 
>>> The layer mechanism exists to allow specific
>>> recipes to be extended if needed. Having the recipes in OE-Core does not
>>> preclude their extension or replacement with newer versions elsewhere for
>>> those that need it.
>>
>> I have followed the model you advocate for over a year with clutter, and
>> it is a PITA, so I am thinking that perhaps there are others who are
>> doing the same and we could do it in one well known place.
> 
> +1
> 
>>> You may well be right about the need to test on other GL implementations.
>>> That does not explain how moving them to a separate layer directly helps to address
>>> that need. You must also expect to make some changes to the recipes
>>> themselves, so what changes would you be making?
>>
>> It's not just about testing, you have to build it first: I would like to
>> see a set of recipes that can support a whole bunch of machines in the
>> public OE BSP layers out of the box: configs that work and make sense,
>> patches where needed, documentation, including documentation of BSP
>> specific issues.
>>
>> In the absence of a community-owned meta-clutter layer, if anyone is
>> stuck maintaining their own clutter recipes, I have a set at
>> https://github.com/Guacamayo/meta-clutter which can perhaps be of some use.
> 
> +1
> 
> I share same feeling of Tomas and I agree that a new layer is the way
> to go. Having it in a specific layer will allow for more shared work
> and easy a community creation around it.

I fail to see why the presence of meta-clutter contradicts keeping
clutter and related recipes in oe-core. If you add meta-clutter to your
layers, then its recipes will override those in oe-core anyway.

Of course, clutter recipes in oe-core could see some maintenance, but
that's IMO a different topic.

Regards,
Andreas

Re: proposal to move cogl, clutter and related recipes from oe-core to dedicated meta-clutter layer

[Paul Eggleton]

On Wednesday 15 May 2013 12:35:13 Tomas Frydrych wrote:
> On 15/05/13 10:49, Paul Eggleton wrote:
> >> It prevents efficiently supporting clutter on any real machine that does
> >> not use mesa's GL, which means all machines not in meta-intel, and some
> >> machines in meta-intel. This is the main issue, real HW support.
> > 
> > How does it prevent that? Surely if machine-specific changes are required
> > then they will be required on top of a separate layer as much as they are
> > if the recipes remain in OE-Core.
> 
> It could be all pulled together into the meta-clutter layer, the
> supported BSPs and machines documented, etc, so that common machines
> just work out of the box. We could have a dedicated mailing list, a bug
> tracker, build a community around it, pull resources.

So we already have those things covering OE-Core. I would note that we already 
get complaints from users that there are multiple mailing lists covering OE, 
adding another one is just going to make it more difficult for users to figure 
out where to send questions, split discussions that start out being Clutter-
related but have a wider impact than just Clutter, etc. Clutter being a core 
piece of functionality and depending on BSP-specific functionality strengthens 
the argument for having discussions about it in a more general place rather 
than a specific one.

I understand the problem of Clutter releases apparently having unfortunate 
timing in relation to the OE-Core stabilisation period. That may be grounds 
for maintaining a staging layer for newer Clutter recipes that can be shared 
by those who need to have the latest stable version. It is not in itself 
grounds for completely moving those recipes out of the core.

> > You may well be right about the need to test on other GL implementations.
> > That does not explain how moving them to a separate layer directly helps
> > to address that need. You must also expect to make some changes to the
> > recipes themselves, so what changes would you be making?
> 
> It's not just about testing, you have to build it first: I would like to
> see a set of recipes that can support a whole bunch of machines in the
> public OE BSP layers out of the box: configs that work and make sense,
> patches where needed, documentation, including documentation of BSP
> specific issues.

If you're suggesting effectively taking a subset of machine-specific items out 
of BSPs and into a separate non-BSP layer, this is not a good plan IMO. I know 
you have been doing this in Guacamayo already and frankly it has always 
concerned me. Can you not just get the appropriate changes into the BSP layers 
so that when you add the BSP on top of OE-Core it does just work "out of the 
box"? If clutter is taken out of OE-Core this becomes even harder because then 
BSPs can no longer bbappend clutter or cogl (if that is indeed what is 
required in order to enable machine-specific functionality) without fiddling 
around with BBMASK or keeping the appends in yet another separate layer so 
they don't impact other BSP users who aren't building Clutter.

Cheers,
Paul

-- 

Paul Eggleton
Intel Open Source Technology Centre

Re: proposal to move cogl, clutter and related recipes from oe-core to dedicated meta-clutter layer

[Tomas Frydrych]

Hi Paul,

On 15/05/13 15:09, Paul Eggleton wrote:
> Can you not just get the appropriate changes into the BSP layers 
> so that when you add the BSP on top of OE-Core it does just work "out of the 
> box"?

What you are really saying is that the onus of maintaining working
clutter packages should be on the BSP maintainers. I have already
explained previously in this thread why I don't think this is going to
work out in practice, so I am not going to repeat myself again.


> If clutter is taken out of OE-Core this becomes even harder because then 
> BSPs can no longer bbappend clutter or cogl (if that is indeed what is 
> required in order to enable machine-specific functionality) without fiddling 
> around with BBMASK or keeping the appends in yet another separate layer so 
> they don't impact other BSP users who aren't building Clutter.

The BSPs would not need to do anything regarding clutter, no bbappends,
meta-clutter would be self-contained in this regard.

I don't think there is any point in bothering the list with this
discussion any further, we have gone around the circle more than once.
The whole point of the proposal was to make maintaining, using and
supporting the clutter packages easier for everyone, but it seems I
underestimated how important Clutter is to oe-core, and I am already
looking forward to this newly discovered importance being translated
into user happiness before too long. :)

Tomas

Re: proposal to move cogl, clutter and related recipes from oe-core to dedicated meta-clutter layer

[Otavio Salvador]

On Wed, May 15, 2013 at 1:34 PM, Tomas Frydrych
<tf+lists.yocto@r-finger.com> wrote:
> Hi Paul,
>
> On 15/05/13 15:09, Paul Eggleton wrote:
>> Can you not just get the appropriate changes into the BSP layers
>> so that when you add the BSP on top of OE-Core it does just work "out of the
>> box"?
>
> What you are really saying is that the onus of maintaining working
> clutter packages should be on the BSP maintainers. I have already
> explained previously in this thread why I don't think this is going to
> work out in practice, so I am not going to repeat myself again.
>
>
>> If clutter is taken out of OE-Core this becomes even harder because then
>> BSPs can no longer bbappend clutter or cogl (if that is indeed what is
>> required in order to enable machine-specific functionality) without fiddling
>> around with BBMASK or keeping the appends in yet another separate layer so
>> they don't impact other BSP users who aren't building Clutter.
>
> The BSPs would not need to do anything regarding clutter, no bbappends,
> meta-clutter would be self-contained in this regard.
>
> I don't think there is any point in bothering the list with this
> discussion any further, we have gone around the circle more than once.
> The whole point of the proposal was to make maintaining, using and
> supporting the clutter packages easier for everyone, but it seems I
> underestimated how important Clutter is to oe-core, and I am already
> looking forward to this newly discovered importance being translated
> into user happiness before too long. :)

I'd prefer we keep meta-clutter working until it happens ;-)

--
Otavio Salvador                             O.S. Systems
E-mail: otavio@ossystems.com.br  http://www.ossystems.com.br
Mobile: +55 53 9981-7854              http://projetos.ossystems.com.br

Re: proposal to move cogl, clutter and related recipes from oe-core to dedicated meta-clutter layer

[Paul Eggleton]

On Wednesday 15 May 2013 17:34:45 Tomas Frydrych wrote:
> On 15/05/13 15:09, Paul Eggleton wrote:
> > Can you not just get the appropriate changes into the BSP layers
> > so that when you add the BSP on top of OE-Core it does just work "out of
> > the box"?
> 
> What you are really saying is that the onus of maintaining working
> clutter packages should be on the BSP maintainers. I have already
> explained previously in this thread why I don't think this is going to
> work out in practice, so I am not going to repeat myself again.

No, I'm not. I'm saying configuration specific to a machine, if there is any, 
needs to be in the BSP layer for that machine. Whether the maintainer of that 
BSP or some external contributor provides and maintains that is a separate 
question.

If there are clutter tests we can run on the Yocto Project reference hardware 
platforms (which I understand from Bruce may be refreshed for 1.5) and they 
could be run in an automated manner, we can run these on the autobuilder and 
this will help prove the functionality of clutter itself. Once the tests are 
set up it should be easy for others to run these regularly on their own 
autobuilders with other BSPs.

> > If clutter is taken out of OE-Core this becomes even harder because then
> > BSPs can no longer bbappend clutter or cogl (if that is indeed what is
> > required in order to enable machine-specific functionality) without
> > fiddling around with BBMASK or keeping the appends in yet another
> > separate layer so they don't impact other BSP users who aren't building
> > Clutter.
> 
> The BSPs would not need to do anything regarding clutter, no bbappends,
> meta-clutter would be self-contained in this regard.

First you say there are hardware specific bits, then you say there aren't any 
changes required. I'm not following at all.

> I don't think there is any point in bothering the list with this
> discussion any further, we have gone around the circle more than once.

Sorry, but I don't think the important questions have been answered yet.

Cheers,
Paul

-- 

Paul Eggleton
Intel Open Source Technology Centre

Re: proposal to move cogl, clutter and related recipes from oe-core to dedicated meta-clutter layer

[Richard Purdie]

I'd ask that we give trying to make this work from OE-Core a fair try. 

If in say 3 months from now OE-Core is causing clutter users pain we can
look again at the problem. Equally, if nobody has sent me any clutter
patches in the next three months, I wouldn't consider than a fair try.

I'm happy enough to see the clutter recipes architecture changed to
better support their users. If that means adding some of the machine
specific tweaks to the core as examples I'm also ok with that rather
than forcing those into BSP layers. I'm not saying we'll take every
machine entry into the core but enough to show its usages would be
acceptable in my view.

I do agree with the view that we have enough mailing lists and so on and
adding more is not appropriate where this is at right now.

Cheers,

Richard

Re: proposal to move cogl, clutter and related recipes from oe-core to dedicated meta-clutter layer

[Otavio Salvador]

On Wed, May 15, 2013 at 2:30 PM, Richard Purdie
<richard.purdie@linuxfoundation.org> wrote:
> I'd ask that we give trying to make this work from OE-Core a fair try.
>
> If in say 3 months from now OE-Core is causing clutter users pain we can
> look again at the problem. Equally, if nobody has sent me any clutter
> patches in the next three months, I wouldn't consider than a fair try.

No, this is not fair. If you wish to keep it you need to do the work
or find someone to make it. Tomas is willing to work in meta-clutter
and he has the right to work in his solution.

--
Otavio Salvador                             O.S. Systems
E-mail: otavio@ossystems.com.br  http://www.ossystems.com.br
Mobile: +55 53 9981-7854              http://projetos.ossystems.com.br

Re: proposal to move cogl, clutter and related recipes from oe-core to dedicated meta-clutter layer

[Paul Eggleton]

On Wednesday 15 May 2013 14:36:16 Otavio Salvador wrote:
> On Wed, May 15, 2013 at 2:30 PM, Richard Purdie
> <richard.purdie@linuxfoundation.org> wrote:
> > I'd ask that we give trying to make this work from OE-Core a fair try.
> > 
> > If in say 3 months from now OE-Core is causing clutter users pain we can
> > look again at the problem. Equally, if nobody has sent me any clutter
> > patches in the next three months, I wouldn't consider than a fair try.
> 
> No, this is not fair. If you wish to keep it you need to do the work
> or find someone to make it. Tomas is willing to work in meta-clutter
> and he has the right to work in his solution.

Nobody's suggesting Tomas can't work on meta-clutter. This is an open source 
project, people are pretty much free to contribute as they see fit. For my 
part, I will try to make sure the clutter recipes in OE-Core are better 
maintained in future.

However, I do think it's important for us to try to work together for the 
benefit of everyone, and when it comes to OE-Core and other central pieces of 
the OE ecosystem, that means trying to understand the bigger picture outside 
of the day-to-day requirements that we all have on the projects we work on 
individually.

Cheers,
Paul

-- 

Paul Eggleton
Intel Open Source Technology Centre

Re: proposal to move cogl, clutter and related recipes from oe-core to dedicated meta-clutter layer

[Otavio Salvador]

On Wed, May 15, 2013 at 3:24 PM, Paul Eggleton
<paul.eggleton@linux.intel.com> wrote:
> On Wednesday 15 May 2013 14:36:16 Otavio Salvador wrote:
>> On Wed, May 15, 2013 at 2:30 PM, Richard Purdie
>> <richard.purdie@linuxfoundation.org> wrote:
>> > I'd ask that we give trying to make this work from OE-Core a fair try.
>> >
>> > If in say 3 months from now OE-Core is causing clutter users pain we can
>> > look again at the problem. Equally, if nobody has sent me any clutter
>> > patches in the next three months, I wouldn't consider than a fair try.
>>
>> No, this is not fair. If you wish to keep it you need to do the work
>> or find someone to make it. Tomas is willing to work in meta-clutter
>> and he has the right to work in his solution.
>
> Nobody's suggesting Tomas can't work on meta-clutter. This is an open source
> project, people are pretty much free to contribute as they see fit. For my
> part, I will try to make sure the clutter recipes in OE-Core are better
> maintained in future.
>
> However, I do think it's important for us to try to work together for the
> benefit of everyone, and when it comes to OE-Core and other central pieces of
> the OE ecosystem, that means trying to understand the bigger picture outside
> of the day-to-day requirements that we all have on the projects we work on
> individually.

I agree but it seems it hadn't succeed in this specific case until
now. I personally think Clutter will benefit from getting a specific
place to look at and it does seem multiple people has been adding
Clutter recipes in their internal layers (Phil for example, Guacamayo
project and so on).

People did group in Guacamayo community and got it working in some
platforms, it might have been backported to OE-Core by Yocto team as
it is important for the test coverage but it didn't happen until now.
So far, meta-clutter seems to be the way to go for me.

--
Otavio Salvador                             O.S. Systems
E-mail: otavio@ossystems.com.br  http://www.ossystems.com.br
Mobile: +55 53 9981-7854              http://projetos.ossystems.com.br

Re: proposal to move cogl, clutter and related recipes from oe-core to dedicated meta-clutter layer

[Phil Blundell]

On Wed, 2013-05-15 at 16:28 -0300, Otavio Salvador wrote:
> I agree but it seems it hadn't succeed in this specific case until
> now. I personally think Clutter will benefit from getting a specific
> place to look at and it does seem multiple people has been adding
> Clutter recipes in their internal layers (Phil for example, Guacamayo
> project and so on).

I did actually have a couple of goes at trying to factor out some of our
local changes to the clutter recipes for submission to oe-core, but on
each occasion I gave up because the effort seemed to outweigh the
benefits. 

For what it's worth, some of the ways in which our local recipes diverge
from what's in oe-core are:

- we have a different (newer) version

- we build from a local git checkout, srctree-style, because our sources
are significantly patched compared to upstream

- we use eglnative mostly, though we might start wanting to use glx
under qemu for testing (subject to getting a suitable mesa)

- we have a slightly funky 2-stage bootstrap process for cogl in order
to break the dependency cycle with cairo; this involves hacks to the
recipes for cogl, cairo, pango and harfbuzz (at least) which I suspect
would not be very palatable to oe-core.

The net result of all this is that, whenever I try to factor out a set
of stuff that's "generic clutter" and could go into oe-core, I end up
with recipes that have virtually nothing in common with what we're
actually using and consequently don't actually solve any of my problems.
However, I have no doubt that someone cleverer than me could do a better
job of it.

p.

Re: proposal to move cogl, clutter and related recipes from oe-core to dedicated meta-clutter layer

[Tomas Frydrych]

Hi Phil,

On 15/05/13 21:49, Phil Blundell wrote:
> On Wed, 2013-05-15 at 16:28 -0300, Otavio Salvador wrote:
> - we have a different (newer) version

I think particularly for point updates we should be able to minimize the
pain if the base recipes are set up well.


> - we use eglnative mostly, though we might start wanting to use glx
> under qemu for testing (subject to getting a suitable mesa)

This is the crux of the difficulties with cogl/clutter. In Guacamayo I
need to be able to use both eglnative and glx. I prefer eglnative +
gles2, because I only need a single clutter based app running and the
X11 overhead is not negligible, but on intel HW I have not been able to
get this working satisfactorily in the past, so ended up using GLX for
the likes of atom-pc and NUC.

The solution I came up with is to predefine a bunch common
configure+depends+rdepends sets in the clutter/cogl includes (there is
only a finite number of configurations that makes sense, though my
recipes do not cover them all), and then in a Guacamayo-specific
bbappend choose a suitable configuration on per-machine basis.


> - we have a slightly funky 2-stage bootstrap process for cogl in order
> to break the dependency cycle with cairo; this involves hacks to the
> recipes for cogl, cairo, pango and harfbuzz (at least) which I suspect
> would not be very palatable to oe-core.

I have never run into this, is this with recent cogls?


> The net result of all this is that, whenever I try to factor out a set
> of stuff that's "generic clutter" and could go into oe-core, I end up
> with recipes that have virtually nothing in common with what we're
> actually using and consequently don't actually solve any of my problems.

I don't think we can create a set of recipes that 'just work' for
everyone, but we can have recipes that minimal effort and that also
provide sensible defaults for the common machines out there.

There are some broader issues here though that need thought,
particularly, a bbappend should be a method of last resort, it would be
nice have a feature mechanism to ease the configuration. But distro
features are of no use here because it is not possible to differentiate
packages with different configs based on DF; machine features don't have
this limitation, but are the wrong place for this, plus it is entirely
conceivable you might want be able to build different configs for the
same machine on the same tmp dir (I use pseudo-machines for this, like
atom-egl, but that is just a nasty hack).

Tomas

-- 
http://sleepfive.com

Re: proposal to move cogl, clutter and related recipes from oe-core to dedicated meta-clutter layer

[Phil Blundell]

On Thu, 2013-05-16 at 10:01 +0100, Tomas Frydrych wrote:
> The solution I came up with is to predefine a bunch common
> configure+depends+rdepends sets in the clutter/cogl includes (there is
> only a finite number of configurations that makes sense, though my
> recipes do not cover them all), and then in a Guacamayo-specific
> bbappend choose a suitable configuration on per-machine basis.

Right, that sounds fairly reasonable.  Or one could presumably use
PACKAGECONFIG for this sort of thing.

> > - we have a slightly funky 2-stage bootstrap process for cogl in order
> > to break the dependency cycle with cairo; this involves hacks to the
> > recipes for cogl, cairo, pango and harfbuzz (at least) which I suspect
> > would not be very palatable to oe-core.
> 
> I have never run into this, is this with recent cogls?

It's because we build Cairo with the cogl backend enabled.  That
introduces a dependency of cairo on cogl (obviously), which is a problem
because cogl-pango needs pango, which needs harfbuzz, which needs cairo.
So what we do is build cogl initially with pango disabled, then use that
to compile cairo and the rest of the stack, and then finally build the
"real" cogl with everything enabled.

Obviously the other option would be to build cairo twice, firstly
without cogl and the second time with it.  I don't think there's much to
choose between the two.

> this limitation, but are the wrong place for this, plus it is entirely
> conceivable you might want be able to build different configs for the
> same machine on the same tmp dir (I use pseudo-machines for this, like
> atom-egl, but that is just a nasty hack).

This is something that's just fundamentally difficult in OE; there
simply isn't any namespace to express that degree of freedom.  DISTRO is
essentially invariant for any given tmpdir, and the hierarchy in there
reflects MACHINE and PN.  So, if you want to build the same package with
a different configuration then either MACHINE or PN is going to have to
change.  Traditionally of course it's been PN that changes in this
situation.

p.

Re: proposal to move cogl, clutter and related recipes from oe-core to dedicated meta-clutter layer

[Tomas Frydrych]

On 16/05/13 11:35, Phil Blundell wrote:
> On Thu, 2013-05-16 at 10:01 +0100, Tomas Frydrych wrote:
>> The solution I came up with is to predefine a bunch common
>> configure+depends+rdepends sets in the clutter/cogl includes (there is
>> only a finite number of configurations that makes sense, though my
>> recipes do not cover them all), and then in a Guacamayo-specific
>> bbappend choose a suitable configuration on per-machine basis.
> 
> Right, that sounds fairly reasonable.  Or one could presumably use
> PACKAGECONFIG for this sort of thing.

Yep, that's one of the things I need to clean up in my own recipes.


> It's because we build Cairo with the cogl backend enabled.  That
> introduces a dependency of cairo on cogl (obviously), which is a problem
> because cogl-pango needs pango, which needs harfbuzz, which needs cairo.
> So what we do is build cogl initially with pango disabled, then use that
> to compile cairo and the rest of the stack, and then finally build the
> "real" cogl with everything enabled.

This would probably merit some sort of cogl-initial recipe to add.


> This is something that's just fundamentally difficult in OE; there
> simply isn't any namespace to express that degree of freedom.  DISTRO is
> essentially invariant for any given tmpdir, and the hierarchy in there
> reflects MACHINE and PN.  So, if you want to build the same package with
> a different configuration then either MACHINE or PN is going to have to
> change.  Traditionally of course it's been PN that changes in this
> situation.

I originally went down the PN route, but that meant having to specify
preferred providers and in my use case the sole criterion was the
MACHINE. But for a generic solution, it's probably necessary to have
some PN mechanism in place, maybe the keys in PACKAGECONFIG could be
used to automatically create a mangled PN for non-standard configs.

Tomas

Re: proposal to move cogl, clutter and related recipes from oe-core to dedicated meta-clutter layer

[Phil Blundell]

On Thu, 2013-05-16 at 12:21 +0100, Tomas Frydrych wrote:
> This would probably merit some sort of cogl-initial recipe to add.

Yeah.  If it was just a case of doing that then I would have done so;
the trouble is that you end up having to put garbage like

# We need to use the nopango build of cogl because we are a build dependency of pango
PKG_CONFIG_PATH_prepend = "${STAGING_DIR_HOST}/${exec_prefix}/cogl-nopango/${baselib}/pkgconfig:"
TARGET_LDFLAGS += "-Wl,-rpath-link,${STAGING_DIR_HOST}/${exec_prefix}/cogl-nopango/${baselib}"

in the recipes for everything else that participates in this dependency
chain and needs building before the "real" cogl.  Which is ugly.

p.

Re: proposal to move cogl, clutter and related recipes from oe-core to dedicated meta-clutter layer

[Paul Eggleton]

On Thursday 16 May 2013 11:35:55 Phil Blundell wrote:
> On Thu, 2013-05-16 at 10:01 +0100, Tomas Frydrych wrote:
> > On 15/05/13 21:49, Phil Blundell wrote:
> > > - we have a slightly funky 2-stage bootstrap process for cogl in order
> > > to break the dependency cycle with cairo; this involves hacks to the
> > > recipes for cogl, cairo, pango and harfbuzz (at least) which I suspect
> > > would not be very palatable to oe-core.
> > 
> > I have never run into this, is this with recent cogls?
> 
> It's because we build Cairo with the cogl backend enabled.  That
> introduces a dependency of cairo on cogl (obviously), which is a problem
> because cogl-pango needs pango, which needs harfbuzz, which needs cairo.
> So what we do is build cogl initially with pango disabled, then use that
> to compile cairo and the rest of the stack, and then finally build the
> "real" cogl with everything enabled.
> 
> Obviously the other option would be to build cairo twice, firstly
> without cogl and the second time with it.  I don't think there's much to
> choose between the two.

I was just speaking to one of the cogl developers and he was surprised that 
anyone would be using cairo's cogl backend since it's never really been 
finished. Is that backend definitely functionality that you're using?

Cheers,
Paul

-- 

Paul Eggleton
Intel Open Source Technology Centre

Re: proposal to move cogl, clutter and related recipes from oe-core to dedicated meta-clutter layer

[Tomas Frydrych]

On 15/05/13 19:24, Paul Eggleton wrote:
> However, I do think it's important for us to try to work together for the 
> benefit of everyone, 

I entirely agree with this; I certainly don't want to maintain a public
meta-clutter layer that competes with, or is perceived to compete with
some other recipes or efforts (oe-core or otherwise), this benefits no
one, least of all anyone interested in being just a 'user' of these
packages.

Tomas

-- 
http://sleepfive.com

Re: proposal to move cogl, clutter and related recipes from oe-core to dedicated meta-clutter layer

[Richard Purdie]

On Wed, 2013-05-15 at 20:30 +0300, Richard Purdie wrote:
> I'd ask that we give trying to make this work from OE-Core a fair try. 
> 
> If in say 3 months from now OE-Core is causing clutter users pain we can
> look again at the problem. Equally, if nobody has sent me any clutter
> patches in the next three months, I wouldn't consider than a fair try.
> 
> I'm happy enough to see the clutter recipes architecture changed to
> better support their users. If that means adding some of the machine
> specific tweaks to the core as examples I'm also ok with that rather
> than forcing those into BSP layers. I'm not saying we'll take every
> machine entry into the core but enough to show its usages would be
> acceptable in my view.
> 
> I do agree with the view that we have enough mailing lists and so on and
> adding more is not appropriate where this is at right now.

Just to add, I went and looked at the code/data in meta-clutter again.
The README says "meta-clutter depends only on oe-core and is meant to be
used with the current stable release of it (currently dylan).".

How about the master branch gets renamed "dylan" so it clearly show what
its meant to work against and follows the other repositories?

I'd still propose we try and get the core pieces into OE-Core master so
there is no need for a master branch of that repo. Currently if you try
that repo with master, there will be duplicated sstate postfuncs being
run which are probably harmless. There are also other duplicated recipes
(master glib-2.0 is newer). Over time as master and dylan diverge there
probably will be worse compatibility issues.

Cheers,

Richard

Re: proposal to move cogl, clutter and related recipes from oe-core to dedicated meta-clutter layer

[Tomas Frydrych]

On 15/05/13 20:43, Richard Purdie wrote:
> On Wed, 2013-05-15 at 20:30 +0300, Richard Purdie wrote:
> How about the master branch gets renamed "dylan" so it clearly show what
> its meant to work against and follows the other repositories?

Yes, that's on my TODO list, together with removing the couple of
unnecessary updated recipes from master when that is done (glib,
libxkbcommon).

Just to clarify the Guacamayo/meta-clutter was not set up with the view
to this proposal for an separate meta-clutter layer. I created it
because I need somewhere with uptodate clutter and associated recipes
(e.g., Mutter) so I could get on with the meta-gir (GObject
Introspection) work and did not want to make that depend on
meta-gucamayo. But I'd really prefer any such meta-clutter layer not to
be my private sandbox, hence this proposal.

Tomas

-- 
http://sleepfive.com

Re: proposal to move cogl, clutter and related recipes from oe-core to dedicated meta-clutter layer

[Martin Jansa]

[-- Attachment #1: Type: text/plain, Size: 1091 bytes --]

On Fri, May 10, 2013 at 05:22:47PM -0300, Otavio Salvador wrote:
> Well if layers make life harder it invalidates one of points of Yocto
> which I always liked and depends on heavily - modular design.

+1

people are using many layers, so oe-core being regularly tested together with
some other layers is just another test for propagated modular design.

> I am sure we all want a solid release so I am also sure Tomas wouldn't
> put experimental version of clutter near of release. We are eating our
> own dog food so we are all interested in make it work, not the
> opposed. In this case I think it can be well coordinated.

he can even put experimental version there if it's in recipe with
negative DEFAULT_PREFERRENCE, unfortunately he cannot do the same if
experimantal recipes are in meta-clutter with higher priority and stable
recipes in oe-core (unless he makes sure that everybody who is using
meta-clutter will understand why he needs to set PREFERRED_VERSION to
prevent experimental being used).

-- 
Martin 'JaMa' Jansa     jabber: Martin.Jansa@gmail.com

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 205 bytes --]

Re: proposal to move cogl, clutter and related recipes from oe-core to dedicated meta-clutter layer

[Richard Purdie]

On Fri, 2013-05-10 at 17:22 -0300, Otavio Salvador wrote:
> On Fri, May 10, 2013 at 2:19 PM, Richard Purdie
> <richard.purdie@linuxfoundation.org> wrote:
> > On Fri, 2013-05-10 at 17:39 +0100, Tomas Frydrych wrote:
> >> On 10/05/13 12:32, Richard Purdie wrote:
> >> > On Fri, 2013-05-10 at 11:56 +0100, Tomas Frydrych wrote:
> >> >> On 10/05/13 10:05, Richard Purdie wrote:
> >> > The closely track upstream is the key part and I think the core can do
> >> > this, apart from the ~six week stabilisation window.
> >>
> >> If you mean you are prepared to do frequent point releases to keep up
> >> with clutter, that could work. But if you mean that those interested can
> >> closely track oe-core master, then that is not that useful, there are
> >> too many other changes happening at the same time. A small single
> >> purpose layer means updates (and breakages) can be very contained.
> >
> > Point releases of what? I don't mind clutter in master changing quite a
> > lot up to our stabilisation point. Once we've released, I don't mind
> > someone else picking up a danny-clutter branch or something like that
> > which is backporting specific changes.
> 
> So we're going to have <branch>-<pet package> branches?

There may be a time and a place for such things and its not different to
the situation you'll end up in with a meta-clutter repository where
you'll likely end up with danny-1.12 and danny-1.14 branches, or
multiple sets of recipes in a single danny branch, thereby changing the
definition of "stable" from that used in the core which will be
confusing in itself.

>  I think this does not scale and I am at Tomas side. The layer seems
> the right thing to do so it does not need to be done by branch and
> avoid confusing users.

You're swapping one set of problems for another set the way I see it.

> >> > My argument for this is that I really do want to stress out the graphics
> >> > stack we have, clutter provides a good way to test some of those
> >> > components, particularly some of the more unusual parts. Currently its
> >> > mostly build test however we do have plans to make that runtime too. I
> >> > do think that clutters unusual usage of the stack makes it particular
> >> > useful in this role. I'd appreciate help from anyone who can help make
> >> > this all work.
> >>
> >> I am all for this, but does using clutter for automated tests require
> >> for the clutter packages to be in oe-core? The only thing you can stress
> >> test in oe-core using clutter is mesa, which is only applicable to the
> >> i915/i965 chip sets. I think it would be useful if any such tests could
> >> be applied to other graphics stacks provided by BSPs; I think all of the
> >> BSPs would benefit from this.
> >
> > We'd be able to test mesa and the software fallbacks under qemu which
> > would be a start. If we can get those working with a decent framework
> > for the tests, I'm hoping wider BSP testing would follow. I don't have
> > unlimited resources available but I can try and get the software
> > infrastructure to the point where doing the testing could be picked up
> > by someone else relatively easily.
> 
> You could do just the same but adding meta-clutter to the AB setup for
> testing. One thing does not conflict with the another.

I've always been extremely clear that I believe the core needs to stand
on its own, including being testable. I appreciate some people disagree
with that however I believe that without this, the quality in the core
would drop substantially. Even maintaining the testability of what we
have now is actually an incredible challenge. Anyone who thinks
otherwise should spend a week triaging the autobuilder failures.

Adding mode dimensions to a problem we already struggle with seems like
a bad idea to me.

> > It is a big help in trying to achieve this if we don't have many
> > different layers involved as that would complicate the problem, even
> > coordinating layer releases between different maintainers is tricky
> > right now and trying to develop something like this over layer
> > boundaries sounds like adding to the complexity to the point I'd rather
> > just scrap the idea :(.
> 
> Well if layers make life harder it invalidates one of points of Yocto
> which I always liked and depends on heavily - modular design.

The layer model has its risks as well as its advantages. The risk is
that we end up too fragmented, with too many maintainers who don't
communicate and nothing ever works together.

Even as the model stands today, its showing signs of strain. I posted
patches near enough three weeks ago for meta-raspberrypi, no response. I
know there are reasons why but they're not visible. The meta-fsl layers
appear to spend more of their time not building than building and
getting people to fix those in a timely fashion can sometimes be
challenging, particularly with the split between arm and ppc and some of
the transitions going on there etc.

Right now, I think there are enough splits causing enough
interoperability issues to be going on with in the layers and no, to be
quite honest I don't think more layers is the right answer right now.

I do continue to believe in the layer model but it is still evolving and
I'd really like us to focus on the current issues.

> I am sure we all want a solid release so I am also sure Tomas wouldn't
> put experimental version of clutter near of release.

From what I read of Tomas' emails, I actually think his plans differ
from that as he finds that aspect of OE-Core frustrating.

>  We are eating our
> own dog food so we are all interested in make it work, not the
> opposed. In this case I think it can be well coordinated.
>
> One possible thing is you could require AB used layers to be at
> git.yoctoproject.org so in case of broken recipes (bbappend or so) you
> could  rename it.

I don't touch layers I don't maintain. If I did that, I would get hung,
drawn and quartered. Please at least think through things like this
before saying them :(. Can you imagine what you'd say to me if I pushed
commits to meta-fsl-arm? 

This isn't even hypothetical, I have already been "told off" for pushing
to repos I am not the official maintainer for in much less controversial
circumstances than this. The offence was not writing a detailed enough
commit message to conform to that repository's guidelines. FWIW, I
believe the maintainer was right to object.

Cheers,

Richard

Re: proposal to move cogl, clutter and related recipes from oe-core to dedicated meta-clutter layer

[Otavio Salvador]

On Fri, May 10, 2013 at 7:18 PM, Richard Purdie
<richard.purdie@linuxfoundation.org> wrote:
> On Fri, 2013-05-10 at 17:22 -0300, Otavio Salvador wrote:
>> On Fri, May 10, 2013 at 2:19 PM, Richard Purdie
>> <richard.purdie@linuxfoundation.org> wrote:
>> > On Fri, 2013-05-10 at 17:39 +0100, Tomas Frydrych wrote:
>> >> On 10/05/13 12:32, Richard Purdie wrote:
>> >> > On Fri, 2013-05-10 at 11:56 +0100, Tomas Frydrych wrote:
>> >> >> On 10/05/13 10:05, Richard Purdie wrote:
>> >> > The closely track upstream is the key part and I think the core can do
>> >> > this, apart from the ~six week stabilisation window.
>> >>
>> >> If you mean you are prepared to do frequent point releases to keep up
>> >> with clutter, that could work. But if you mean that those interested can
>> >> closely track oe-core master, then that is not that useful, there are
>> >> too many other changes happening at the same time. A small single
>> >> purpose layer means updates (and breakages) can be very contained.
>> >
>> > Point releases of what? I don't mind clutter in master changing quite a
>> > lot up to our stabilisation point. Once we've released, I don't mind
>> > someone else picking up a danny-clutter branch or something like that
>> > which is backporting specific changes.
>>
>> So we're going to have <branch>-<pet package> branches?
>
> There may be a time and a place for such things and its not different to
> the situation you'll end up in with a meta-clutter repository where
> you'll likely end up with danny-1.12 and danny-1.14 branches, or
> multiple sets of recipes in a single danny branch, thereby changing the
> definition of "stable" from that used in the core which will be
> confusing in itself.

I don't think we need separated branches in a clutter specific layer,
we just need to keep versions around so custom BSPs can stick to a
specific one. Putting multiple branches on OE-Core/Poky confuse all
users and also blocks on you for merges, fixes and like. It just does
not scale.

>>  I think this does not scale and I am at Tomas side. The layer seems
>> the right thing to do so it does not need to be done by branch and
>> avoid confusing users.
>
> You're swapping one set of problems for another set the way I see it.

No. We're making use of modular design and taking profit of it.

>> >> > My argument for this is that I really do want to stress out the graphics
>> >> > stack we have, clutter provides a good way to test some of those
>> >> > components, particularly some of the more unusual parts. Currently its
>> >> > mostly build test however we do have plans to make that runtime too. I
>> >> > do think that clutters unusual usage of the stack makes it particular
>> >> > useful in this role. I'd appreciate help from anyone who can help make
>> >> > this all work.
>> >>
>> >> I am all for this, but does using clutter for automated tests require
>> >> for the clutter packages to be in oe-core? The only thing you can stress
>> >> test in oe-core using clutter is mesa, which is only applicable to the
>> >> i915/i965 chip sets. I think it would be useful if any such tests could
>> >> be applied to other graphics stacks provided by BSPs; I think all of the
>> >> BSPs would benefit from this.
>> >
>> > We'd be able to test mesa and the software fallbacks under qemu which
>> > would be a start. If we can get those working with a decent framework
>> > for the tests, I'm hoping wider BSP testing would follow. I don't have
>> > unlimited resources available but I can try and get the software
>> > infrastructure to the point where doing the testing could be picked up
>> > by someone else relatively easily.
>>
>> You could do just the same but adding meta-clutter to the AB setup for
>> testing. One thing does not conflict with the another.
>
> I've always been extremely clear that I believe the core needs to stand
> on its own, including being testable. I appreciate some people disagree
> with that however I believe that without this, the quality in the core
> would drop substantially. Even maintaining the testability of what we
> have now is actually an incredible challenge. Anyone who thinks
> otherwise should spend a week triaging the autobuilder failures.

And I fully agree but this does not mean we should have core growing.
We can have core and also a small set of layers which are added to
some tests, this also helps to uncover layer related issues and is
also import to test.

Nobody thinks it is easy but being hard doesn't mean it is right.
Yocto community is growing and we will always have good and bad
layers. The bad ones shouldn't be part of the test base, just it.

> Adding mode dimensions to a problem we already struggle with seems like
> a bad idea to me.

This sounds like you're against the modular design and it is no sense
for me. These dimensions also need test and if it cannot be done, we
have a design flaw.

>> > It is a big help in trying to achieve this if we don't have many
>> > different layers involved as that would complicate the problem, even
>> > coordinating layer releases between different maintainers is tricky
>> > right now and trying to develop something like this over layer
>> > boundaries sounds like adding to the complexity to the point I'd rather
>> > just scrap the idea :(.
>>
>> Well if layers make life harder it invalidates one of points of Yocto
>> which I always liked and depends on heavily - modular design.
>
> The layer model has its risks as well as its advantages. The risk is
> that we end up too fragmented, with too many maintainers who don't
> communicate and nothing ever works together.
>
> Even as the model stands today, its showing signs of strain. I posted
> patches near enough three weeks ago for meta-raspberrypi, no response. I
> know there are reasons why but they're not visible. The meta-fsl layers
> appear to spend more of their time not building than building and
> getting people to fix those in a timely fashion can sometimes be
> challenging, particularly with the split between arm and ppc and some of
> the transitions going on there etc.

It is all about community, as I said above we will always have the
good and bad layers. This does not mean the design is wrong neither
that layers is a bad thing but that you should choose carefully the
ones to depends on.

I can only talk by myself so I think the meta-fsl-arm is in a very
good shape and I've been quite responsive to fix issues. We had
problems of course and we tried very hard to fix them. I am sure you
know it.

> Right now, I think there are enough splits causing enough
> interoperability issues to be going on with in the layers and no, to be
> quite honest I don't think more layers is the right answer right now.

I disagree. More specialized layers allow for community grow around
them and centering it around core seems the wrong direction for me.

> I do continue to believe in the layer model but it is still evolving and
> I'd really like us to focus on the current issues.

Sure and I agree but bear on mind people issues are not always the
same. Some people are involved in customer projects, other busy with
his/her pet project, with a deadline in horizon or prefer a more up to
date clutter package set. As result these people focus on the issues
with high priority for them and it may not be the issues you think it
has higher priority.

It seems, checking commit logs, clutter have not been touch since last
year so as I said previously it was not being of use in regular test.
People seem to have forked it over and over again and I think
community will gain a lot of a central layer to share all this work.

>> I am sure we all want a solid release so I am also sure Tomas wouldn't
>> put experimental version of clutter near of release.
>
> From what I read of Tomas' emails, I actually think his plans differ
> from that as he finds that aspect of OE-Core frustrating.

He wants a more up to date clutter but he also wants people using his
recipes and contributing to it so he won't do crazy things.

He can add the new release and set D_P as -1, for example. This hurts
no one and allow for easy testing and sharing of work.

>>  We are eating our
>> own dog food so we are all interested in make it work, not the
>> opposed. In this case I think it can be well coordinated.
>>
>> One possible thing is you could require AB used layers to be at
>> git.yoctoproject.org so in case of broken recipes (bbappend or so) you
>> could  rename it.
>
> I don't touch layers I don't maintain. If I did that, I would get hung,
> drawn and quartered. Please at least think through things like this
> before saying them :(. Can you imagine what you'd say to me if I pushed
> commits to meta-fsl-arm?

For version upgrades, for example, I think it would be acceptable
(rename bbappends and so on) and you could do it for non active layers
in case it starts to cause problems, not as a rule but the exception.

> This isn't even hypothetical, I have already been "told off" for pushing
> to repos I am not the official maintainer for in much less controversial
> circumstances than this. The offence was not writing a detailed enough
> commit message to conform to that repository's guidelines. FWIW, I
> believe the maintainer was right to object.

I fully agree. However if the layer is not being maintained it might
justify a fix for a build failure for example.

--
Otavio Salvador                             O.S. Systems
E-mail: otavio@ossystems.com.br  http://www.ossystems.com.br
Mobile: +55 53 9981-7854              http://projetos.ossystems.com.br

Re: proposal to move cogl, clutter and related recipes from oe-core to dedicated meta-clutter layer

[Richard Purdie]

On Sat, 2013-05-11 at 17:39 -0300, Otavio Salvador wrote:
> On Fri, May 10, 2013 at 7:18 PM, Richard Purdie
> > Adding mode dimensions to a problem we already struggle with seems like
> > a bad idea to me.
> 
> This sounds like you're against the modular design and it is no sense
> for me. These dimensions also need test and if it cannot be done, we
> have a design flaw.

You think I'm against modular design? Seriously? Keep in mind I'm the
person who played a big part in helping transition to the layers model
in the first place instead of the monolithic OE classic, split up the
fetchers in bitbake, split bitbake into client/server/uis and so on.

My instinct is suggesting the timing is wrong for this particular change
(it may be right for Tomas, I don't think its right for OE-Core). I've
said what I plan to say on this now though.

I will however raise the point that a lot of people currently have a
tendency to "hide behind" me in discussions. They see me reply and
assume that since I've said something, that is it and they can keep
quiet. I want to be clear this isn't going to work as on this and some
other topics, it looks like I'm a lone voice. If people do believe in a
particular issue, they do need to stand up and add weight to an argument
themselves.

Cheers,

Richard

Re: proposal to move cogl, clutter and related recipes from oe-core to dedicated meta-clutter layer

[Philip Balister]

On 05/11/2013 05:49 PM, Richard Purdie wrote:
> 
> I will however raise the point that a lot of people currently have a
> tendency to "hide behind" me in discussions. They see me reply and
> assume that since I've said something, that is it and they can keep
> quiet. I want to be clear this isn't going to work as on this and some
> other topics, it looks like I'm a lone voice. If people do believe in a
> particular issue, they do need to stand up and add weight to an argument
> themselves.

I'd like to emphasize this point. Sometimes, Richard seems to be all
alone defending decisions that are made. If he is supporting your
position, support him publicly. Otherwise, there is a perception in the
community that Richard is acting alone.

Also, getting a variety of viewpoints out really helps technical
discussion. Sometimes it is the little details in individual use cases
that become really important.

Philip

Re: proposal to move cogl, clutter and related recipes from oe-core to dedicated meta-clutter layer

[Tomas Frydrych]

Hi Richard,

On 10/05/13 23:18, Richard Purdie wrote:
>> I am sure we all want a solid release so I am also sure Tomas wouldn't
>> put experimental version of clutter near of release.
> 
>>From what I read of Tomas' emails, I actually think his plans differ
> from that as he finds that aspect of OE-Core frustrating.

I am no sure what 'aspect' of OE-Core you mean. If you mean the
six-monthly releases, then you are reading my emails all wrong -- I am a
firm believer.

Regarding experimental releases -- I am not interested in them much
myself, but with meta-clutter, we could, of course, have negative
priority recipes for any interim developer snapshots, if that is what
people need. But what I am looking for most are uptodate recipes for
*stable* releases of clutter more or less as they happen.


> I don't touch layers I don't maintain. If I did that, I would get hung,
> drawn and quartered. Please at least think through things like this
> before saying them :(. Can you imagine what you'd say to me if I pushed
> commits to meta-fsl-arm? 

Just so we are clear, I am not trying to wrench clutter out of your
control. For all I care, set up meta-clutter on g.yp.o, and put the
current official maintainer of the clutter packages in oe-core in
charge, but I don't think we can get a usable clutter support without a
dedicated layer in the long run.

Tomas

-- 
http://sleepfive.com