Distro policy ?

Distro policy ?

[Wolfgang Denk]

HI,

we're trying to build our own distribution based on Yocto 1.0.  We
would like to push all modifications / extensions and our distro stuff
upstream into the mainline repository, but we are not sure what is the
best way to do this:

So far we have put our distro-related changes into a separate layer
but it seems that it was not so good idea: it looks like bitbake
can't include files from one layer to another and we have to copy
includes even if we don't want to change them.

Putting our changes back into the main layer would be pretty trivial,
but it appears that current tendency is to move all
distribution-specific things to their respective layers (like the
poky distro configuration itself was moved to the meta-yocto layer).

So, from the policy side I think we should go with our own layer.
On the technical side I'm disappointed with this 'include' problem: I
find it rather impractical to copy things we don't want to change and
just want to use from one layer to another and then having to keep
these in sync with mainline.

What is the recommended way to address this issue?

Best regards,

Wolfgang Denk

-- 
DENX Software Engineering GmbH,     MD: Wolfgang Denk & Detlev Zundel
HRB 165235 Munich, Office: Kirchenstr.5, D-82194 Groebenzell, Germany
Phone: (+49)-8142-66989-10 Fax: (+49)-8142-66989-80 Email: wd@denx.de
How come everyone's going so slow if it's called rush hour?

Re: Distro policy ?

[Saul Wold]

On 04/14/2011 12:44 PM, Wolfgang Denk wrote:
> HI,
>
> we're trying to build our own distribution based on Yocto 1.0.  We
> would like to push all modifications / extensions and our distro stuff
> upstream into the mainline repository, but we are not sure what is the
> best way to do this:
>
> So far we have put our distro-related changes into a separate layer
> but it seems that it was not so good idea: it looks like bitbake
> can't include files from one layer to another and we have to copy
> includes even if we don't want to change them.
>
Wolfgang,

I would like to get more details about which include files (.incs) you 
are needing to modify.  I know that there was some discussion at ELC 
this week about similar issues.  I also know that we have a "Layering 
Tools" item on the 1.1 feature list to address these issues.

> Putting our changes back into the main layer would be pretty trivial,
> but it appears that current tendency is to move all
> distribution-specific things to their respective layers (like the
> poky distro configuration itself was moved to the meta-yocto layer).
>
Correct, the core is supposed to be clean and distributions are supposed 
to be in layers.  Your layer can be linked to or hosted on the 
git.yoctoproject.org site if you would like. Pulling distro specific 
changes into core is not appropriate. If you have generic changes to 
core that would be appropriate to all distro, we would consider them.

> So, from the policy side I think we should go with our own layer.
> On the technical side I'm disappointed with this 'include' problem: I
> find it rather impractical to copy things we don't want to change and
> just want to use from one layer to another and then having to keep
> these in sync with mainline.
>
> What is the recommended way to address this issue?
>
For the short term, I think that you may need to deal with the copy 
issue, until we have tooling in place.

I know this is not the answer you wanted to hear right now, we know 
about the issue and are looking at it.

Sau!

> Best regards,
>
> Wolfgang Denk
>

Re: Distro policy ?

[Chris Larson]

On Thu, Apr 14, 2011 at 1:28 PM, Saul Wold <saul.wold@intel.com> wrote:
>> So, from the policy side I think we should go with our own layer.
>> On the technical side I'm disappointed with this 'include' problem: I
>> find it rather impractical to copy things we don't want to change and
>> just want to use from one layer to another and then having to keep
>> these in sync with mainline.

There's no reason you should have to copy unmodified files from one
layer to another, ever.  Configuration files are found via BBPATH when
they're referenced via a relative path, and file:// files via
FILESPATH are easily dealt with as well.  The only time you should
have to copy is if you have a need to modify them, particularly as
there's no "bbappend" that corresponds to recipe .inc files.  Is there
a particular case you're thinking of here?
-- 
Christopher Larson
clarson at kergoth dot com
Founder - BitBake, OpenEmbedded, OpenZaurus
Maintainer - Tslib
Senior Software Engineer, Mentor Graphics

Re: Distro policy ?

[Martin Jansa]

[-- Attachment #1: Type: text/plain, Size: 1219 bytes --]

On Thu, Apr 14, 2011 at 01:57:16PM -0700, Chris Larson wrote:
> On Thu, Apr 14, 2011 at 1:28 PM, Saul Wold <saul.wold@intel.com> wrote:
> >> So, from the policy side I think we should go with our own layer.
> >> On the technical side I'm disappointed with this 'include' problem: I
> >> find it rather impractical to copy things we don't want to change and
> >> just want to use from one layer to another and then having to keep
> >> these in sync with mainline.
> 
> There's no reason you should have to copy unmodified files from one
> layer to another, ever.  Configuration files are found via BBPATH when
> they're referenced via a relative path, and file:// files via
> FILESPATH are easily dealt with as well.  The only time you should
> have to copy is if you have a need to modify them, particularly as
> there's no "bbappend" that corresponds to recipe .inc files.  Is there
> a particular case you're thinking of here?

I guess he was talking about
require/include some.inc
used in recipe

Then I had to copy some.inc to same folder there my recipe is. Even when
it's used in same path relative to layer top dir

Regards,

-- 
Martin 'JaMa' Jansa     jabber: Martin.Jansa@gmail.com

[-- Attachment #2: Type: application/pgp-signature, Size: 205 bytes --]

Re: Distro policy ?

[Chris Larson]

On Thu, Apr 14, 2011 at 2:12 PM, Martin Jansa <martin.jansa@gmail.com> wrote:
> On Thu, Apr 14, 2011 at 01:57:16PM -0700, Chris Larson wrote:
>> On Thu, Apr 14, 2011 at 1:28 PM, Saul Wold <saul.wold@intel.com> wrote:
>> >> So, from the policy side I think we should go with our own layer.
>> >> On the technical side I'm disappointed with this 'include' problem: I
>> >> find it rather impractical to copy things we don't want to change and
>> >> just want to use from one layer to another and then having to keep
>> >> these in sync with mainline.
>>
>> There's no reason you should have to copy unmodified files from one
>> layer to another, ever.  Configuration files are found via BBPATH when
>> they're referenced via a relative path, and file:// files via
>> FILESPATH are easily dealt with as well.  The only time you should
>> have to copy is if you have a need to modify them, particularly as
>> there's no "bbappend" that corresponds to recipe .inc files.  Is there
>> a particular case you're thinking of here?
>
> I guess he was talking about
> require/include some.inc
> used in recipe
>
> Then I had to copy some.inc to same folder there my recipe is. Even when
> it's used in same path relative to layer top dir

Change the recipe to use require/include recipes/foo/bar.inc, relative
to the top of the entry in BBPATH.
-- 
Christopher Larson
clarson at kergoth dot com
Founder - BitBake, OpenEmbedded, OpenZaurus
Maintainer - Tslib
Senior Software Engineer, Mentor Graphics

Re: Distro policy ?

[Joshua Lock]

On Thu, 2011-04-14 at 21:44 +0200, Wolfgang Denk wrote:
> HI,
> 
> we're trying to build our own distribution based on Yocto 1.0.  We
> would like to push all modifications / extensions and our distro stuff
> upstream into the mainline repository, but we are not sure what is the
> best way to do this:
> 
> So far we have put our distro-related changes into a separate layer
> but it seems that it was not so good idea: it looks like bitbake
> can't include files from one layer to another and we have to copy
> includes even if we don't want to change them.

You can you just have to use a path from the top of the BBPATH entry for
the layer you're using, so in our meta-demo layer I've used entries
like:

require recipes-graphics/clutter/clutter.inc

to use an inc file from meta in Poky.

Cheers,
Joshua
-- 
Joshua Lock
        Yocto Build System Monkey
        Intel Open Source Technology Centre

Re: Distro policy ?

[Matt Madison]

How about when it's the other way round -- you need to modify just the .inc
for your layer, and not the recipes that require/include them?  Should all
recipes in the core be changed to use the path name syntax to enable that?

Thanks,
-Matt


On 4/14/11 14:43 , "Joshua Lock" <josh@linux.intel.com> wrote:

> On Thu, 2011-04-14 at 21:44 +0200, Wolfgang Denk wrote:
>> HI,
>> 
>> we're trying to build our own distribution based on Yocto 1.0.  We
>> would like to push all modifications / extensions and our distro stuff
>> upstream into the mainline repository, but we are not sure what is the
>> best way to do this:
>> 
>> So far we have put our distro-related changes into a separate layer
>> but it seems that it was not so good idea: it looks like bitbake
>> can't include files from one layer to another and we have to copy
>> includes even if we don't want to change them.
> 
> You can you just have to use a path from the top of the BBPATH entry for
> the layer you're using, so in our meta-demo layer I've used entries
> like:
> 
> require recipes-graphics/clutter/clutter.inc
> 
> to use an inc file from meta in Poky.
> 
> Cheers,
> Joshua

Re: Distro policy ?

[Richard Purdie]

On Thu, 2011-04-14 at 16:00 -0700, Matt Madison wrote:
> How about when it's the other way round -- you need to modify just the .inc
> for your layer, and not the recipes that require/include them?  Should all
> recipes in the core be changed to use the path name syntax to enable that?

Yes, I think this sounds like its causing a few people problems and we
should be using that syntax by default.

Cheers,

Richard

Re: Distro policy ?

[Ilya Yanok]

Hi Chris,

Chris Larson <clarson@...> writes:
> > I guess he was talking about
> > require/include some.inc
> > used in recipe
> >
> > Then I had to copy some.inc to same folder there my recipe is. Even when
> > it's used in same path relative to layer top dir
> 
> Change the recipe to use require/include recipes/foo/bar.inc, relative
> to the top of the entry in BBPATH.

Yes, that's exactly the situation we have. Thanks for the tip!

Regards, Ilya.

Re: Distro policy ?

[Ilya Yanok]

Hi Joshua,

Joshua Lock <josh@...> writes:
> > So far we have put our distro-related changes into a separate layer
> > but it seems that it was not so good idea: it looks like bitbake
> > can't include files from one layer to another and we have to copy
> > includes even if we don't want to change them.
> 
> You can you just have to use a path from the top of the BBPATH entry for

Thanks for the advice, this really does help.

Regards, Ilya.

Re: Distro policy ?

[Ilya Yanok]

Hi Saul,

Saul Wold <saul.wold@...> writes:
> I would like to get more details about which include files (.incs) you 
> are needing to modify.  I know that there was some discussion at ELC 
> this week about similar issues.  I also know that we have a "Layering 
> Tools" item on the 1.1 feature list to address these issues.

We had one problem with the recipes that need to include files from the main
layer. Thanks to the guy now I know that I just need to use the full path to
solve this.
Another problem was a strange one. It was about poky's distro includes. When I
started working on our distro I've just copied conf/distro/poky.conf to our new
layer with new name and changed it a bit. This worked but I got a bunch of
warnings about multiply providers. Then I've copied the conf/distro/include
directory into our layer and all these warnings gone. Unfortunately I can't
reproduce this behavior any more.

So, it looks like we have solved the problem with includes and now we are fine
with our own layer.

Thanks for your help, guys!

Regards, Ilya.