wanting clarification on overrides, _append, += and combinations of those

wanting clarification on overrides, _append, += and combinations of those

[Robert P. J. Day]

  preparing my own write-up on how bitbake processes combinations of
overrides, _append and += and ran across an example in oe-core that is
unnecessarily whitespacey, just wanted to make sure i was
understanding this properly.

  in the current bitbake user manual, there's this example of mixing
overrides, _append and +=:

  OVERRIDES = "foo"
  A_foo_append = "X"
  A_foo_append += "Y"

  This behaves as per the first case above, but the value of A would
  be "X Y" instead of just "X".

  that, of course, makes sense as the "+=" operator automatically adds
a leading space to what is being appended.  so for fun, i scanned all
of oe-core looking for examples of "_append.*+=" for classroom demo
purposes, and the *only* examples i found were in
meta/recipes-devtools/python/python-smartpm_1.4.1.bb:

RDEPENDS_python-smartpm_append = " python-smartpm-backend-rpm python-codecs python-textutils python-xml"
RDEPENDS_python-smartpm_append += " python-fcntl python-pickle python-crypt python-compression python-shell"
RDEPENDS_python-smartpm_append += " python-resource python-netclient python-threading python-unixadmin"

  i can see that that first line *does* need an explicit leading space
in the value being assigned, but the next two "+=" lines don't, do
they?  it won't hurt, of course, you'll just *two* spaces inserted
instead of one.  but i just want to make sure i understand what's
happening there since it has the potential to be confusing for someone
trying to follow along.

rday

-- 

========================================================================
Robert P. J. Day                                 Ottawa, Ontario, CANADA
                        http://crashcourse.ca

Twitter:                                       http://twitter.com/rpjday
LinkedIn:                               http://ca.linkedin.com/in/rpjday
========================================================================

Re: wanting clarification on overrides, _append, += and combinations of those

[Robert P. J. Day]

  one more question, if i might ...

On Sat, 24 Nov 2012, Robert P. J. Day wrote:

> RDEPENDS_python-smartpm_append = " python-smartpm-backend-rpm python-codecs python-textutils python-xml"
> RDEPENDS_python-smartpm_append += " python-fcntl python-pickle python-crypt python-compression python-shell"
> RDEPENDS_python-smartpm_append += " python-resource python-netclient python-threading python-unixadmin"

  is there a reason the above couldn't have been written as something
like:

RDEPENDS_python-smartpm += "python-fcntl python-pickle python-crypt python-compression python-shell"
RDEPENDS_python-smartpm += "python-resource python-netclient python-threading python-unixadmin"

  as i understand it, the purpose of "_append" is to give the
developer control over being able to append something at the very end
of the processing -- in other words, it's used when ordering is
important.

  but is there any ordering issue when simply creating the value of
RDEPENDS?  does it matter how the items are ordered there?  if not,
then using "_append" seems like overkill.

rday

-- 

========================================================================
Robert P. J. Day                                 Ottawa, Ontario, CANADA
                        http://crashcourse.ca

Twitter:                                       http://twitter.com/rpjday
LinkedIn:                               http://ca.linkedin.com/in/rpjday
========================================================================

Re: wanting clarification on overrides, _append, += and combinations of those

[Richard Purdie]

On Sat, 2012-11-24 at 09:42 -0500, Robert P. J. Day wrote:
> one more question, if i might ...
> 
> On Sat, 24 Nov 2012, Robert P. J. Day wrote:
> 
> > RDEPENDS_python-smartpm_append = " python-smartpm-backend-rpm python-codecs python-textutils python-xml"
> > RDEPENDS_python-smartpm_append += " python-fcntl python-pickle python-crypt python-compression python-shell"
> > RDEPENDS_python-smartpm_append += " python-resource python-netclient python-threading python-unixadmin"
> 
>   is there a reason the above couldn't have been written as something
> like:
> 
> RDEPENDS_python-smartpm += "python-fcntl python-pickle python-crypt python-compression python-shell"
> RDEPENDS_python-smartpm += "python-resource python-netclient python-threading python-unixadmin"
> 
>   as i understand it, the purpose of "_append" is to give the
> developer control over being able to append something at the very end
> of the processing -- in other words, it's used when ordering is
> important.
> 
>   but is there any ordering issue when simply creating the value of
> RDEPENDS?  does it matter how the items are ordered there?  if not,
> then using "_append" seems like overkill.

In this case I'd agree, it does seem a little over the top.

Cheers,

Richard

Re: wanting clarification on overrides, _append, += and combinations of those

[Richard Purdie]

On Sat, 2012-11-24 at 09:23 -0500, Robert P. J. Day wrote:
> preparing my own write-up on how bitbake processes combinations of
> overrides, _append and += and ran across an example in oe-core that is
> unnecessarily whitespacey, just wanted to make sure i was
> understanding this properly.
> 
>   in the current bitbake user manual, there's this example of mixing
> overrides, _append and +=:
> 
>   OVERRIDES = "foo"
>   A_foo_append = "X"
>   A_foo_append += "Y"
> 
>   This behaves as per the first case above, but the value of A would
>   be "X Y" instead of just "X".
> 
>   that, of course, makes sense as the "+=" operator automatically adds
> a leading space to what is being appended.  so for fun, i scanned all
> of oe-core looking for examples of "_append.*+=" for classroom demo
> purposes, and the *only* examples i found were in
> meta/recipes-devtools/python/python-smartpm_1.4.1.bb:
> 
> RDEPENDS_python-smartpm_append = " python-smartpm-backend-rpm python-codecs python-textutils python-xml"
> RDEPENDS_python-smartpm_append += " python-fcntl python-pickle python-crypt python-compression python-shell"
> RDEPENDS_python-smartpm_append += " python-resource python-netclient python-threading python-unixadmin"
> 
>   i can see that that first line *does* need an explicit leading space
> in the value being assigned, but the next two "+=" lines don't, do
> they?

No, they don't.

Cheers,

Richard

Re: wanting clarification on overrides, _append, += and combinations of those

[Paul Eggleton]

On Saturday 24 November 2012 09:23:20 Robert P. J. Day wrote:
>   preparing my own write-up on how bitbake processes combinations of
> overrides, _append and += and ran across an example in oe-core that is
> unnecessarily whitespacey, just wanted to make sure i was
> understanding this properly.
> 
>   in the current bitbake user manual, there's this example of mixing
> overrides, _append and +=:
> 
>   OVERRIDES = "foo"
>   A_foo_append = "X"
>   A_foo_append += "Y"
> 
>   This behaves as per the first case above, but the value of A would
>   be "X Y" instead of just "X".
> 
>   that, of course, makes sense as the "+=" operator automatically adds
> a leading space to what is being appended.  so for fun, i scanned all
> of oe-core looking for examples of "_append.*+=" for classroom demo
> purposes, and the *only* examples i found were in
> meta/recipes-devtools/python/python-smartpm_1.4.1.bb:
> 
> RDEPENDS_python-smartpm_append = " python-smartpm-backend-rpm python-codecs
> python-textutils python-xml" RDEPENDS_python-smartpm_append += "
> python-fcntl python-pickle python-crypt python-compression python-shell"
> RDEPENDS_python-smartpm_append += " python-resource python-netclient
> python-threading python-unixadmin"
> 
>   i can see that that first line *does* need an explicit leading space
> in the value being assigned, but the next two "+=" lines don't, do
> they?  it won't hurt, of course, you'll just *two* spaces inserted
> instead of one.  but i just want to make sure i understand what's
> happening there since it has the potential to be confusing for someone
> trying to follow along.

I would strongly discourage the use of += in conjunction with _append because 
the intent is unclear - I think we've discussed this before; this is just a 
case where it slipped through. Ideally this would just be done as a += split 
onto multiple lines and I will make sure it gets changed to that if someone 
else doesn't get to it first.

Cheers,
Paul

-- 

Paul Eggleton
Intel Open Source Technology Centre

Re: wanting clarification on overrides, _append, += and combinations of those

[Robert P. J. Day]

On Mon, 26 Nov 2012, Paul Eggleton wrote:

> On Saturday 24 November 2012 09:23:20 Robert P. J. Day wrote:
> >   preparing my own write-up on how bitbake processes combinations of
> > overrides, _append and += and ran across an example in oe-core that is
> > unnecessarily whitespacey, just wanted to make sure i was
> > understanding this properly.
> >
> >   in the current bitbake user manual, there's this example of mixing
> > overrides, _append and +=:
> >
> >   OVERRIDES = "foo"
> >   A_foo_append = "X"
> >   A_foo_append += "Y"
> >
> >   This behaves as per the first case above, but the value of A would
> >   be "X Y" instead of just "X".
> >
> >   that, of course, makes sense as the "+=" operator automatically adds
> > a leading space to what is being appended.  so for fun, i scanned all
> > of oe-core looking for examples of "_append.*+=" for classroom demo
> > purposes, and the *only* examples i found were in
> > meta/recipes-devtools/python/python-smartpm_1.4.1.bb:
> >
> > RDEPENDS_python-smartpm_append = " python-smartpm-backend-rpm python-codecs
> > python-textutils python-xml" RDEPENDS_python-smartpm_append += "
> > python-fcntl python-pickle python-crypt python-compression python-shell"
> > RDEPENDS_python-smartpm_append += " python-resource python-netclient
> > python-threading python-unixadmin"
> >
> >   i can see that that first line *does* need an explicit leading space
> > in the value being assigned, but the next two "+=" lines don't, do
> > they?  it won't hurt, of course, you'll just *two* spaces inserted
> > instead of one.  but i just want to make sure i understand what's
> > happening there since it has the potential to be confusing for someone
> > trying to follow along.
>
> I would strongly discourage the use of += in conjunction with _append because
> the intent is unclear - I think we've discussed this before ...

  yes, i've whined about this sort of thing before :-P, not because it
makes any *functional* difference but because (and i know i've said
this before) when i teach OE/yocto, i like to use examples right out
of the source code, and oddities like the above sometimes cause
students to wonder if perhaps they've misunderstood something when
they see something like that.  it's a perfect example where telling
someone to RTFS can actually be counter-productive.

> Ideally this would just be done as a += split onto multiple lines
> and I will make sure it gets changed to that if someone else doesn't
> get to it first.

  help yourself.  on a related note, though, note that this feature is
described in the bitbake user manual which, while it's not clear where
to find it online, is in the bitbake git repo where one can build it.
so two questions.

  first, is the *individual* bitbake user manual considered part of
the official yocto documentation?  should it be listed here?

  https://www.yoctoproject.org/documentation

if not, where should one go online to read it?  is it regularly
rebuilt from the bitbake/doc directory?  and published where?  there
are enough references to the bitbake user manual that people should
know where to find it easily.

  also, WRT the above, it's the bitbake user manual itself that shows
the reader how to do this:

OVERRIDES = "foo"
A_foo_append = "X"
A_foo_append += "Y"

without perhaps explaining that that's a really weird thing to do and
should be discouraged.  so there's a definite logical disconnect
between things that are deprecated while still being explained in the
user manual.

  just my $0.02.  another doc note coming shortly ...

rday

-- 

========================================================================
Robert P. J. Day                                 Ottawa, Ontario, CANADA
                        http://crashcourse.ca

Twitter:                                       http://twitter.com/rpjday
LinkedIn:                               http://ca.linkedin.com/in/rpjday
========================================================================

Re: wanting clarification on overrides, _append, += and combinations of those

[Robert P. J. Day]

On Mon, 26 Nov 2012, Paul Eggleton wrote:

> On Saturday 24 November 2012 09:23:20 Robert P. J. Day wrote:

> > RDEPENDS_python-smartpm_append = " python-smartpm-backend-rpm python-codecs
> > python-textutils python-xml" RDEPENDS_python-smartpm_append += "
> > python-fcntl python-pickle python-crypt python-compression python-shell"
> > RDEPENDS_python-smartpm_append += " python-resource python-netclient
> > python-threading python-unixadmin"
>
> I would strongly discourage the use of += in conjunction with
> _append because the intent is unclear - I think we've discussed this
> before; this is just a case where it slipped through. Ideally this
> would just be done as a += split onto multiple lines and I will make
> sure it gets changed to that if someone else doesn't get to it
> first.

  two more bits of pedantry that i think would improve the
code/documentation, related to the above.

  first, as i understand it, the "_append" feature is used to do
appending at the *end* of the, what, parsing?  what do you call that
step, and how would you describe what happens when you use "_append"?
because it seems that, while it's a useful feature when ordering *is*
important, it's overkill when ordering doesn't matter.

  consider this snippet from linux-yocto_3.2.bb:

KERNEL_FEATURES_append=" features/netfilter"
KERNEL_FEATURES_append=" features/taskstats"
KERNEL_FEATURES_append_qemux86=" cfg/sound"
KERNEL_FEATURES_append_qemux86-64=" cfg/sound"

since the above is defining kernel features, is there anything
important about the ordering?  unless there's something tricky going
on, couldn't one just have coded:

KERNEL_FEATURES += "features/netfilter"
KERNEL_FEATURES += "features/taskstats"
KERNEL_FEATURES_qemux86 += "cfg/sound"
KERNEL_FEATURES_qemux86-64 +="cfg/sound"

  that second form is clearly simpler, as long as it represents
identical functionality.  in short, don't use "_append" unless it's
actually necessary.  which brings us to my second suggestion.

  use actual OE/yocto code to explain features.  from the bitbake user
manual, remember this snippet explaining how to use overrides and
appending:

===== start =====

OVERRIDES = "foo"
A_foo_append = "X"
In this case, X is unconditionally appended to the variable A_foo.
Since foo is an override, A_foo would then replace A.

OVERRIDES = "foo"
A = "X"
A_append_foo = "Y"
In this case, only when foo is in OVERRIDES, Y is appended to the
variable A so the value of A would become XY (NB: no spaces are
appended).

OVERRIDES = "foo"
A_foo_append = "X"
A_foo_append += "Y"

===== end =====

  but the above is not all that informative since the reader has no
clue what "A" or "foo" might represent.  so use excerpts from recipes
themselves that are immediately obvious; in fact, the above kernel
features can be used for that:

KERNEL_FEATURES += "features/netfilter"
KERNEL_FEATURES += "features/taskstats"
KERNEL_FEATURES_qemux86 += "cfg/sound"
KERNEL_FEATURES_qemux86-64 +="cfg/sound"

that at least makes it spectacularly clear that, given a particular
architecture, additional features will be selected for the kernel.
simply using "A" and "foo" and "X" and "Y" is fairly meaningless.

  i may shortly have another suggestion or two that have worked for me
in explaining stuff to students.

rday

-- 

========================================================================
Robert P. J. Day                                 Ottawa, Ontario, CANADA
                        http://crashcourse.ca

Twitter:                                       http://twitter.com/rpjday
LinkedIn:                               http://ca.linkedin.com/in/rpjday
========================================================================