RFC: strip "-native" from $S for native packages automatically (bug 1856)

RFC: strip "-native" from $S for native packages automatically (bug 1856)

[Rolf Leggewie]

Hi,

I elicit your comments on bug 1856, especially comment3.

The bug is about stripping "-native" automatically from the S variable
for native packages since it will hardly ever occur that a tar ball will
unpack to $PN-native-$PV/  This would save us resetting $S manually for
native packages all the time.  Koen suggested the fix to go into
native.bbclass. When I revisited the bug today, I thought this must be
as easy to fix as simply adding

  export S=`echo $S|sed s/\-native\-/\-/`

to native.bbclass which is what I propose.  This might still have some
issues that I so far overlooked such as the need to further escape
special characters or something else, but the general idea should be
clear.  Please be aware that I have not yet tested this.  Any comments
as to why this should or should not work?

If there is no opposition, I'll make some simple tests to verify correct
functioning of the code before finally committing.

Regards

Rolf


PS: http://bugs.openembedded.org/show_bug.cgi?id=1856#c3

Re: RFC: strip "-native" from $S for native packages automatically (bug 1856)

[Michael 'Mickey' Lauer]

On Friday 25 January 2008 12:33:39 Rolf Leggewie wrote:
> Hi,
>
> I elicit your comments on bug 1856, especially comment3.
>
> The bug is about stripping "-native" automatically from the S variable
> for native packages since it will hardly ever occur that a tar ball will
> unpack to $PN-native-$PV/  This would save us resetting $S manually for
> native packages all the time.  Koen suggested the fix to go into
> native.bbclass. When I revisited the bug today, I thought this must be
> as easy to fix as simply adding
>
>   export S=`echo $S|sed s/\-native\-/\-/`

That looks like an infinite recursion to me.
S:= ... may work, but I don't like launching a shell for that.

If we really want to have that (I'm -0 on that), I would rather propose

S = ${@"%s/%s" % ( bb.data.getVar("WORKDIR", d, 1), bb.data.getVar("P", d, 1).replace( "-native", "" ) ) }

-- 
Dr. Michael 'Mickey' Lauer | IT-Freelancer | http://www.vanille-media.de

Re: RFC: strip "-native" from $S for native packages automatically (bug 1856)

[Richard Purdie]

On Sat, 2008-01-26 at 01:51 +0100, Michael 'Mickey' Lauer wrote:
> On Friday 25 January 2008 12:33:39 Rolf Leggewie wrote:
> > Hi,
> >
> > I elicit your comments on bug 1856, especially comment3.
> >
> > The bug is about stripping "-native" automatically from the S variable
> > for native packages since it will hardly ever occur that a tar ball will
> > unpack to $PN-native-$PV/  This would save us resetting $S manually for
> > native packages all the time.  Koen suggested the fix to go into
> > native.bbclass. When I revisited the bug today, I thought this must be
> > as easy to fix as simply adding
> >
> >   export S=`echo $S|sed s/\-native\-/\-/`
> 
> That looks like an infinite recursion to me.
> S:= ... may work, but I don't like launching a shell for that.

Agreed, it won't work and := is not appropriate either, I still maintain
that := should be used with caution and only when people really
understand what its implications are.

> If we really want to have that (I'm -0 on that), I would rather propose
> 
> S = ${@"%s/%s" % ( bb.data.getVar("WORKDIR", d, 1), bb.data.getVar("P", d, 1).replace( "-native", "" ) ) }

Would it be worth adding syntax to bitbake for this kind of operation,
something like:

S *= "${S}-native"

?

I don't know if many people had noticed but I've recently added a lot of
-sdk packages to poky and this would be useful for sdk.bbclass too. The
sdk packages are really useful but the .bb files are tedious, just like
-native.

I've been thinking about the -native/-sdk problem again and I'd like to
try and plan an extension to bitbake to help with this. The idea would
be to have a list of "extension" classes like sdk and native. During
parsing bitbake bitbake would first parse as usual but then it would
automatically "inherit native" and then "inherit sdk" for each .bb file.

The extension classes could be named in some variable
BB_PARSING_EXTENSION_CLASSES = "native sdk" or I wondered if we add a
new type of file (.bbextclass?). The variable approach would mean a
package could easily choose not to have this behaviour or to extend
it...

This does assume we can reduce native recipes to "inherit native" but
with suitable changes to native.bbclass that should be possible?

Cheers,

Richard

Re: RFC: strip "-native" from $S for native packages automatically (bug 1856)

[Paul Sokolovsky]

Hello Richard,

Saturday, January 26, 2008, 3:43:41 PM, you wrote:

> On Sat, 2008-01-26 at 01:51 +0100, Michael 'Mickey' Lauer wrote:
>> On Friday 25 January 2008 12:33:39 Rolf Leggewie wrote:
>> > Hi,
>> >
>> > I elicit your comments on bug 1856, especially comment3.
>> >
>> > The bug is about stripping "-native" automatically from the S variable
>> > for native packages since it will hardly ever occur that a tar ball will
>> > unpack to $PN-native-$PV/  This would save us resetting $S manually for
>> > native packages all the time.  Koen suggested the fix to go into
>> > native.bbclass. When I revisited the bug today, I thought this must be
>> > as easy to fix as simply adding
>> >
>> >   export S=`echo $S|sed s/\-native\-/\-/`
>> 
>> That looks like an infinite recursion to me.
>> S:= ... may work, but I don't like launching a shell for that.

> Agreed, it won't work and := is not appropriate either, I still maintain
> that := should be used with caution and only when people really
> understand what its implications are.

>> If we really want to have that (I'm -0 on that), I would rather propose
>> 
>> S = ${@"%s/%s" % ( bb.data.getVar("WORKDIR", d, 1), bb.data.getVar("P", d, 1).replace( "-native", "" ) ) }

> Would it be worth adding syntax to bitbake for this kind of operation,
> something like:

> S *= "${S}-native"

> ?

> I don't know if many people had noticed but I've recently added a lot of
> -sdk packages to poky and this would be useful for sdk.bbclass too. The
> sdk packages are really useful but the .bb files are tedious, just like
> -native.

> I've been thinking about the -native/-sdk problem again and I'd like to
> try and plan an extension to bitbake to help with this. The idea would
> be to have a list of "extension" classes like sdk and native. During
> parsing bitbake bitbake would first parse as usual but then it would
> automatically "inherit native" and then "inherit sdk" for each .bb file.

> The extension classes could be named in some variable
> BB_PARSING_EXTENSION_CLASSES = "native sdk" or I wondered if we add a
> new type of file (.bbextclass?). The variable approach would mean a
> package could easily choose not to have this behaviour or to extend
> it...

  Sounds like something complex and obscure ;-I. However, here's
another usecase for such scheme:

   I've been pondering about 2-stage kernel building, which would
allow to have kernel with builtin initramfs. For this, modules should
be built first, then initramfs image which includes them, and then
zImage which includes initramfs. Even if modules vs zImage building
will be reduced to "inherit kernel-modules" and "inherit
kernel-image", that would mean that such 2 extra .bb files are needed
for all kernel recipes. Your template/generator idea might handle this
too. (But at what price...?).

> This does assume we can reduce native recipes to "inherit native" but
> with suitable changes to native.bbclass that should be possible?

> Cheers,

> Richard



-- 
Best regards,
 Paul                            mailto:pmiscml@gmail.com

Re: RFC: strip "-native" from $S for native packages automatically (bug 1856)

[Richard Purdie]

On Sat, 2008-01-26 at 13:43 +0000, Richard Purdie wrote:
> On Sat, 2008-01-26 at 01:51 +0100, Michael 'Mickey' Lauer wrote:
> > If we really want to have that (I'm -0 on that), I would rather propose
> > 
> > S = ${@"%s/%s" % ( bb.data.getVar("WORKDIR", d, 1), bb.data.getVar("P", d, 1).replace( "-native", "" ) ) }
> 
> Would it be worth adding syntax to bitbake for this kind of operation,
> something like:
> 
> S *= "${S}-native"
> 
> ?

This syntax doesn't solve the original problem of course.

People have requested -= and I've been wondering if we should add that
as a search and remove operator. To match the existing language that
would have to work on space delimited lists though so its of no use to
this use case. We could pair this with a *= or operator which worked the
same way without the space delimiter like .= does...

S *= "-native"

?

maybe ~= would be a better pairing with -=?

Having syntax which allowed variable reuse like the original example
above could also be useful:

S = "foo"
S *= "${S}-native"
print bb.data.getVar("S", d, 1)
    foo-native

Does anyone have any other language enhancement requests for bitbake
while I'm thinking about it (or any thoughts on the above)?

Cheers,

Richard

Re: RFC: strip "-native" from $S for native packages automatically (bug 1856)

[Michael 'Mickey' Lauer]

On Sunday 27 January 2008 01:23:27 Richard Purdie wrote:
> On Sat, 2008-01-26 at 13:43 +0000, Richard Purdie wrote:
> > On Sat, 2008-01-26 at 01:51 +0100, Michael 'Mickey' Lauer wrote:
> > > If we really want to have that (I'm -0 on that), I would rather propose
> > >
> > > S = ${@"%s/%s" % ( bb.data.getVar("WORKDIR", d, 1), bb.data.getVar("P",
> > > d, 1).replace( "-native", "" ) ) }
> >
> > Would it be worth adding syntax to bitbake for this kind of operation,
> > something like:
> >
> > S *= "${S}-native"
> >
> > ?
>
> This syntax doesn't solve the original problem of course.
>
> People have requested -= and I've been wondering if we should add that
> as a search and remove operator. To match the existing language that
> would have to work on space delimited lists though so its of no use to
> this use case. We could pair this with a *= or operator which worked the
> same way without the space delimiter like .= does...

Aah, finally you're getting sane :D

>
> S *= "-native"
>
> ?
>
> maybe ~= would be a better pairing with -=?

Yes, I would it prefer that way.

We should be very careful with that though feature though, so it doesn't get 
overused.

Re: RFC: strip "-native" from $S for native packages automatically (bug 1856)

[Paul Sokolovsky]

Hello Richard,

Sunday, January 27, 2008, 2:23:27 AM, you wrote:

> On Sat, 2008-01-26 at 13:43 +0000, Richard Purdie wrote:
>> On Sat, 2008-01-26 at 01:51 +0100, Michael 'Mickey' Lauer wrote:
>> > If we really want to have that (I'm -0 on that), I would rather propose
>> > 
>> > S = ${@"%s/%s" % ( bb.data.getVar("WORKDIR", d, 1), bb.data.getVar("P", d, 1).replace( "-native", "" ) ) }
>> 
>> Would it be worth adding syntax to bitbake for this kind of operation,
>> something like:
>> 
>> S *= "${S}-native"
>> 
>> ?

> This syntax doesn't solve the original problem of course.

> People have requested -= and I've been wondering if we should add that
> as a search and remove operator. To match the existing language that
> would have to work on space delimited lists though so its of no use to
> this use case. We could pair this with a *= or operator which worked the
> same way without the space delimiter like .= does...

> S *= "-native"

> ?

> maybe ~= would be a better pairing with -=?

  I personally don't like this operator obfuscation, which becomes
just that once obscure symbol combinations are used or well-known
operators from other languages are reused with different semantics.
All that will lead to very that Perl's $@#%&* stuff.

> Having syntax which allowed variable reuse like the original example
> above could also be useful:

> S = "foo"
> S *= "${S}-native"
> print bb.data.getVar("S", d, 1)
>     foo-native

> Does anyone have any other language enhancement requests for bitbake
> while I'm thinking about it (or any thoughts on the above)?

  Yes, I have. Those 'bb.data.getVar("S", d, 1)' looks quite
unaesthetically and brings thought that Python expression support is a
quick hack. Instead, it should be something like:

d2.get("S")
d2.get("S", expand = 0)

  Note the default for expanded grounded in common use.

  Stuff above could be then:

d2.get("S").replace("-native", "")

> Cheers,

> Richard


-- 
Best regards,
 Paul                            mailto:pmiscml@gmail.com

Re: RFC: strip "-native" from $S for native packages automatically (bug 1856)

[pHilipp Zabel]

On Jan 27, 2008 1:23 AM, Richard Purdie <rpurdie@rpsys.net> wrote:
> On Sat, 2008-01-26 at 13:43 +0000, Richard Purdie wrote:
> > On Sat, 2008-01-26 at 01:51 +0100, Michael 'Mickey' Lauer wrote:
> > > If we really want to have that (I'm -0 on that), I would rather propose
> > >
> > > S = ${@"%s/%s" % ( bb.data.getVar("WORKDIR", d, 1), bb.data.getVar("P", d, 1).replace( "-native", "" ) ) }
> >
> > Would it be worth adding syntax to bitbake for this kind of operation,
> > something like:
> >
> > S *= "${S}-native"
> >
> > ?
>
> This syntax doesn't solve the original problem of course.
>
> People have requested -= and I've been wondering if we should add that
> as a search and remove operator. To match the existing language that
> would have to work on space delimited lists though so its of no use to
> this use case. We could pair this with a *= or operator which worked the
> same way without the space delimiter like .= does...
>
> S *= "-native"
>
> ?
>
> maybe ~= would be a better pairing with -=?
>
> Having syntax which allowed variable reuse like the original example
> above could also be useful:
>
> S = "foo"
> S *= "${S}-native"
> print bb.data.getVar("S", d, 1)
>     foo-native
>
> Does anyone have any other language enhancement requests for bitbake
> while I'm thinking about it (or any thoughts on the above)?

While you're already busy proposing obscure syntax enhancements :)
What about allowing bashisms/dashisms in the ${...} variable expansion?
We could write
S = "${S%-native}"
or
S = "${S/-native/}"
then.

regards
Philipp