bb.data.*Var -> d.*Var conversion

bb.data.*Var -> d.*Var conversion

[Richard Purdie]

I'm tempted to run the following over the metata to convert the
bb.data.*Var(...,d) and similar expressions to the form d.*Var(...).

Why? We get a lot of people doing copy, paste and edit of the code and
this way, we'll increase the chances of them finding better examples.

I'm still looking at the diff this generates to see if there are any
more corner cases I need to tweak the expression for but feedback
welcome.

sed \
-e 's:bb.data.\(setVar([^,]*,[^,]*\), \([^ )]*\) *):\2.\1):g' \
-e 's:bb.data.\(setVarFlag([^,]*,[^,]*,[^,]*\), \([^) ]*\) *):\2.\1):g' \
-e 's:bb.data.\(getVar([^,]*\), \([^, ]*\) *,\([^)]*\)):\2.\1,\3):g' \
-e 's:bb.data.\(getVarFlag([^,]*,[^,]*\), \([^, ]*\) *,\([^)]*\)):\2.\1,\3):g' \
-e 's:bb.data.\(getVarFlag([^,]*,[^,]*\), \([^) ]*\) *):\2.\1):g' \
-e 's:bb.data.\(getVar([^,]*\), \([^) ]*\) *):\2.\1):g' \
-i `grep -ril bb.data *`

Cheers,

Richard

Re: bb.data.*Var -> d.*Var conversion

[Darren Hart]

Hi Richard,

On 11/09/2011 03:52 AM, Richard Purdie wrote:
> I'm tempted to run the following over the metata to convert the
> bb.data.*Var(...,d) and similar expressions to the form d.*Var(...).

Oh yes please!

> 
> Why? We get a lot of people doing copy, paste and edit of the code and
> this way, we'll increase the chances of them finding better examples.
> 
> I'm still looking at the diff this generates to see if there are any
> more corner cases I need to tweak the expression for but feedback
> welcome.
> 
> sed \
> -e 's:bb.data.\(setVar([^,]*,[^,]*\), \([^ )]*\) *):\2.\1):g' \

Be sure to escape your periods, they are single character wildcards.
It's unlikely you'll match something else, but better safe than sorry.

> -e 's:bb.data.\(setVarFlag([^,]*,[^,]*,[^,]*\), \([^) ]*\) *):\2.\1):g' \
> -e 's:bb.data.\(getVar([^,]*\), \([^, ]*\) *,\([^)]*\)):\2.\1,\3):g' \
> -e 's:bb.data.\(getVarFlag([^,]*,[^,]*\), \([^, ]*\) *,\([^)]*\)):\2.\1,\3):g' \
> -e 's:bb.data.\(getVarFlag([^,]*,[^,]*\), \([^) ]*\) *):\2.\1):g' \
> -e 's:bb.data.\(getVar([^,]*\), \([^) ]*\) *):\2.\1):g' \
> -i `grep -ril bb.data *`

Why ignore case?

I suggest running the grep first redirected to the file, then editing
the contents to ensure you don't get things like CHANGELOG and probably
be careful with the Documentation.

The groups look sane to me.

The only other thing I'd make sure to try and watch (not necessarily
handle) are multi-line bb.data.[sg]et(Flag)? calls. I suspect there are
few enough of them, if any, that they could be managed by hand.

-- 
Darren Hart
Intel Open Source Technology Center
Yocto Project - Linux Kernel

Re: bb.data.*Var -> d.*Var conversion

[Chris Larson]

On Wed, Nov 9, 2011 at 7:59 AM, Darren Hart <dvhart@linux.intel.com> wrote:
> Hi Richard,
>
> On 11/09/2011 03:52 AM, Richard Purdie wrote:
>> I'm tempted to run the following over the metata to convert the
>> bb.data.*Var(...,d) and similar expressions to the form d.*Var(...).
>
> Oh yes please!
>
>>
>> Why? We get a lot of people doing copy, paste and edit of the code and
>> this way, we'll increase the chances of them finding better examples.
>>
>> I'm still looking at the diff this generates to see if there are any
>> more corner cases I need to tweak the expression for but feedback
>> welcome.
>>
>> sed \
>> -e 's:bb.data.\(setVar([^,]*,[^,]*\), \([^ )]*\) *):\2.\1):g' \
>
> Be sure to escape your periods, they are single character wildcards.
> It's unlikely you'll match something else, but better safe than sorry.
>
>> -e 's:bb.data.\(setVarFlag([^,]*,[^,]*,[^,]*\), \([^) ]*\) *):\2.\1):g' \
>> -e 's:bb.data.\(getVar([^,]*\), \([^, ]*\) *,\([^)]*\)):\2.\1,\3):g' \
>> -e 's:bb.data.\(getVarFlag([^,]*,[^,]*\), \([^, ]*\) *,\([^)]*\)):\2.\1,\3):g' \
>> -e 's:bb.data.\(getVarFlag([^,]*,[^,]*\), \([^) ]*\) *):\2.\1):g' \
>> -e 's:bb.data.\(getVar([^,]*\), \([^) ]*\) *):\2.\1):g' \
>> -i `grep -ril bb.data *`
>
> Why ignore case?
>
> I suggest running the grep first redirected to the file, then editing
> the contents to ensure you don't get things like CHANGELOG and probably
> be careful with the Documentation.
>
> The groups look sane to me.
>
> The only other thing I'd make sure to try and watch (not necessarily
> handle) are multi-line bb.data.[sg]et(Flag)? calls. I suspect there are
> few enough of them, if any, that they could be managed by hand.

Heh, too bad bitbake doesn't know how to emit metadata back out from
its ast. This seems like it'd be an ideal case for writing a custom
2to3 fixer rather than using sed. It would catch the multi-line case
you mention, since it lets you specify grammatical patterns against
the python parse tree and do transforms against it.
-- 
Christopher Larson
clarson at kergoth dot com
Founder - BitBake, OpenEmbedded, OpenZaurus
Maintainer - Tslib
Senior Software Engineer, Mentor Graphics

Re: bb.data.*Var -> d.*Var conversion

[Richard Purdie]

On Wed, 2011-11-09 at 08:24 -0700, Chris Larson wrote:
> On Wed, Nov 9, 2011 at 7:59 AM, Darren Hart <dvhart@linux.intel.com> wrote:
> > Hi Richard,
> >
> > On 11/09/2011 03:52 AM, Richard Purdie wrote:
> >> I'm tempted to run the following over the metata to convert the
> >> bb.data.*Var(...,d) and similar expressions to the form d.*Var(...).
> >
> > Oh yes please!
> >
> >>
> >> Why? We get a lot of people doing copy, paste and edit of the code and
> >> this way, we'll increase the chances of them finding better examples.
> >>
> >> I'm still looking at the diff this generates to see if there are any
> >> more corner cases I need to tweak the expression for but feedback
> >> welcome.
> >>
> >> sed \
> >> -e 's:bb.data.\(setVar([^,]*,[^,]*\), \([^ )]*\) *):\2.\1):g' \
> >
> > Be sure to escape your periods, they are single character wildcards.
> > It's unlikely you'll match something else, but better safe than sorry.

Right, I've not noticed it doing anything nasty but I understand the
concern.

> >> -e 's:bb.data.\(setVarFlag([^,]*,[^,]*,[^,]*\), \([^) ]*\) *):\2.\1):g' \
> >> -e 's:bb.data.\(getVar([^,]*\), \([^, ]*\) *,\([^)]*\)):\2.\1,\3):g' \
> >> -e 's:bb.data.\(getVarFlag([^,]*,[^,]*\), \([^, ]*\) *,\([^)]*\)):\2.\1,\3):g' \
> >> -e 's:bb.data.\(getVarFlag([^,]*,[^,]*\), \([^) ]*\) *):\2.\1):g' \
> >> -e 's:bb.data.\(getVar([^,]*\), \([^) ]*\) *):\2.\1):g' \
> >> -i `grep -ril bb.data *`
> >
> > Why ignore case?

Accidentally left in from something else I borrowed it from. Its
harmless.

> > I suggest running the grep first redirected to the file, then editing
> > the contents to ensure you don't get things like CHANGELOG and probably
> > be careful with the Documentation.

Well, we'd probably want to update those references too. I checked the
resulting diff and it didn't find anything like that though.

> > The groups look sane to me.
> >
> > The only other thing I'd make sure to try and watch (not necessarily
> > handle) are multi-line bb.data.[sg]et(Flag)? calls. I suspect there are
> > few enough of them, if any, that they could be managed by hand.

I was careful not to make this replacement too aggressive. Its currently
ignoring anything multiline and anything where there are expressions
within expressions. There are much less of those and can be handled
either manually or a more targeted expression.

I also want to change:

d.getVar(x, 1) -> d.getVar(x, True)

but that is something for a follow up too.

> Heh, too bad bitbake doesn't know how to emit metadata back out from
> its ast. This seems like it'd be an ideal case for writing a custom
> 2to3 fixer rather than using sed. It would catch the multi-line case
> you mention, since it lets you specify grammatical patterns against
> the python parse tree and do transforms against it.

Indeed, if only I had more time...

Cheers,

Richard

Re: bb.data.*Var -> d.*Var conversion

[Richard Purdie]

On Wed, 2011-11-09 at 11:52 +0000, Richard Purdie wrote:
> I'm tempted to run the following over the metata to convert the
> bb.data.*Var(...,d) and similar expressions to the form d.*Var(...).
> 
> Why? We get a lot of people doing copy, paste and edit of the code and
> this way, we'll increase the chances of them finding better examples.
> 
> I'm still looking at the diff this generates to see if there are any
> more corner cases I need to tweak the expression for but feedback
> welcome.
> 
> sed \
> -e 's:bb.data.\(setVar([^,]*,[^,]*\), \([^ )]*\) *):\2.\1):g' \
> -e 's:bb.data.\(setVarFlag([^,]*,[^,]*,[^,]*\), \([^) ]*\) *):\2.\1):g' \
> -e 's:bb.data.\(getVar([^,]*\), \([^, ]*\) *,\([^)]*\)):\2.\1,\3):g' \
> -e 's:bb.data.\(getVarFlag([^,]*,[^,]*\), \([^, ]*\) *,\([^)]*\)):\2.\1,\3):g' \
> -e 's:bb.data.\(getVarFlag([^,]*,[^,]*\), \([^) ]*\) *):\2.\1):g' \
> -e 's:bb.data.\(getVar([^,]*\), \([^) ]*\) *):\2.\1):g' \
> -i `grep -ril bb.data *`

Incidentally this is worth about a 1.5% increase in parsing speed which
makes sense since it removed one level of function indirection.

Cheers,

Richard