Cleanup / janitors

Cleanup / janitors

[Chris Larson]

Greetings all,

Per discussion in the TSC meeting this month, I'm sending out this email as
a probe.  Would anyone on the list benefit from a "janitors" type project,
to outline and document specific cleanup tasks and how they should be done?
There are many tasks that need doing which involve editing of large amounts
of files in relatively simple ways, and I suspect, but am not yet certain,
that there are a group of individuals who wish to contribute to the project,
but are new enough that they don't feel confident diving into anything
invasive.  This sort of a task could be a starting point for that sort of
individual.  Thoughts?  Anyone out there in that camp currently that would
benefit from something like this?
-- 
Christopher Larson
clarson at kergoth dot com
Founder - BitBake, OpenEmbedded, OpenZaurus
Maintainer - Tslib
Senior Software Engineer, Mentor Graphics

Re: Cleanup / janitors

[Koen Kooi]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On 16-02-10 00:38, Chris Larson wrote:
> Greetings all,
> 
> Per discussion in the TSC meeting this month, I'm sending out this email as
> a probe.  Would anyone on the list benefit from a "janitors" type project,
> to outline and document specific cleanup tasks and how they should be done?
> There are many tasks that need doing which involve editing of large amounts
> of files in relatively simple ways, and I suspect, but am not yet certain,
> that there are a group of individuals who wish to contribute to the project,
> but are new enough that they don't feel confident diving into anything
> invasive.  This sort of a task could be a starting point for that sort of
> individual.  Thoughts?  Anyone out there in that camp currently that would
> benefit from something like this?

The first project that comes to mind is converting simple recipes (e.g.
not perl) to new style staging. It's a fairly simple job that has a huge
impact.

regards,

Koen
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.5 (Darwin)

iD8DBQFLelOMMkyGM64RGpERApKkAJ4+QNjJ/0ZMK4wCG2CI6scbEiZCmgCghho1
lvW3hA7NjMtVFID666fT2O4=
=sxuZ
-----END PGP SIGNATURE-----

Re: Cleanup / janitors

[Frans Meulenbroeks]

2010/2/16 Koen Kooi <k.kooi@student.utwente.nl>:
> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
>
> On 16-02-10 00:38, Chris Larson wrote:
>> Greetings all,
>>
>> Per discussion in the TSC meeting this month, I'm sending out this email as
>> a probe.  Would anyone on the list benefit from a "janitors" type project,
>> to outline and document specific cleanup tasks and how they should be done?
>> There are many tasks that need doing which involve editing of large amounts
>> of files in relatively simple ways, and I suspect, but am not yet certain,
>> that there are a group of individuals who wish to contribute to the project,
>> but are new enough that they don't feel confident diving into anything
>> invasive.  This sort of a task could be a starting point for that sort of
>> individual.  Thoughts?  Anyone out there in that camp currently that would
>> benefit from something like this?
>
> The first project that comes to mind is converting simple recipes (e.g.
> not perl) to new style staging. It's a fairly simple job that has a huge
> impact.

Problem is to do that reliable.
It is easy to remove the do_stage section from a recipe, and it is
easy to verify that it builds, but it is already becoming less trivial
to verify that everything that depends on it builds too, and it
becomes complicated if you want to verify that it actually works
(especially if you have no clue what the package does).
(btw actually when trying to build dependend packages you should
either use packaged staging or remove your work dir).

I don't mind spending an hour or so removing some do_stage things, and
I don't mind if my computer hums along a night or so to compile those
packages, but I'd rather not burn my fingers and get blamed for
breaking things outside my control & competenence.

The other thing that comes to mind is better describing the job. If it
indeed is only removing do_stage it is probably better done by someone
with commit access, otherwise someone else has to spend the same
amount of time to apply the patch, remove it from patchwork etc.

Frans

Re: Cleanup / janitors

[Andrea Adami]

> Problem is to do that reliable.
I would follow RP steps: starting in nov 2009
http://git.pokylinux.org/cgit.cgi/poky/log/

>It is easy to remove the do_stage section from a recipe, and it is
>easy to verify that it builds, but it is already becoming less trivial
>to verify that everything that depends on it builds too, and it
>becomes complicated if you want to verify that it actually works
>(especially if you have no clue what the package does).
That's the reason why I did not dare too

> The other thing that comes to mind is better describing the job. If it
I'd define it "Monkey-picking from Poky" for the most recipes.

How dangerous could it be? I have no idea about eventual subtle diffs
between Poky and OE.

Regards

Andrea

Re: Cleanup / janitors

[Richard Purdie]

On Tue, 2010-02-16 at 10:08 +0100, Andrea Adami wrote:
> > Problem is to do that reliable.
> I would follow RP steps: starting in nov 2009
> http://git.pokylinux.org/cgit.cgi/poky/log/
> 
> >It is easy to remove the do_stage section from a recipe, and it is
> >easy to verify that it builds, but it is already becoming less trivial
> >to verify that everything that depends on it builds too, and it
> >becomes complicated if you want to verify that it actually works
> >(especially if you have no clue what the package does).
> That's the reason why I did not dare too
> 
> > The other thing that comes to mind is better describing the job. If it
> I'd define it "Monkey-picking from Poky" for the most recipes.
> 
> How dangerous could it be? I have no idea about eventual subtle diffs
> between Poky and OE.

The key thing is to make sure the starting recipes were the same. If
they are, the same change probably applies to OE as they are very
similar, I try and keep them roughly in sync in all important ways.

It would actually make sense to add some function which compares the
results of a staging changes, I don't know if anyone would feel up to
that?

Cheers,

Richard

Re: Cleanup / janitors

[Richard Purdie]

On Tue, 2010-02-16 at 09:39 +0100, Frans Meulenbroeks wrote:
> 2010/2/16 Koen Kooi <k.kooi@student.utwente.nl>:
> > On 16-02-10 00:38, Chris Larson wrote:

> The other thing that comes to mind is better describing the job. If it
> indeed is only removing do_stage it is probably better done by someone
> with commit access, otherwise someone else has to spend the same
> amount of time to apply the patch, remove it from patchwork etc.

I'd like to see a contrib tree which anyone can be given access to.
People can queue commits in this, then someone with access to OE.dev can
just pull (or cherrypick) from it.

This already exists for Poky.

Cheers,

Richard

contrib tree (was: Cleanup / janitors)

[Rolf Leggewie]

Richard Purdie wrote:
> I'd like to see a contrib tree which anyone can be given access to.
> People can queue commits in this, then someone with access to OE.dev can
> just pull (or cherrypick) from it.

Yes, I think that is a good idea and we discussed this earlier.  Maybe
we can use the mob branch on repo.or.cz?  That would save us the admin
overhead and worries.  On my request, the repo.or.cz admin recently
added support for an infinite number of mob branches rather than just a
single mob branch, so contributors can commit conflicting things for
review as well.  Unfortunately, I haven't yet found the time to test the
new setup.

Re: Cleanup / janitors

[Koen Kooi]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On 16-02-10 09:39, Frans Meulenbroeks wrote:
> 2010/2/16 Koen Kooi <k.kooi@student.utwente.nl>:
>> -----BEGIN PGP SIGNED MESSAGE-----
>> Hash: SHA1
>>
>> On 16-02-10 00:38, Chris Larson wrote:
>>> Greetings all,
>>>
>>> Per discussion in the TSC meeting this month, I'm sending out this email as
>>> a probe.  Would anyone on the list benefit from a "janitors" type project,
>>> to outline and document specific cleanup tasks and how they should be done?
>>> There are many tasks that need doing which involve editing of large amounts
>>> of files in relatively simple ways, and I suspect, but am not yet certain,
>>> that there are a group of individuals who wish to contribute to the project,
>>> but are new enough that they don't feel confident diving into anything
>>> invasive.  This sort of a task could be a starting point for that sort of
>>> individual.  Thoughts?  Anyone out there in that camp currently that would
>>> benefit from something like this?
>>
>> The first project that comes to mind is converting simple recipes (e.g.
>> not perl) to new style staging. It's a fairly simple job that has a huge
>> impact.
> 
> Problem is to do that reliable.
> It is easy to remove the do_stage section from a recipe, and it is
> easy to verify that it builds, but it is already becoming less trivial
> to verify that everything that depends on it builds too,

A simpile first step is:

1) build recipe
2) bump PR
3) fix staging
4) build recipe
5) diff the 2 packaged-staging recipes

If the result of 4) is a superset of 1) then you most likely did the
right thing.

regards,

Koen
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.5 (Darwin)

iD8DBQFLemg9MkyGM64RGpERAsJhAKCVNBO2i/El72HF3YNgz9qdm8QenQCfbwg9
g2/Zq8mnEUP7Z5OqNdYX7xA=
=pT6Y
-----END PGP SIGNATURE-----

Re: Cleanup / janitors

[Tom Rini]

On Tue, 2010-02-16 at 09:39 +0100, Frans Meulenbroeks wrote:
> 2010/2/16 Koen Kooi <k.kooi@student.utwente.nl>:
> > -----BEGIN PGP SIGNED MESSAGE-----
> > Hash: SHA1
> >
> > On 16-02-10 00:38, Chris Larson wrote:
> >> Greetings all,
> >>
> >> Per discussion in the TSC meeting this month, I'm sending out this email as
> >> a probe.  Would anyone on the list benefit from a "janitors" type project,
> >> to outline and document specific cleanup tasks and how they should be done?
> >> There are many tasks that need doing which involve editing of large amounts
> >> of files in relatively simple ways, and I suspect, but am not yet certain,
> >> that there are a group of individuals who wish to contribute to the project,
> >> but are new enough that they don't feel confident diving into anything
> >> invasive.  This sort of a task could be a starting point for that sort of
> >> individual.  Thoughts?  Anyone out there in that camp currently that would
> >> benefit from something like this?
> >
> > The first project that comes to mind is converting simple recipes (e.g.
> > not perl) to new style staging. It's a fairly simple job that has a huge
> > impact.
> 
> Problem is to do that reliable.

Isn't that a document it right, for Janitors problem?  ie:
1. Build recipe foo, with packaged-staging, your resuling pstage file
is ..../foo_VERSION...ipk.  Copy this somewhere safe.
2. Modify recipe foo to use new style
3. -c clean foo, build again.
4. Examine contents of old and new pstaging file.  If the contents are
the same (and say, diff rather than just checking names, so that any
munging is caught), good to go!  If not ...

The idea behind janitors is that there's tasks that can be explained and
people can pick up and do, but there's some manual intervention and
checking.  So, if someone fleshed out the above a little more, that
should be enough for a legacy staging task.

-- 
Tom Rini <tom_rini@mentor.com>
Mentor Graphics Corporation

Re: Cleanup / janitors

[Chris Larson]

On Tue, Feb 16, 2010 at 4:30 PM, Tom Rini <tom_rini@mentor.com> wrote:

> On Tue, 2010-02-16 at 09:39 +0100, Frans Meulenbroeks wrote:
> > 2010/2/16 Koen Kooi <k.kooi@student.utwente.nl>:
> > > -----BEGIN PGP SIGNED MESSAGE-----
> > > Hash: SHA1
> > >
> > > On 16-02-10 00:38, Chris Larson wrote:
> > >> Greetings all,
> > >>
> > >> Per discussion in the TSC meeting this month, I'm sending out this
> email as
> > >> a probe.  Would anyone on the list benefit from a "janitors" type
> project,
> > >> to outline and document specific cleanup tasks and how they should be
> done?
> > >> There are many tasks that need doing which involve editing of large
> amounts
> > >> of files in relatively simple ways, and I suspect, but am not yet
> certain,
> > >> that there are a group of individuals who wish to contribute to the
> project,
> > >> but are new enough that they don't feel confident diving into anything
> > >> invasive.  This sort of a task could be a starting point for that sort
> of
> > >> individual.  Thoughts?  Anyone out there in that camp currently that
> would
> > >> benefit from something like this?
> > >
> > > The first project that comes to mind is converting simple recipes (e.g.
> > > not perl) to new style staging. It's a fairly simple job that has a
> huge
> > > impact.
> >
> > Problem is to do that reliable.
>
> Isn't that a document it right, for Janitors problem?  ie:
> 1. Build recipe foo, with packaged-staging, your resuling pstage file
> is ..../foo_VERSION...ipk.  Copy this somewhere safe.
> 2. Modify recipe foo to use new style
> 3. -c clean foo, build again.
> 4. Examine contents of old and new pstaging file.  If the contents are
> the same (and say, diff rather than just checking names, so that any
> munging is caught), good to go!  If not ...
>
> The idea behind janitors is that there's tasks that can be explained and
> people can pick up and do, but there's some manual intervention and
> checking.  So, if someone fleshed out the above a little more, that
> should be enough for a legacy staging task.


Exactly, yes.  The expectation is these are things that may require a
certain amount of manual work.  If they didn't, we could write a sed script
and call it done.  The thread seems to have gotten quickly sidetracked into
the particular details of this one task.
-- 
Christopher Larson
clarson at kergoth dot com
Founder - BitBake, OpenEmbedded, OpenZaurus
Maintainer - Tslib
Senior Software Engineer, Mentor Graphics