Patchwork & patch handling improvements

Patchwork & patch handling improvements

[Paul Eggleton]

Hi all,

Over the past several years one of the regular complaints people have made 
about our project has been that patches sometimes take a long time to make it 
into master, and it's not always clear what the state of a patch is during 
that time. On the other side of things, maintainers are finding it increasingly 
hard to keep up with testing and integrating incoming patches. Additionally, 
trivial mistakes sometimes creep in that would be fairly easy to catch with an 
automated process. We've been talking about this for a while and now I'd like 
to propose a plan to finally address this:

1) Upgrade the OE Patchwork instance [0] to a newer release; this should fix 
some of the problems we are having [1] plus give us additional features. I 
propose using the fork that freedesktop.org are using [2] [3] which is moving 
a bit faster than upstream Patchwork; whilst the changes there may eventually 
make it upstream (and work is ongoing there) we have a much greater ability to 
influence the fork given that it's being worked on by one of my colleagues who 
is pushing it in the direction we need it to go e.g. proper support for series 
as opposed to treating every patch individually, improved UI, etc.

2) Trigger automatic testing of submitted patches from Patchwork. We'd have a 
script look at the contents of a patch and first check that the expected style 
has been adhered to; second it would do some quick tests to verify that the 
patch hasn't caused any immediately obvious regressions. I've filed some bugs 
to cover this in more detail [4].

3) Provide a means to easily schedule an overnight build on the autobuilder 
for the set of patches that have passed the initial testing, as well as 
present the results in a form that's easier to review for maintainers. For OE-
Core this would be tied into the Yocto Project autobuilder, but I expect the 
tools could be made flexible.

At all stages through this process the patch status in patchwork will be kept 
up-to-date so it's clear to everyone what's happening. I'm also thinking we 
could do email notifications to the submitter (opt-in) though that would be a 
later add-on.

Whilst the initial plan covers only OE-Core, once we get them working the 
tools and scripts used should be just as applicable to other layers. I'm also 
trying to ensure that the patch validation is generic enough so it can live in 
OE-Core, and thus we can easily update and refine it over time in line with the 
code itself as well as encourage submitters to use the script on their own 
changes before sending.

Please let me know your thoughts on the above, most importantly on the 
patchwork upgrade, since most of this hinges on that; I don't believe we can 
practically base it on the version of Patchwork we are using now. 

[Note that this email is cross-posted - it may be best to reply on OE-Core 
only to save people's inboxes.]

Thanks,
Paul

[0] http://patchwork.openembedded.org/
[1] https://bugzilla.yoctoproject.org/show_bug.cgi?id=7657#c21
[2] http://patchwork.freedesktop.org/ 
[3] https://github.com/dlespiau/patchwork
[4] https://bugzilla.yoctoproject.org/show_bug.cgi?id=8648

-- 

Paul Eggleton
Intel Open Source Technology Centre

Re: [OE-core] Patchwork & patch handling improvements

[Trevor Woerner]

On 11/26/15 16:00, Paul Eggleton wrote:
> I'm also 
> trying to ensure that the patch validation is generic enough so it can live in 
> OE-Core, and thus we can easily update and refine it over time in line with the 
> code itself as well as encourage submitters to use the script on their own 
> changes before sending.

This all sounds like an improvement and is therefore a step in the right
direction :-)

A while back I had the idea of "porting" the kernel's "checkpatch.pl" to
The Yocto Project (it was around the same time that I was trying to
float the whole "Maintainers File" idea too, since I was also trying to
re-purpose "get-maintainer.pl" as well). About one minute into that
effort I realized the existing *.bb files were all over the place in
terms of the order of statements and the order of the blocks of
statements. At that time I found one recipe style guide from OE, and
another one from The Yocto Project, each of which described a slightly
different preference. So I asked on the mailing list and quickly
discovered that both groups prefer a different style.

I'm not saying this job isn't worth doing, but I am pointing out there's
the potential for feathers to be ruffled on both sides if someone tries
to produce a definitive style guide for recipe files and then enforces
it in an automated way. Since it is the OpenEmbedded Project's job to
provide the recipes for The Yocto Project, I'm guessing this question
needs to be decided by them? If that sounds reasonable, then maybe The
Yocto Project needs to acquiesce to OE's decision?

Instead of cross-posting, maybe this would be a good email for the new
architecture list (CC'ed)?

Re: [OE-core] Patchwork & patch handling improvements

[Paul Eggleton]

Hi Trevor,

On Mon, 30 Nov 2015 10:19:35 Trevor Woerner wrote:
> On 11/26/15 16:00, Paul Eggleton wrote:
> > I'm also
> > trying to ensure that the patch validation is generic enough so it can
> > live in OE-Core, and thus we can easily update and refine it over time in
> > line with the code itself as well as encourage submitters to use the
> > script on their own changes before sending.
> 
> This all sounds like an improvement and is therefore a step in the right
> direction :-)
> 
> A while back I had the idea of "porting" the kernel's "checkpatch.pl" to
> The Yocto Project (it was around the same time that I was trying to
> float the whole "Maintainers File" idea too, since I was also trying to
> re-purpose "get-maintainer.pl" as well). About one minute into that
> effort I realized the existing *.bb files were all over the place in
> terms of the order of statements and the order of the blocks of
> statements. At that time I found one recipe style guide from OE, and
> another one from The Yocto Project, each of which described a slightly
> different preference. So I asked on the mailing list and quickly
> discovered that both groups prefer a different style.
> 
> I'm not saying this job isn't worth doing, but I am pointing out there's
> the potential for feathers to be ruffled on both sides if someone tries
> to produce a definitive style guide for recipe files and then enforces
> it in an automated way. Since it is the OpenEmbedded Project's job to
> provide the recipes for The Yocto Project, I'm guessing this question
> needs to be decided by them? If that sounds reasonable, then maybe The
> Yocto Project needs to acquiesce to OE's decision?

I don't think there's that much of a division. I don't recall if it was you 
that raised it at the time but the issue of having two style guides did get 
rectified - I changed the one on the Yocto Project wiki to simply be a link to 
the OE style guide in June last year. It certainly didn't come about through a 
conscious decision to have a different style.

However there is a minor disagreement over indentation for shell functions 
between OE-Core and other layers - this persists because of the backporting 
pain a blanket replacement would potentially lead to. As I recall this did get 
discussed at the OE TSC level. I think that's one thing we could just not 
evaluate (or make an option) until such time as we resolve the difference - and 
I do mean to see it resolved at some point in the future.

> Instead of cross-posting, maybe this would be a good email for the new
> architecture list (CC'ed)?

Perhaps yes; I'm a bit concerned that list still doesn't have that many 
subscribers though (currently 28, two of which are the same person).

Cheers,
Paul

-- 

Paul Eggleton
Intel Open Source Technology Centre

Re: [OE-core] Patchwork & patch handling improvements

[Martin Jansa]

[-- Attachment #1: Type: text/plain, Size: 3859 bytes --]

On Tue, Dec 01, 2015 at 07:49:50AM +1300, Paul Eggleton wrote:
> Hi Trevor,
> 
> On Mon, 30 Nov 2015 10:19:35 Trevor Woerner wrote:
> > On 11/26/15 16:00, Paul Eggleton wrote:
> > > I'm also
> > > trying to ensure that the patch validation is generic enough so it can
> > > live in OE-Core, and thus we can easily update and refine it over time in
> > > line with the code itself as well as encourage submitters to use the
> > > script on their own changes before sending.
> > 
> > This all sounds like an improvement and is therefore a step in the right
> > direction :-)
> > 
> > A while back I had the idea of "porting" the kernel's "checkpatch.pl" to
> > The Yocto Project (it was around the same time that I was trying to
> > float the whole "Maintainers File" idea too, since I was also trying to
> > re-purpose "get-maintainer.pl" as well). About one minute into that
> > effort I realized the existing *.bb files were all over the place in
> > terms of the order of statements and the order of the blocks of
> > statements. At that time I found one recipe style guide from OE, and
> > another one from The Yocto Project, each of which described a slightly
> > different preference. So I asked on the mailing list and quickly
> > discovered that both groups prefer a different style.
> > 
> > I'm not saying this job isn't worth doing, but I am pointing out there's
> > the potential for feathers to be ruffled on both sides if someone tries
> > to produce a definitive style guide for recipe files and then enforces
> > it in an automated way. Since it is the OpenEmbedded Project's job to
> > provide the recipes for The Yocto Project, I'm guessing this question
> > needs to be decided by them? If that sounds reasonable, then maybe The
> > Yocto Project needs to acquiesce to OE's decision?
> 
> I don't think there's that much of a division. I don't recall if it was you 
> that raised it at the time but the issue of having two style guides did get 
> rectified - I changed the one on the Yocto Project wiki to simply be a link to 
> the OE style guide in June last year. It certainly didn't come about through a 
> conscious decision to have a different style.
> 
> However there is a minor disagreement over indentation for shell functions 
> between OE-Core and other layers - this persists because of the backporting 
> pain a blanket replacement would potentially lead to. As I recall this did get 
> discussed at the OE TSC level. I think that's one thing we could just not 
> evaluate (or make an option) until such time as we resolve the difference - and 
> I do mean to see it resolved at some point in the future.

Using consistent indentation (4 spaces) at least for new metadata would
be step in right direction.

With the amount of changes which are backported to older releases I
still don't see this "backporting pain" argument. Doing it just before
the release is of course useful, because e.g. now more changes will be
backported to Jethro than to Fido or Dizzy. So having consistent
indentation in Jethro and master would prevent 95% of "backporting
pain". Maybe some Yocto 10.0 will finally get the meaning of
"consistent" indentation.

Regards,

> > Instead of cross-posting, maybe this would be a good email for the new
> > architecture list (CC'ed)?
> 
> Perhaps yes; I'm a bit concerned that list still doesn't have that many 
> subscribers though (currently 28, two of which are the same person).
> 
> Cheers,
> Paul
> 
> -- 
> 
> Paul Eggleton
> Intel Open Source Technology Centre
> -- 
> _______________________________________________
> Openembedded-core mailing list
> Openembedded-core@lists.openembedded.org
> http://lists.openembedded.org/mailman/listinfo/openembedded-core

-- 
Martin 'JaMa' Jansa     jabber: Martin.Jansa@gmail.com

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 188 bytes --]

Re: [OE-core] Patchwork & patch handling improvements

[Paul Eggleton]

On Tue, 01 Dec 2015 11:47:20 Martin Jansa wrote:
> On Tue, Dec 01, 2015 at 07:49:50AM +1300, Paul Eggleton wrote:
> > Hi Trevor,
> > 
> > On Mon, 30 Nov 2015 10:19:35 Trevor Woerner wrote:
> > > On 11/26/15 16:00, Paul Eggleton wrote:
> > > > I'm also
> > > > trying to ensure that the patch validation is generic enough so it can
> > > > live in OE-Core, and thus we can easily update and refine it over time
> > > > in
> > > > line with the code itself as well as encourage submitters to use the
> > > > script on their own changes before sending.
> > > 
> > > This all sounds like an improvement and is therefore a step in the right
> > > direction :-)
> > > 
> > > A while back I had the idea of "porting" the kernel's "checkpatch.pl" to
> > > The Yocto Project (it was around the same time that I was trying to
> > > float the whole "Maintainers File" idea too, since I was also trying to
> > > re-purpose "get-maintainer.pl" as well). About one minute into that
> > > effort I realized the existing *.bb files were all over the place in
> > > terms of the order of statements and the order of the blocks of
> > > statements. At that time I found one recipe style guide from OE, and
> > > another one from The Yocto Project, each of which described a slightly
> > > different preference. So I asked on the mailing list and quickly
> > > discovered that both groups prefer a different style.
> > > 
> > > I'm not saying this job isn't worth doing, but I am pointing out there's
> > > the potential for feathers to be ruffled on both sides if someone tries
> > > to produce a definitive style guide for recipe files and then enforces
> > > it in an automated way. Since it is the OpenEmbedded Project's job to
> > > provide the recipes for The Yocto Project, I'm guessing this question
> > > needs to be decided by them? If that sounds reasonable, then maybe The
> > > Yocto Project needs to acquiesce to OE's decision?
> > 
> > I don't think there's that much of a division. I don't recall if it was
> > you
> > that raised it at the time but the issue of having two style guides did
> > get
> > rectified - I changed the one on the Yocto Project wiki to simply be a
> > link to the OE style guide in June last year. It certainly didn't come
> > about through a conscious decision to have a different style.
> > 
> > However there is a minor disagreement over indentation for shell functions
> > between OE-Core and other layers - this persists because of the
> > backporting
> > pain a blanket replacement would potentially lead to. As I recall this did
> > get discussed at the OE TSC level. I think that's one thing we could just
> > not evaluate (or make an option) until such time as we resolve the
> > difference - and I do mean to see it resolved at some point in the
> > future.
> 
> Using consistent indentation (4 spaces) at least for new metadata would
> be step in right direction.
> 
> With the amount of changes which are backported to older releases I
> still don't see this "backporting pain" argument. Doing it just before
> the release is of course useful, because e.g. now more changes will be
> backported to Jethro than to Fido or Dizzy. So having consistent
> indentation in Jethro and master would prevent 95% of "backporting
> pain". Maybe some Yocto 10.0 will finally get the meaning of
> "consistent" indentation.

I agree it's not ideal. I said above, I do want to see it resolved.

Leaving indentation aside for a moment do you have any comments on my 
proposal?

Thanks,
Paul

-- 

Paul Eggleton
Intel Open Source Technology Centre

Re: [OE-core] Patchwork & patch handling improvements

[Martin Jansa]

[-- Attachment #1: Type: text/plain, Size: 4882 bytes --]

On Wed, Dec 02, 2015 at 04:01:40PM +1300, Paul Eggleton wrote:
> On Tue, 01 Dec 2015 11:47:20 Martin Jansa wrote:
> > On Tue, Dec 01, 2015 at 07:49:50AM +1300, Paul Eggleton wrote:
> > > Hi Trevor,
> > > 
> > > On Mon, 30 Nov 2015 10:19:35 Trevor Woerner wrote:
> > > > On 11/26/15 16:00, Paul Eggleton wrote:
> > > > > I'm also
> > > > > trying to ensure that the patch validation is generic enough so it can
> > > > > live in OE-Core, and thus we can easily update and refine it over time
> > > > > in
> > > > > line with the code itself as well as encourage submitters to use the
> > > > > script on their own changes before sending.
> > > > 
> > > > This all sounds like an improvement and is therefore a step in the right
> > > > direction :-)
> > > > 
> > > > A while back I had the idea of "porting" the kernel's "checkpatch.pl" to
> > > > The Yocto Project (it was around the same time that I was trying to
> > > > float the whole "Maintainers File" idea too, since I was also trying to
> > > > re-purpose "get-maintainer.pl" as well). About one minute into that
> > > > effort I realized the existing *.bb files were all over the place in
> > > > terms of the order of statements and the order of the blocks of
> > > > statements. At that time I found one recipe style guide from OE, and
> > > > another one from The Yocto Project, each of which described a slightly
> > > > different preference. So I asked on the mailing list and quickly
> > > > discovered that both groups prefer a different style.
> > > > 
> > > > I'm not saying this job isn't worth doing, but I am pointing out there's
> > > > the potential for feathers to be ruffled on both sides if someone tries
> > > > to produce a definitive style guide for recipe files and then enforces
> > > > it in an automated way. Since it is the OpenEmbedded Project's job to
> > > > provide the recipes for The Yocto Project, I'm guessing this question
> > > > needs to be decided by them? If that sounds reasonable, then maybe The
> > > > Yocto Project needs to acquiesce to OE's decision?
> > > 
> > > I don't think there's that much of a division. I don't recall if it was
> > > you
> > > that raised it at the time but the issue of having two style guides did
> > > get
> > > rectified - I changed the one on the Yocto Project wiki to simply be a
> > > link to the OE style guide in June last year. It certainly didn't come
> > > about through a conscious decision to have a different style.
> > > 
> > > However there is a minor disagreement over indentation for shell functions
> > > between OE-Core and other layers - this persists because of the
> > > backporting
> > > pain a blanket replacement would potentially lead to. As I recall this did
> > > get discussed at the OE TSC level. I think that's one thing we could just
> > > not evaluate (or make an option) until such time as we resolve the
> > > difference - and I do mean to see it resolved at some point in the
> > > future.
> > 
> > Using consistent indentation (4 spaces) at least for new metadata would
> > be step in right direction.
> > 
> > With the amount of changes which are backported to older releases I
> > still don't see this "backporting pain" argument. Doing it just before
> > the release is of course useful, because e.g. now more changes will be
> > backported to Jethro than to Fido or Dizzy. So having consistent
> > indentation in Jethro and master would prevent 95% of "backporting
> > pain". Maybe some Yocto 10.0 will finally get the meaning of
> > "consistent" indentation.
> 
> I agree it's not ideal. I said above, I do want to see it resolved.
> 
> Leaving indentation aside for a moment do you have any comments on my 
> proposal?

I'm not familiar with FDO fork, so I don't know how it looks and
behaves, but any improvement on patchwork side is definitely welcome and
I appreciate it.

Does it support e.g. moving the patches to given bundle based on some
substring in subject? To sort e.g. meta-networking, meta-java,
meta-browser, .. patches automatically?

I don't expect FDO fork to provide other features I'm used to from
gerrit like cherry-picking to selected branch from the UI or doing the
review there. But still if we're stuck with patchwork forever (because
some people hate gerrit), then any improvement is really appreciated,
thanks for looking into it.

My only concern is about migrating current database, do you know if the
migration will keep the database including bundles as they are or do you
plan to set FDO version in parallel at least for some transition period?
Currently I have many patches in flight, because jenkins is running full
test-dependencies job for last 11 and based on progress it will take
14-21 more days to finish.

Regards,

-- 
Martin 'JaMa' Jansa     jabber: Martin.Jansa@gmail.com

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 188 bytes --]

Re: [OE-core] Patchwork & patch handling improvements

[Richard Purdie]

On Mon, 2015-11-30 at 10:19 -0500, Trevor Woerner wrote:
> On 11/26/15 16:00, Paul Eggleton wrote:
> > I'm also 
> > trying to ensure that the patch validation is generic enough so it can live in 
> > OE-Core, and thus we can easily update and refine it over time in line with the 
> > code itself as well as encourage submitters to use the script on their own 
> > changes before sending.
> 
> This all sounds like an improvement and is therefore a step in the right
> direction :-)
> 
> A while back I had the idea of "porting" the kernel's "checkpatch.pl" to
> The Yocto Project (it was around the same time that I was trying to
> float the whole "Maintainers File" idea too, since I was also trying to
> re-purpose "get-maintainer.pl" as well). About one minute into that
> effort I realized the existing *.bb files were all over the place in
> terms of the order of statements and the order of the blocks of
> statements. At that time I found one recipe style guide from OE, and
> another one from The Yocto Project, each of which described a slightly
> different preference. So I asked on the mailing list and quickly
> discovered that both groups prefer a different style.
> 
> I'm not saying this job isn't worth doing, but I am pointing out there's
> the potential for feathers to be ruffled on both sides if someone tries
> to produce a definitive style guide for recipe files and then enforces
> it in an automated way. Since it is the OpenEmbedded Project's job to
> provide the recipes for The Yocto Project, I'm guessing this question
> needs to be decided by them? If that sounds reasonable, then maybe The
> Yocto Project needs to acquiesce to OE's decision?
> 
> Instead of cross-posting, maybe this would be a good email for the new
> architecture list (CC'ed)?

I think the areas where there are disagreements are comparatively small,
really just on shell whitespace. Where they do exist, they are
problematic, not least as some layers effectively ignored an agreement
made by the TSC simply because they didn't agree with it. It basically
means the OE TSC only applies to OE-Core as far as I can tell, which is
sad but is the decision that was made. This also means the TSC has no
real influence over any proposed coding style being used outside
OE-Core.

I do still believe shell whitespace changes would cause significant
patch compatibility issues, I know I disagree on Martin over that. I
still don't like the idea of people blindly running a formatting script
since we'll than start seeing patches every time there is a single space
in the wrong place. We simply don't want that amount of churn on the
metadata.

I can imagine several people replying and saying that patch churn is not
an issue but having seen the things people send patches for, I believe
it will be. I don't want to encourage such things as I believe there are
better things to do with our time (mine included as I'd have to review
them, even to just say 'no').

The maintainers file is a different problem and its one of maintenance,
and more specifically what being listed means, who can be listed, how
that listing can be changed and so on. The Yocto Project has some notion
of maintainer and there its easy, Ross and I can made decisions on who
is listed and what those people are expected to do and we can make it
work (its how we ensure things get upgraded with some regularity). For
OE, who would do this and what would the maintainer file mean? If
someone patches something, are they required to cc the maintainer on
patches for example? (that would imply workflow overhead) What if they
don't cc a maintainer? Should we be forced to revert such a patch?

In many ways its like the "what is a stable branch?" question. Some
people want to use a maintainers file as a way of having a veto on
certain patches. Others want to use it as a way of finding people to fix
bugs. Others again want it to help review patches. The uses vary and you
need a clear definition about what its being used for to make it work.

If someone wants to put together a proposal about which problem this
solves, with clear definitions/charter about how it would all work,
great, but I've seen a lot of problems with such files and I worry it
creates more problems than it would solve.

Cheers,

Richard