[Buildroot] autobuild annotations

[Buildroot] autobuild annotations

[Thomas De Schampheleire]

Hi,

What I am missing in the current autobuild website/flow is a way to
indicate that a certain problem is already analyzed by someone, and
what the problem is. Maybe a patch has been submitted already and
waiting to be merged. Maybe the solution is clear but someone still
needs to make a patch. Maybe it's more complex and we have to wait for
feedback from upstream.

If we could have such a 'comment' or 'annotations' field, we could
avoid duplicating the analyis / resolution effort.

One way to proceed is to simply keep a comment for each package. When
there is a failure, we show (or link to) the comment for that package,
even though at that time we are not yet sure that the problem is
exactly the same (one package could fail in multiple ways). The
comment should have a link to the problem on which it was added
originally, so that the developer looking at the new problem can
analyze if it is the same problem or not.

To implement this, we'd need an extra table in the database to map a
package to a comment, and we need a query to extract the comments for
a given package from that table when displaying the results.

What do you think?
Do you have alternative proposals?

Thanks,
Thomas

[Buildroot] autobuild annotations

[arnaud aujon]

Hi,

You're right I think there is something missing in the current workflow.
I was wondering if Jenkins can be used to solve this problem and bring some
meatrics and code conventions checking.

I noticed that free-electrons has a Jenkins instance building buildroot :
http://jenkins.free-electrons.com/job/buildroot/ is it used by the
community too ?
.
I think the advantage of Jenkins against the current flow is that there is
plenty of plugins that can be (easily?) used to fulfil the project needs
(comment on failure, meatrics, code convention checking...)
Did you ever talk about that ?

Regards,

Arnaud



2013/11/20 Thomas De Schampheleire <patrickdepinguin@gmail.com>

> Hi,
>
> What I am missing in the current autobuild website/flow is a way to
> indicate that a certain problem is already analyzed by someone, and
> what the problem is. Maybe a patch has been submitted already and
> waiting to be merged. Maybe the solution is clear but someone still
> needs to make a patch. Maybe it's more complex and we have to wait for
> feedback from upstream.
>
> If we could have such a 'comment' or 'annotations' field, we could
> avoid duplicating the analyis / resolution effort.
>
> One way to proceed is to simply keep a comment for each package. When
> there is a failure, we show (or link to) the comment for that package,
> even though at that time we are not yet sure that the problem is
> exactly the same (one package could fail in multiple ways). The
> comment should have a link to the problem on which it was added
> originally, so that the developer looking at the new problem can
> analyze if it is the same problem or not.
>
> To implement this, we'd need an extra table in the database to map a
> package to a comment, and we need a query to extract the comments for
> a given package from that table when displaying the results.
>
> What do you think?
> Do you have alternative proposals?
>
> Thanks,
> Thomas
> _______________________________________________
> buildroot mailing list
> buildroot at busybox.net
> http://lists.busybox.net/mailman/listinfo/buildroot
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.busybox.net/pipermail/buildroot/attachments/20131120/ea16414b/attachment.html>

[Buildroot] autobuild annotations

[Thomas Petazzoni]

Dear arnaud aujon,

On Wed, 20 Nov 2013 11:38:32 +0100, arnaud aujon wrote:

> You're right I think there is something missing in the current
> workflow. I was wondering if Jenkins can be used to solve this
> problem and bring some meatrics and code conventions checking.
> 
> I noticed that free-electrons has a Jenkins instance building
> buildroot : http://jenkins.free-electrons.com/job/buildroot/ is it
> used by the community too ?

Yes. My colleague Maxime Ripard is maintaining that, and Peter, Maxime
and myself are receiving e-mails whenever a build fails.

> I think the advantage of Jenkins against the current flow is that
> there is plenty of plugins that can be (easily?) used to fulfil the
> project needs (comment on failure, meatrics, code convention
> checking...) Did you ever talk about that ?

Jenkins just doesn't work for random builds.

Jenkins is designed to build the same configuration over and over
again, and make comparisons from one build to another: the build was
working yesterday, it's failing today, here is the set of commits that
have been made between yesterday and today, and therefore the problem
is somewhere in these commits. Jenkins is really good for that, and we
already use it to build all the Buildroot defconfigs every day.

However, what autobuild.buildroot.org is doing are *random*
configurations. I.e you cannot compare one build to another. This
really doesn't fit well in the model of Jenkins in my opinion, and is
the reason why a small specific tool was written.

Best regards,

Thomas
-- 
Thomas Petazzoni, CTO, Free Electrons
Embedded Linux, Kernel and Android engineering
http://free-electrons.com

[Buildroot] autobuild annotations

[Jeremy Rosen]

> 
> Jenkins just doesn't work for random builds.
> 
> Jenkins is designed to build the same configuration over and over
> again, and make comparisons from one build to another: the build was
> working yesterday, it's failing today, here is the set of commits
> that
> have been made between yesterday and today, and therefore the problem
> is somewhere in these commits. Jenkins is really good for that, and
> we
> already use it to build all the Buildroot defconfigs every day.
> 
> However, what autobuild.buildroot.org is doing are *random*
> configurations. I.e you cannot compare one build to another. This
> really doesn't fit well in the model of Jenkins in my opinion, and is
> the reason why a small specific tool was written.
> 

couldn't failed build configurations be forwarded to jenkins for monitoring ?

Jenkins could drop those compilations once they work again, that would provide
some form of followup...

[Buildroot] autobuild annotations

[Thomas Petazzoni]

Dear Jeremy Rosen,

On Wed, 20 Nov 2013 12:03:30 +0100 (CET), Jeremy Rosen wrote:

> > However, what autobuild.buildroot.org is doing are *random*
> > configurations. I.e you cannot compare one build to another. This
> > really doesn't fit well in the model of Jenkins in my opinion, and
> > is the reason why a small specific tool was written.
> 
> couldn't failed build configurations be forwarded to jenkins for
> monitoring ?
> 
> Jenkins could drop those compilations once they work again, that
> would provide some form of followup...

Is this really needed? I mean, the autobuilders are already showing us
when a problem occurs, and then we fairly quickly see when it no longer
occurs.

Of course, if someone wants to set up a Jenkins that does this, I have
no problem changing the autobuild.buildroot.org thing to forward broken
configurations to some Jenkins instance. But I personally don't think
it's really worth the effort.

Thomas
-- 
Thomas Petazzoni, CTO, Free Electrons
Embedded Linux, Kernel and Android engineering
http://free-electrons.com