Re: Analyzing the nightlies

[Yuri Weinstein <yweinste@redhat.com>]

Loic 

You description on high level is correct.  There are, of cause, more details when someone actually goes thru the nightlies results.

As far as you question "Is there a time like bug scrubbing or sprint planning when developers say "Let's analyze QA results and dig bugs" ?" - I surely hope so and do see updates and triages on bugs in the tracker, but not 100% sure what exactly our process is, so Sage and development leads are better persons to ask this.

Also when you say "What I'm not sure about is if it's best effort ? "  - do you have something in mind instead or in addition to what we do now?
( I hope something that can lighten the burden :) )


Thx
YuriW

----- Original Message -----
From: "Loic Dachary" <loic@dachary.org>
To: "Yuri Weinstein" <yweinste@redhat.com>
Cc: "Ceph Development" <ceph-devel@vger.kernel.org>
Sent: Saturday, May 9, 2015 2:31:03 AM
Subject: Analyzing the nightlies

Hi Yuri,

It would be useful to add more information bout how the nightlies are analyzed at

   http://tracker.ceph.com/projects/ceph-releases/wiki/HOWTO_monitor_the_automated_tests_AKA_nightlies

At this point my understanding is that you look over all of them and you carry the burden of 

* sorting out the environmental noise
* creating new bugs for errors for which there is no match in the tracker
* add a link to the failed job in pre-existing issues found in the tracker (useful to figure out the frequency and helps with debug when there are multiple outputs / logs)

You do so by using tools such as https://github.com/jcsp/scrape/blob/master/scrape.py and maybe others and you also format your mail messages so that they can be parsed by a program (although such a program does not exist yet, it could go over all your messages and build a database from the mails you sent).

In the http://lists.ceph.com/private.cgi/ceph-qa-ceph.com/ archives, I see that Greg also regularly goes over the errors and other developers also do. What I'm not sure about is if it's best effort ? Is there a time like bug scrubbing or sprint planning when developers say "Let's analyze QA results and dig bugs" ?

Cheers

-- 
Loïc Dachary, Artisan Logiciel Libre
--
To unsubscribe from this list: send the line "unsubscribe ceph-devel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html