[PATCH 0/7] Fixes to oe-build-perf-report

[PATCH 0/7] Fixes to oe-build-perf-report

[Markus Lehtonen]

A collection of miscellanious small fixes to scripts/oe-build-perf-report. No
changes in the output of the script, except for a more valid html syntax.


The following changes since commit d72d116e0228cc0e4391d0558b2d32c3fd5e399c:

  README.qemu: Add from meta-yocto as it belongs in core (2017-09-14 12:39:29 +0100)

are available in the git repository at:

  git://git.openembedded.org/openembedded-core-contrib marquiz/buildperf/fixes
  http://cgit.openembedded.org/openembedded-core-contrib/log/?h=marquiz/buildperf/fixes

Markus Lehtonen (7):
  scripts/oe-build-perf-report: fix dumping buildstats
  scripts/oe-build-perf-report: add AggregateTestData class
  scripts/oe-build-perf-report: fix handling of --history-length
  scripts/oe-build-perf-report: use --hostname in --list
  scripts/oe-build-perf-report: remove dead code
  scripts/oe-build-perf-report: provide valid title in the html report
  scripts/oe-build-perf-report: tidy up html syntax

 scripts/lib/build_perf/html/report.html | 36 ++++++++++++++++++---------------
 scripts/lib/build_perf/report.py        |  5 ++++-
 scripts/oe-build-perf-report            | 33 ++++++++++++++++--------------
 3 files changed, 42 insertions(+), 32 deletions(-)

-- 
2.12.3

[PATCH 1/7] scripts/oe-build-perf-report: fix dumping buildstats

[Markus Lehtonen]

Fix a misbehavior when some of the buildstats are missing.

Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com>
---
 scripts/oe-build-perf-report | 1 +
 1 file changed, 1 insertion(+)

diff --git a/scripts/oe-build-perf-report b/scripts/oe-build-perf-report
index b5ad42bc8a..23081db173 100755
--- a/scripts/oe-build-perf-report
+++ b/scripts/oe-build-perf-report
@@ -422,6 +422,7 @@ def dump_buildstats(repo, outdir, notes_ref, revs):
                                                   'show', tag + '^0']))
             except GitError:
                 log.warning("Buildstats not found for %s", tag)
+                bs_all = {}
                 missing = True
             for measurement, buildstats in bs_all.items():
                 tag_base, run_id = tag.rsplit('/', 1)
-- 
2.12.3

[PATCH 2/7] scripts/oe-build-perf-report: add AggregateTestData class

[Markus Lehtonen]

Making the code a bit more readable.

Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com>
---
 scripts/lib/build_perf/report.py |  5 ++++-
 scripts/oe-build-perf-report     | 16 +++++++++-------
 2 files changed, 13 insertions(+), 8 deletions(-)

diff --git a/scripts/lib/build_perf/report.py b/scripts/lib/build_perf/report.py
index eb00ccca2d..d99a36797f 100644
--- a/scripts/lib/build_perf/report.py
+++ b/scripts/lib/build_perf/report.py
@@ -11,12 +11,15 @@
 # more details.
 #
 """Handling of build perf test reports"""
-from collections import OrderedDict, Mapping
+from collections import OrderedDict, Mapping, namedtuple
 from datetime import datetime, timezone
 from numbers import Number
 from statistics import mean, stdev, variance
 
 
+AggregateTestData = namedtuple('AggregateTestData', ['metadata', 'results'])
+
+
 def isofmt_to_timestamp(string):
     """Convert timestamp string in ISO 8601 format into unix timestamp"""
     if '.' in string:
diff --git a/scripts/oe-build-perf-report b/scripts/oe-build-perf-report
index 23081db173..3a76ab621d 100755
--- a/scripts/oe-build-perf-report
+++ b/scripts/oe-build-perf-report
@@ -29,7 +29,8 @@ sys.path.append(os.path.join(scripts_path, 'lib'))
 import scriptpath
 from build_perf import print_table
 from build_perf.report import (metadata_xml_to_json, results_xml_to_json,
-                               aggregate_data, aggregate_metadata, measurement_stats)
+                               aggregate_data, aggregate_metadata, measurement_stats,
+                               AggregateTestData)
 from build_perf import html
 
 scriptpath.add_oe_lib_path()
@@ -337,13 +338,13 @@ def print_html_report(data, id_comp):
                 'hostname': {'title': 'Hostname', 'value': 'foobar'},
                 'commit': {'title': 'Commit', 'value': '1234'}
                }
-    metadata = metadata_diff(data[id_comp][0], data[-1][0])
+    metadata = metadata_diff(data[id_comp].metadata, data[-1].metadata)
 
 
     # Generate list of tests
     tests = []
-    for test in data[-1][1]['tests'].keys():
-        test_r = data[-1][1]['tests'][test]
+    for test in data[-1].results['tests'].keys():
+        test_r = data[-1].results['tests'][test]
         new_test = {'name': test_r['name'],
                     'description': test_r['description'],
                     'status': test_r['status'],
@@ -576,7 +577,8 @@ def main(argv=None):
 
     data = []
     for raw_m, raw_d in raw_data:
-        data.append((aggregate_metadata(raw_m), aggregate_data(raw_d)))
+        data.append(AggregateTestData(aggregate_metadata(raw_m),
+                                      aggregate_data(raw_d)))
 
     # Re-map list indexes to the new table starting from index 0
     index_r = index_r - index_0
@@ -584,8 +586,8 @@ def main(argv=None):
 
     # Print report
     if not args.html:
-        print_diff_report(data[index_l][0], data[index_l][1],
-                          data[index_r][0], data[index_r][1])
+        print_diff_report(data[index_l].metadata, data[index_l].results,
+                          data[index_r].metadata, data[index_r].results)
     else:
         print_html_report(data, index_l)
 
-- 
2.12.3

[PATCH 3/7] scripts/oe-build-perf-report: fix handling of --history-length

[Markus Lehtonen]

Don't crash if 'left' revision is older than the range of commits
specified with '--history-length'. In this case the 'left' revision
takes precedence.

Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com>
---
 scripts/oe-build-perf-report | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/scripts/oe-build-perf-report b/scripts/oe-build-perf-report
index 3a76ab621d..defd930b04 100755
--- a/scripts/oe-build-perf-report
+++ b/scripts/oe-build-perf-report
@@ -563,7 +563,7 @@ def main(argv=None):
     xml = is_xml_format(repo, revs[index_r].tags[-1])
 
     if args.html:
-        index_0 = max(0, index_r - args.history_length)
+        index_0 = max(0, min(index_l, index_r - args.history_length))
         rev_range = range(index_0, index_r + 1)
     else:
         # We do not need range of commits for text report (no graphs)
-- 
2.12.3

[PATCH 4/7] scripts/oe-build-perf-report: use --hostname in --list

[Markus Lehtonen]

Makes it possible to list test results for certain host only, instead of
always listing all results from all hosts.

Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com>
---
 scripts/oe-build-perf-report | 6 ++++--
 1 file changed, 4 insertions(+), 2 deletions(-)

diff --git a/scripts/oe-build-perf-report b/scripts/oe-build-perf-report
index defd930b04..89d9d6f6b5 100755
--- a/scripts/oe-build-perf-report
+++ b/scripts/oe-build-perf-report
@@ -85,7 +85,9 @@ def get_test_runs(repo, tag_name, **kwargs):
 
 def list_test_revs(repo, tag_name, verbosity, **kwargs):
     """Get list of all tested revisions"""
-    fields, revs = get_test_runs(repo, tag_name, **kwargs)
+    valid_kwargs = dict([(k, v) for k, v in kwargs.items() if v is not None])
+
+    fields, revs = get_test_runs(repo, tag_name, **valid_kwargs)
     ignore_fields = ['tag_number']
     if verbosity < 2:
         extra_fields = ['COMMITS', 'TEST RUNS']
@@ -509,7 +511,7 @@ def main(argv=None):
     repo = GitRepo(args.repo)
 
     if args.list:
-        list_test_revs(repo, args.tag_name, args.list)
+        list_test_revs(repo, args.tag_name, args.list, hostname=args.hostname)
         return 0
 
     # Determine hostname which to use
-- 
2.12.3

[PATCH 5/7] scripts/oe-build-perf-report: remove dead code

[Markus Lehtonen]

Some leftover from an early prototype.

Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com>
---
 scripts/oe-build-perf-report | 4 ----
 1 file changed, 4 deletions(-)

diff --git a/scripts/oe-build-perf-report b/scripts/oe-build-perf-report
index 89d9d6f6b5..4e067caba0 100755
--- a/scripts/oe-build-perf-report
+++ b/scripts/oe-build-perf-report
@@ -336,10 +336,6 @@ def print_diff_report(metadata_l, data_l, metadata_r, data_r):
 def print_html_report(data, id_comp):
     """Print report in html format"""
     # Handle metadata
-    metadata = {'branch': {'title': 'Branch', 'value': 'master'},
-                'hostname': {'title': 'Hostname', 'value': 'foobar'},
-                'commit': {'title': 'Commit', 'value': '1234'}
-               }
     metadata = metadata_diff(data[id_comp].metadata, data[-1].metadata)
 
 
-- 
2.12.3

[PATCH 6/7] scripts/oe-build-perf-report: provide valid title in the html report

[Markus Lehtonen]

Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com>
---
 scripts/oe-build-perf-report | 4 +++-
 1 file changed, 3 insertions(+), 1 deletion(-)

diff --git a/scripts/oe-build-perf-report b/scripts/oe-build-perf-report
index 4e067caba0..8d730cd20a 100755
--- a/scripts/oe-build-perf-report
+++ b/scripts/oe-build-perf-report
@@ -396,7 +396,9 @@ def print_html_report(data, id_comp):
                             'max': get_data_item(data[-1][0], 'layers.meta.commit_count')}
                  }
 
-    print(html.template.render(metadata=metadata, test_data=tests, chart_opts=chart_opts))
+    print(html.template.render(title="Build Perf Test Report",
+                               metadata=metadata, test_data=tests,
+                               chart_opts=chart_opts))
 
 
 def dump_buildstats(repo, outdir, notes_ref, revs):
-- 
2.12.3

[PATCH 7/7] scripts/oe-build-perf-report: tidy up html syntax

[Markus Lehtonen]

Fix some problems in the html syntax of the generated report:
- prevent empty rows in the summary table
- add one missing column in the results table

Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com>
---
 scripts/lib/build_perf/html/report.html | 36 ++++++++++++++++++---------------
 1 file changed, 20 insertions(+), 16 deletions(-)

diff --git a/scripts/lib/build_perf/html/report.html b/scripts/lib/build_perf/html/report.html
index 165cbb811c..f983a4a09c 100644
--- a/scripts/lib/build_perf/html/report.html
+++ b/scripts/lib/build_perf/html/report.html
@@ -118,29 +118,32 @@ h3 {
       {% else %}
         {% set row_style = 'style="background-color: #ffffff"' %}
       {% endif %}
-      <tr {{ row_style }}><td>{{ test.name }}: {{ test.description }}</td>
       {% if test.status == 'SUCCESS' %}
         {% for measurement in test.measurements %}
-          {# add empty cell in place of the test name#}
-          {% if loop.index > 1 %}<td></td>{% endif %}
-          {% if measurement.absdiff > 0 %}
-            {% set result_style = "color: red" %}
-          {% elif measurement.absdiff == measurement.absdiff %}
-            {% set result_style = "color: green" %}
-          {% else %}
-            {% set result_style = "color: orange" %}
-          {%endif %}
-          <td>{{ measurement.description }}</td>
-          <td style="font-weight: bold">{{ measurement.value.mean }}</td>
-          <td style="{{ result_style }}">{{ measurement.absdiff_str }}</td>
-          <td style="{{ result_style }}">{{ measurement.reldiff }}</td>
-          </tr><tr {{ row_style }}>
+          <tr {{ row_style }}>
+            {% if loop.index == 1 %}
+              <td>{{ test.name }}: {{ test.description }}</td>
+            {% else %}
+              {# add empty cell in place of the test name#}
+              <td></td>
+            {% endif %}
+            {% if measurement.absdiff > 0 %}
+              {% set result_style = "color: red" %}
+            {% elif measurement.absdiff == measurement.absdiff %}
+              {% set result_style = "color: green" %}
+            {% else %}
+              {% set result_style = "color: orange" %}
+            {%endif %}
+            <td>{{ measurement.description }}</td>
+            <td style="font-weight: bold">{{ measurement.value.mean }}</td>
+            <td style="{{ result_style }}">{{ measurement.absdiff_str }}</td>
+            <td style="{{ result_style }}">{{ measurement.reldiff }}</td>
+          </tr>
         {% endfor %}
       {% else %}
         <td style="font-weight: bold; color: red;">{{test.status }}</td>
         <td></td> <td></td> <td></td> <td></td>
       {% endif %}
-      </tr>
     {% endfor %}
   </table>
 
@@ -186,6 +189,7 @@ h3 {
                     <th>Stdev</th><td>{{ measurement.value.stdev }}</td>
                   </tr><tr>
                     <th><div id="{{ test.name }}_{{ measurement.name }}_chart_png"></div></th>
+                    <td></td>
                   </tr>
                 </table>
               </td>
-- 
2.12.3