Re: [PATCH v2 0/5] Improvements for performance test report view

Randy MacLeod
2024-05-03T17:22:33Z

[-- Attachment #1: Type: text/plain, Size: 8150 bytes --]

On 2024-05-03 10:43 a.m., Ninette Adhikari wrote:
> This work is done according to "Milestone 9: Build performance test report view" as stated in the Scope of Work with Sovereign Tech Fund (STF) (https://www.sovereigntechfund.de/).
> The current report can be accessed here:
> Performance test report HTML (https://autobuilder.yocto.io/pub/non-release/20240117-15/testresults/buildperf-alma8/perf-alma8_master_20240117090048_663f180574.html)
> The report is created using the `oe-build-perf-report` script in the poky repository. This script generates a performance test report in HTML format using the data from the yocto-buildstats (https://git.yoctoproject.org/yocto-buildstats/) repository.
> The report displays the performance test results in line chart format. The chart x-axis represents the commit numbers, and the y-axis represents the test duration in minutes.
> The report also includes a table that displays the measurement statistics data for each test. The report is interactive and allows users to zoom in on specific sections of the line chart.
>
> The current report format required some updates to make it more interactive and user-friendly. And this patch addresses such improvements:
>
> - Add [Apache echart](https://echarts.apache.org/en/index.html) library to create oe build performance report charts and make them interactive.
> - Restructure data to time and value array format to be used by echarts. It also converts test duration to minutes and adds zoom to the charts.
> - Update measurement statistics data to include `start_time` so that time can be displayed instead of commit numbers on the chart. It also updates default commit history length to 300.
> - Add styling updates including page margin, labels for x and y axis, tooltip, and section descriptions.
The tooltips are very nice and useful so
> - The charts are created as step charts instead of plain line charts.
> - Add start time, size, and commit number to tooltip.
> - Add dark mode view
>
> Updated report screenshots:
> https://github.com/neighbourhoodie/poky/assets/13760198/65a1890c-fd2a-40d4-ac90-f13055735e53
> https://github.com/neighbourhoodie/poky/assets/13760198/b40c326b-17d2-42e2-8e41-72e52ed2c003
> https://github.com/neighbourhoodie/poky/assets/13760198/cc7ec996-9dab-435a-8fdc-82a2a4193c0a
> https://github.com/neighbourhoodie/poky/assets/13760198/6e0fe09d-50e5-4b0b-b70b-6943f71b5208

Very nice! Thanks for the work and the v2 improvements.

I applied the patches by saving them from email and there was a minor
whitespace warning which
is really nothing to worry about but if you want to avoid such noise,
you could send yourself
patches by email and then apply them in a different branch if you want
to check for such problems.

❯ git am /tmp/ninette-v2/*
Applying: oe-build-perf-report: Add apache echarts to make report
interactive
Applying: oe-build-perf-report: Display more than 300 commits and date
instead of commit number
Applying: oe-build-perf-report: Improve report styling and add descriptions
.git/rebase-apply/patch:240: trailing whitespace.
start_time = time
warning: 1 line adds whitespace errors.
Applying: oe-build-perf-report: Update chart tooltip and chart type
Applying: oe-build-perf-report: Add dark mode

poky.git on ninette-v2 [$?]
❯ ls /tmp/ninette-v2/*
'/tmp/ninette-v2/[OE-core] [PATCH v2 1_5] oe-build-perf-report: Add
apache echarts to make report interactive - "Ninette Adhikari via
lists.openembedded.org"
<ninette=thehoodiefirm.com@lists.openembedded.org> - 2024-05-03 1043.eml'
'/tmp/ninette-v2/[PATCH v2 2_5] oe-build-perf-report: Display more than
300 commits and date instead of commit number - Ninette Adhikari
<ninette@thehoodiefirm.com> - 2024-05-03 1043.eml'
'/tmp/ninette-v2/[PATCH v2 3_5] oe-build-perf-report: Improve report
styling and add descriptions - Ninette Adhikari
<ninette@thehoodiefirm.com> - 2024-05-03 1043.eml'
'/tmp/ninette-v2/[PATCH v2 4_5] oe-build-perf-report: Update chart
tooltip and chart type - Ninette Adhikari <ninette@thehoodiefirm.com> -
2024-05-03 1043.eml'
'/tmp/ninette-v2/[PATCH v2 5_5] oe-build-perf-report: Add dark mode -
Ninette Adhikari <ninette@thehoodiefirm.com> - 2024-05-03 1043.eml'

>
> For local setup, you can do the following:
>
> 1. Clone the yocto-buildstats (https://git.yoctoproject.org/yocto-buildstats/) and the poky repository (https://git.yoctoproject.org/poky/)
>
> 2. In the poky repository run the following to build the report HTML:
> ```bash
> ./scripts/oe-build-perf-report -r "LOCAL_PATH_TO_YOCTO_BUILDSTATS" --branch "master" --commit "663f1805742ff6fb6955719d0ab7846a425debcf" --branch2 "master" --html > test.html

These are the same setup step as last time I think and again, they
aren't quite right but
the script tells you do do:

❯ git fetch origin
refs/notes/buildstats/perf-debian11/master/qemux86:refs/notes/buildstats/perf-debian11/master/qemux86

and that works.

> ```
> Note:
> - Add your local path to the yocto-buildstats repo
> - The above command builds the report in a file called `test.html`. You can access it in the root directory in poky.
> - This exmaple report uses the commit `663f1805742ff6fb6955719d0ab7846a425debcf` from `master` branch.

Hmmm, ah, that's a poky commit id. I (stupidly!) expected it to be a
commit ID in the yocto-buildstats repo.

One thing that you can think about fixing or just ignore is that the
--commit arg needs to be the
full commit hash not the shortened version. i.e. this doesn't work:

❯ ./scripts/oe-build-perf-report -r
"/media/rmacleod/gitter/rmacleod/src/distro/yocto/yocto-buildstats.git"
--branch "master" --commit "663f180574" --branch2 "master" --html >
/tmp/ninette-v2-test-663f180574.html
Traceback (most recent call last):
File
"/media/rmacleod/gitter/rmacleod/src/distro/yocto/poky.git/./scripts/oe-build-perf-report",
line 617, in <module>
sys.exit(main())
File
"/media/rmacleod/gitter/rmacleod/src/distro/yocto/poky.git/./scripts/oe-build-perf-report",
line 532, in main
index1 = gitarchive.rev_find(revs, 'commit', args.commit)
File
"/media/rmacleod/gitter/rmacleod/src/distro/yocto/poky.git/meta/lib/oeqa/utils/gitarchive.py",
line 282, in rev_find
raise ValueError("Unable to find '{}' value '{}'".format(attr, val))
ValueError: Unable to find 'commit' value '663f180574'

Since the script is going to be used in other scripts run on the
autobuilder, this is a low priority enhancement.

I was happy to see that a newer poky commit, worked fine:

❯ ./scripts/oe-build-perf-report -r
"/media/rmacleod/gitter/rmacleod/src/distro/yocto/yocto-buildstats.git"
--branch "master" --commit "632e3170595bb32717c0471a55a619b4b33fe787"
--branch2 "master" --html >
/tmp/ninette-v2-test-632e3170595bb32717c0471a55a619b4b33fe787.html
INFO: Parsing buildstats from
'refs/notes/buildstats/perf-debian11/master/qemux86'

but I did notice:  Stdev: nan

whereas in your linked charts, and in my first run with commit:
663f1805742ff6fb6955719d0ab7846a425debcf

the correct numerical standard deviation is displayed.

I can debug this is you can't reproduce it yourself.

Again. these reports and beautiful interactive charts are great to see.

Thanks!

../Randy

>
>
> Ninette Adhikari (5):
> oe-build-perf-report: Add apache echarts to make report interactive
> oe-build-perf-report: Display more than 300 commits and date instead
> of commit number
> oe-build-perf-report: Improve report styling and add descriptions
> oe-build-perf-report: Update chart tooltip and chart type
> oe-build-perf-report: Add dark mode
>
> .../build_perf/html/measurement_chart.html | 140 ++++++++++++------
> scripts/lib/build_perf/html/report.html | 124 +++++++++++-----
> scripts/lib/build_perf/report.py | 5 +-
> scripts/oe-build-perf-report | 6 +-
> 4 files changed, 193 insertions(+), 82 deletions(-)
>

--
# Randy MacLeod
# Wind River Linux

[-- Attachment #2.1: Type: text/html, Size: 12105 bytes --]

[-- Attachment #2.2: ipZPodcPGxdg4E4H.png --]
[-- Type: image/png, Size: 65076 bytes --]