Re: [OE-core] [PATCH 0/3] Improvements for performance test report view

[Randy MacLeod <randy.macleod@windriver.com>]

[-- Attachment #1: Type: text/plain, Size: 6002 bytes --]

On 2024-04-15 10:41 a.m., Ninette Adhikari via lists.openembedded.org wrote:
> This work is done according to "Milestone 9: Build performance test report view" as stated in the Scope of Work with Sovereign Tech Fund (STF) (https://www.sovereigntechfund.de/).
> The current report can be accessed here:
> Performance test report HTML (https://autobuilder.yocto.io/pub/non-release/20240117-15/testresults/buildperf-alma8/perf-alma8_master_20240117090048_663f180574.html)
> The report is created using the `oe-build-perf-report` script in the poky repository. This script generates a performance test report in HTML format using the data from the yocto-buildstats (https://git.yoctoproject.org/yocto-buildstats/) repository.
> The report displays the performance test results in line chart format. The chart x-axis represents the commit numbers, and the y-axis represents the test duration in minutes.
> The report also includes a table that displays the measurement statistics data for each test. The report is interactive and allows users to zoom in on specific sections of the line chart.
>
> The current report format required some updates to make it more interactive and user-friendly. And this patch addresses such improvements:
>
> - Add Apache echart (https://echarts.apache.org/en/index.html) library to create oe build performance report charts and make them interactive.
> - Restructure data to time and value array format to be used by echarts. It also converts test duration to minutes and adds zoom to the line charts.
> - Update measurement statistics data to include `start_time` so that time can be displayed instead of commit numbers on the chart. It also updates default commit history length to 300.
> - Add styling updates including page margin, labels for x and y axis, tooltip, and section descriptions.
>
> Updated report screenshots:
> https://github.com/neighbourhoodie/poky/assets/13760198/65a1890c-fd2a-40d4-ac90-f13055735e53
> https://github.com/neighbourhoodie/poky/assets/13760198/1ed43876-73a9-487e-aed3-ca0edf97514c

Thanks for the nice, dynamic UI web graphs!
Overall, I find them useful and well-designed.


Would it be possible to present the data in a more discrete manner since
the interpolated view presented now, while nice to look at, does not always
reflect the underlying data trend?

For example, if I have this data plotted as 1) stair-steps or 2) line 
segments (I didn't take the time to do a spline fit),

I find that the line style for 1) is usually less misleading about what 
we really know about the data.


1) stair-step



2) straight lines:



and I'd like to see this data:


plotted in the stair-step style.

From:

http://autobuilder.yocto.io/pub/non-release/20240413-19/testresults/buildperf-debian11/perf-debian11_master-next_20240413064005_7116cd908d.html

> For local setup, you can do the following:
>
> 1. Clone the yocto-buildstats (https://git.yoctoproject.org/yocto-buildstats/) and the poky repository (https://git.yoctoproject.org/poky/)
>
> 2. In the poky repository run the following to build the report HTML:
> ```bash
> ./scripts/oe-build-perf-report -r "LOCAL_PATH_TO_YOCTO_BUILDSTATS" --branch "master" --commit "663f1805742ff6fb6955719d0ab7846a425debcf" --branch2 "master" --html > test.html
> ```
> Note:
> - Add your local path to the yocto-buildstats repo
> - The above command builds the report in a file called `test.html`. You can access it in the root directory in poky.
> - This exmaple report uses the commit `663f1805742ff6fb6955719d0ab7846a425debcf` from `master` branch.

I tried that out:

❯ cd .../distro/yocto/yocto-bs-html

❯ ../poky.git/scripts/oe-build-perf-report -r 
"/media/rmacleod/gitter/rmacleod/src/distro/yocto/yocto-bs-html/../yocto-buildstats.git" 
--branch "master" --commit "663f1805742ff6fb6955719d0ab7846a425debcf" 
--branch2 "master" --html > test.html
ERROR: No buildstats found, please try running 'git fetch origin 
refs/notes/buildstats/perf-alma8/master/qemux86:refs/notes/buildstats/perf-alma8/master/qemux86' 
to fetch them from the remote

so I did that.  It was > 3 GB of data...  !!

Receiving objects:  89% (10568/11747), 2.03 GiB | 3.28 MiB/s6 MiB/s

❯ du -sh .
3.2G    .

Also, one shouldn't have to call bash just to run your script since you 
shell
does that already if you have the right #!/path/to/foo and you have made 
the script executable:

❯ file ../poky.git/scripts/oe-build-perf-report
../poky.git/scripts/oe-build-perf-report: Python script, ASCII text 
executable

❯ head -1 ../poky.git/scripts/oe-build-perf-report
#!/usr/bin/env python3


Anyway, that did generate a test.html file in ~ 10 seconds for me on my 
Pop!OS (Ubuntu-based) distro
and it rendered nicely in Firefox 124.0.1 (64-bit) so that's nice to see.


Thanks again,

../Randy



>
>
> Ninette Adhikari (3):
>    oe-build-perf-report: Add apache echarts to make report interactive
>    oe-build-perf-report: Display more than 300 commits and date instead
>      of commit number
>    oe-build-perf-report: Improve report styling and add descriptions
>
>   .../build_perf/html/measurement_chart.html    | 116 +++++++++++-------
>   scripts/lib/build_perf/html/report.html       |  96 ++++++++++-----
>   scripts/lib/build_perf/report.py              |   4 +-
>   scripts/oe-build-perf-report                  |   6 +-
>   4 files changed, 143 insertions(+), 79 deletions(-)
>
>
> -=-=-=-=-=-=-=-=-=-=-=-
> Links: You receive all messages sent to this group.
> View/Reply Online (#198239):https://lists.openembedded.org/g/openembedded-core/message/198239
> Mute This Topic:https://lists.openembedded.org/mt/105537213/3616765
> Group Owner:openembedded-core+owner@lists.openembedded.org
> Unsubscribe:https://lists.openembedded.org/g/openembedded-core/unsub  [randy.macleod@windriver.com]
> -=-=-=-=-=-=-=-=-=-=-=-
>

-- 
# Randy MacLeod
# Wind River Linux

[-- Attachment #2.1: Type: text/html, Size: 9421 bytes --]

[-- Attachment #2.2: C0NlrGL0XCzOLWe5.png --]
[-- Type: image/png, Size: 11077 bytes --]

[-- Attachment #2.3: PK30ZAoDLv3EqTSr.png --]
[-- Type: image/png, Size: 12782 bytes --]

[-- Attachment #2.4: YdeAn9rTv39nGigR.png --]
[-- Type: image/png, Size: 54532 bytes --]