Re: [OE-core] [PATCH v2 0/5] Improvements for performance test report view

[Randy MacLeod <randy.macleod@windriver.com>]

[-- Attachment #1: Type: text/plain, Size: 9686 bytes --]

On 2024-05-03 1:22 p.m., Randy MacLeod via lists.openembedded.org wrote:
> On 2024-05-03 10:43 a.m., Ninette Adhikari wrote:
>> This work is done according to "Milestone 9: Build performance test report view" as stated in the Scope of Work with Sovereign Tech Fund (STF) (https://www.sovereigntechfund.de/).
>> The current report can be accessed here:
>> Performance test report HTML (https://autobuilder.yocto.io/pub/non-release/20240117-15/testresults/buildperf-alma8/perf-alma8_master_20240117090048_663f180574.html)
>> The report is created using the `oe-build-perf-report` script in the poky repository. This script generates a performance test report in HTML format using the data from the yocto-buildstats (https://git.yoctoproject.org/yocto-buildstats/) repository.
>> The report displays the performance test results in line chart format. The chart x-axis represents the commit numbers, and the y-axis represents the test duration in minutes.
>> The report also includes a table that displays the measurement statistics data for each test. The report is interactive and allows users to zoom in on specific sections of the line chart.
>>
>> The current report format required some updates to make it more interactive and user-friendly. And this patch addresses such improvements:
>>
>> - Add [Apache echart](https://echarts.apache.org/en/index.html) library to create oe build performance report charts and make them interactive.
>> - Restructure data to time and value array format to be used by echarts. It also converts test duration to minutes and adds zoom to the charts.
>> - Update measurement statistics data to include `start_time` so that time can be displayed instead of commit numbers on the chart. It also updates default commit history length to 300.
>> - Add styling updates including page margin, labels for x and y axis, tooltip, and section descriptions.
> The tooltips are very nice and useful so
>> - The charts are created as step charts instead of plain line charts.
>> - Add start time, size, and commit number to tooltip.
>> - Add dark mode view
>>
>> Updated report screenshots:
>> https://github.com/neighbourhoodie/poky/assets/13760198/65a1890c-fd2a-40d4-ac90-f13055735e53
>> https://github.com/neighbourhoodie/poky/assets/13760198/b40c326b-17d2-42e2-8e41-72e52ed2c003
>> https://github.com/neighbourhoodie/poky/assets/13760198/cc7ec996-9dab-435a-8fdc-82a2a4193c0a
>> https://github.com/neighbourhoodie/poky/assets/13760198/6e0fe09d-50e5-4b0b-b70b-6943f71b5208
>
>
> Very nice! Thanks for the work and the v2 improvements.
>
>
> I applied the patches by saving them from email and there was a minor 
> whitespace warning which
> is really nothing to worry about but if you want to avoid such noise, 
> you could send yourself
> patches by email and then apply them in a different branch if you want 
> to check for such problems.
>
>
> ❯ git am /tmp/ninette-v2/*
> Applying: oe-build-perf-report: Add apache echarts to make report 
> interactive
> Applying: oe-build-perf-report: Display more than 300 commits and date 
> instead of commit number
> Applying: oe-build-perf-report: Improve report styling and add 
> descriptions
> .git/rebase-apply/patch:240: trailing whitespace.
>     start_time = time
> warning: 1 line adds whitespace errors.
> Applying: oe-build-perf-report: Update chart tooltip and chart type
> Applying: oe-build-perf-report: Add dark mode
>
> poky.git on ninette-v2 [$?]
> ❯ ls /tmp/ninette-v2/*
> '/tmp/ninette-v2/[OE-core] [PATCH v2 1_5] oe-build-perf-report: Add 
> apache echarts to make report interactive - "Ninette Adhikari via 
> lists.openembedded.org" 
> <ninette=thehoodiefirm.com@lists.openembedded.org> - 2024-05-03 1043.eml'
> '/tmp/ninette-v2/[PATCH v2 2_5] oe-build-perf-report: Display more 
> than 300 commits and date instead of commit number - Ninette Adhikari 
> <ninette@thehoodiefirm.com> - 2024-05-03 1043.eml'
> '/tmp/ninette-v2/[PATCH v2 3_5] oe-build-perf-report: Improve report 
> styling and add descriptions - Ninette Adhikari 
> <ninette@thehoodiefirm.com> - 2024-05-03 1043.eml'
> '/tmp/ninette-v2/[PATCH v2 4_5] oe-build-perf-report: Update chart 
> tooltip and chart type - Ninette Adhikari <ninette@thehoodiefirm.com> 
> - 2024-05-03 1043.eml'
> '/tmp/ninette-v2/[PATCH v2 5_5] oe-build-perf-report: Add dark mode - 
> Ninette Adhikari <ninette@thehoodiefirm.com> - 2024-05-03 1043.eml'
>
>> For local setup, you can do the following:
>>
>> 1. Clone the yocto-buildstats (https://git.yoctoproject.org/yocto-buildstats/) and the poky repository (https://git.yoctoproject.org/poky/)
>>
>> 2. In the poky repository run the following to build the report HTML:
>> ```bash
>> ./scripts/oe-build-perf-report -r "LOCAL_PATH_TO_YOCTO_BUILDSTATS" --branch "master" --commit "663f1805742ff6fb6955719d0ab7846a425debcf" --branch2 "master" --html > test.html
>
> These are the same setup step as last time I think and again, they 
> aren't quite right but
> the script tells you do do:
>
> ❯ git fetch origin 
> refs/notes/buildstats/perf-debian11/master/qemux86:refs/notes/buildstats/perf-debian11/master/qemux86
>
>
> and that works.
>
>
>> ```
>> Note:
>> - Add your local path to the yocto-buildstats repo
>> - The above command builds the report in a file called `test.html`. You can access it in the root directory in poky.
>> - This exmaple report uses the commit `663f1805742ff6fb6955719d0ab7846a425debcf` from `master` branch.
>
> Hmmm, ah, that's a poky commit id. I (stupidly!) expected it to be a 
> commit ID in the yocto-buildstats repo.
>
>
> One thing that you can think about fixing or just ignore is that the 
> --commit arg needs to be the
> full commit hash not the shortened version. i.e. this doesn't work:
>
> ❯ ./scripts/oe-build-perf-report -r 
> "/media/rmacleod/gitter/rmacleod/src/distro/yocto/yocto-buildstats.git" 
> --branch "master" --commit "663f180574" --branch2 "master" --html > 
> /tmp/ninette-v2-test-663f180574.html
> Traceback (most recent call last):
>   File 
> "/media/rmacleod/gitter/rmacleod/src/distro/yocto/poky.git/./scripts/oe-build-perf-report", 
> line 617, in <module>
>     sys.exit(main())
>   File 
> "/media/rmacleod/gitter/rmacleod/src/distro/yocto/poky.git/./scripts/oe-build-perf-report", 
> line 532, in main
>     index1 = gitarchive.rev_find(revs, 'commit', args.commit)
>   File 
> "/media/rmacleod/gitter/rmacleod/src/distro/yocto/poky.git/meta/lib/oeqa/utils/gitarchive.py", 
> line 282, in rev_find
>     raise ValueError("Unable to find '{}' value '{}'".format(attr, val))
> ValueError: Unable to find 'commit' value '663f180574'
>
> Since the script is going to be used in other scripts run on the 
> autobuilder, this is a low priority enhancement.
>
>
> I was happy to see that a newer poky commit, worked fine:
>
> ❯ ./scripts/oe-build-perf-report -r 
> "/media/rmacleod/gitter/rmacleod/src/distro/yocto/yocto-buildstats.git" 
> --branch "master" --commit "632e3170595bb32717c0471a55a619b4b33fe787" 
> --branch2 "master" --html > 
> /tmp/ninette-v2-test-632e3170595bb32717c0471a55a619b4b33fe787.html
> INFO: Parsing buildstats from 
> 'refs/notes/buildstats/perf-debian11/master/qemux86'
>
>
> but I did notice:  Stdev: nan
>
>
> whereas in your linked charts, and in my first run with commit: 
> 663f1805742ff6fb6955719d0ab7846a425debcf
>
> the correct numerical standard deviation is displayed.
>
> I can debug this is you can't reproduce it yourself.
>
* if you can't  ...


Also, I wanted to draw your attention to how the vertical dashed line 
(VDL) behaves
to see if it's something that can (or should) be changed.

If you zoom in on a graph the VDL is placed at the left edge of bar 
until the pointer passes the mid-point of the bar:


at which time the VDL moves to the next bar/data point:



It's a small thing but it would be nice if:
   1) the VDL was placed in the centre of the bar and
   2) the commit number /tooltip data only changed when the pointer move 
to a different bar
       so the same data would be displayed as one swept across a given bar.

What do you think?


Picky, eh! ;-)


That's all for now.

../Randy


>
> Again. these reports and beautiful interactive charts are great to see.
>
> Thanks!
>
> ../Randy
>
>
>> Ninette Adhikari (5):
>>    oe-build-perf-report: Add apache echarts to make report interactive
>>    oe-build-perf-report: Display more than 300 commits and date instead
>>      of commit number
>>    oe-build-perf-report: Improve report styling and add descriptions
>>    oe-build-perf-report: Update chart tooltip and chart type
>>    oe-build-perf-report: Add dark mode
>>
>>   .../build_perf/html/measurement_chart.html    | 140 ++++++++++++------
>>   scripts/lib/build_perf/html/report.html       | 124 +++++++++++-----
>>   scripts/lib/build_perf/report.py              |   5 +-
>>   scripts/oe-build-perf-report                  |   6 +-
>>   4 files changed, 193 insertions(+), 82 deletions(-)
>>
>
> -- 
> # Randy MacLeod
> # Wind River Linux
>
> -=-=-=-=-=-=-=-=-=-=-=-
> Links: You receive all messages sent to this group.
> View/Reply Online (#199007):https://lists.openembedded.org/g/openembedded-core/message/199007
> Mute This Topic:https://lists.openembedded.org/mt/105889450/3616765
> Group Owner:openembedded-core+owner@lists.openembedded.org
> Unsubscribe:https://lists.openembedded.org/g/openembedded-core/unsub  [randy.macleod@windriver.com]
> -=-=-=-=-=-=-=-=-=-=-=-
>

-- 
# Randy MacLeod
# Wind River Linux

[-- Attachment #2.1: Type: text/html, Size: 15219 bytes --]

[-- Attachment #2.2: ipZPodcPGxdg4E4H.png --]
[-- Type: image/png, Size: 65076 bytes --]

[-- Attachment #2.3: EVQkTyvIsYDJlcL7.png --]
[-- Type: image/png, Size: 2280301 bytes --]

[-- Attachment #2.4: ceOAY8Zii7KShL9e.png --]
[-- Type: image/png, Size: 2068675 bytes --]