On 2024-05-03 10:43 a.m., Ninette Adhikari wrote: > This work is done according to "Milestone 9: Build performance test report view" as stated in the Scope of Work with Sovereign Tech Fund (STF) (https://www.sovereigntechfund.de/). > The current report can be accessed here: > Performance test report HTML (https://autobuilder.yocto.io/pub/non-release/20240117-15/testresults/buildperf-alma8/perf-alma8_master_20240117090048_663f180574.html) > The report is created using the `oe-build-perf-report` script in the poky repository. This script generates a performance test report in HTML format using the data from the yocto-buildstats (https://git.yoctoproject.org/yocto-buildstats/) repository. > The report displays the performance test results in line chart format. The chart x-axis represents the commit numbers, and the y-axis represents the test duration in minutes. > The report also includes a table that displays the measurement statistics data for each test. The report is interactive and allows users to zoom in on specific sections of the line chart. > > The current report format required some updates to make it more interactive and user-friendly. And this patch addresses such improvements: > > - Add [Apache echart](https://echarts.apache.org/en/index.html) library to create oe build performance report charts and make them interactive. > - Restructure data to time and value array format to be used by echarts. It also converts test duration to minutes and adds zoom to the charts. > - Update measurement statistics data to include `start_time` so that time can be displayed instead of commit numbers on the chart. It also updates default commit history length to 300. > - Add styling updates including page margin, labels for x and y axis, tooltip, and section descriptions. The tooltips are very nice and useful so > - The charts are created as step charts instead of plain line charts. > - Add start time, size, and commit number to tooltip. > - Add dark mode view > > Updated report screenshots: > https://github.com/neighbourhoodie/poky/assets/13760198/65a1890c-fd2a-40d4-ac90-f13055735e53 > https://github.com/neighbourhoodie/poky/assets/13760198/b40c326b-17d2-42e2-8e41-72e52ed2c003 > https://github.com/neighbourhoodie/poky/assets/13760198/cc7ec996-9dab-435a-8fdc-82a2a4193c0a > https://github.com/neighbourhoodie/poky/assets/13760198/6e0fe09d-50e5-4b0b-b70b-6943f71b5208 Very nice! Thanks for the work and the v2 improvements. I applied the patches by saving them from email and there was a minor whitespace warning which is really nothing to worry about but if you want to avoid such noise, you could send yourself patches by email and then apply them in a different branch if you want to check for such problems. ❯ git am /tmp/ninette-v2/* Applying: oe-build-perf-report: Add apache echarts to make report interactive Applying: oe-build-perf-report: Display more than 300 commits and date instead of commit number Applying: oe-build-perf-report: Improve report styling and add descriptions .git/rebase-apply/patch:240: trailing whitespace.     start_time = time warning: 1 line adds whitespace errors. Applying: oe-build-perf-report: Update chart tooltip and chart type Applying: oe-build-perf-report: Add dark mode poky.git on ninette-v2 [$?] ❯ ls /tmp/ninette-v2/* '/tmp/ninette-v2/[OE-core] [PATCH v2 1_5] oe-build-perf-report: Add apache echarts to make report interactive - "Ninette Adhikari via lists.openembedded.org" - 2024-05-03 1043.eml' '/tmp/ninette-v2/[PATCH v2 2_5] oe-build-perf-report: Display more than 300 commits and date instead of commit number - Ninette Adhikari - 2024-05-03 1043.eml' '/tmp/ninette-v2/[PATCH v2 3_5] oe-build-perf-report: Improve report styling and add descriptions - Ninette Adhikari - 2024-05-03 1043.eml' '/tmp/ninette-v2/[PATCH v2 4_5] oe-build-perf-report: Update chart tooltip and chart type - Ninette Adhikari - 2024-05-03 1043.eml' '/tmp/ninette-v2/[PATCH v2 5_5] oe-build-perf-report: Add dark mode - Ninette Adhikari - 2024-05-03 1043.eml' > > For local setup, you can do the following: > > 1. Clone the yocto-buildstats (https://git.yoctoproject.org/yocto-buildstats/) and the poky repository (https://git.yoctoproject.org/poky/) > > 2. In the poky repository run the following to build the report HTML: > ```bash > ./scripts/oe-build-perf-report -r "LOCAL_PATH_TO_YOCTO_BUILDSTATS" --branch "master" --commit "663f1805742ff6fb6955719d0ab7846a425debcf" --branch2 "master" --html > test.html These are the same setup step as last time I think and again, they aren't quite right but the script tells you do do: ❯ git fetch origin refs/notes/buildstats/perf-debian11/master/qemux86:refs/notes/buildstats/perf-debian11/master/qemux86 and that works. > ``` > Note: > - Add your local path to the yocto-buildstats repo > - The above command builds the report in a file called `test.html`. You can access it in the root directory in poky. > - This exmaple report uses the commit `663f1805742ff6fb6955719d0ab7846a425debcf` from `master` branch. Hmmm, ah, that's a poky commit id. I (stupidly!) expected it to be a commit ID in the yocto-buildstats repo. One thing that you can think about fixing or just ignore is that the --commit arg needs to be the full commit hash not the shortened version. i.e. this doesn't work: ❯ ./scripts/oe-build-perf-report -r "/media/rmacleod/gitter/rmacleod/src/distro/yocto/yocto-buildstats.git" --branch "master" --commit "663f180574" --branch2 "master" --html > /tmp/ninette-v2-test-663f180574.html Traceback (most recent call last):   File "/media/rmacleod/gitter/rmacleod/src/distro/yocto/poky.git/./scripts/oe-build-perf-report", line 617, in     sys.exit(main())   File "/media/rmacleod/gitter/rmacleod/src/distro/yocto/poky.git/./scripts/oe-build-perf-report", line 532, in main     index1 = gitarchive.rev_find(revs, 'commit', args.commit)   File "/media/rmacleod/gitter/rmacleod/src/distro/yocto/poky.git/meta/lib/oeqa/utils/gitarchive.py", line 282, in rev_find     raise ValueError("Unable to find '{}' value '{}'".format(attr, val)) ValueError: Unable to find 'commit' value '663f180574' Since the script is going to be used in other scripts run on the autobuilder, this is a low priority enhancement. I was happy to see that a newer poky commit, worked fine: ❯ ./scripts/oe-build-perf-report -r "/media/rmacleod/gitter/rmacleod/src/distro/yocto/yocto-buildstats.git" --branch "master" --commit "632e3170595bb32717c0471a55a619b4b33fe787" --branch2 "master" --html > /tmp/ninette-v2-test-632e3170595bb32717c0471a55a619b4b33fe787.html INFO: Parsing buildstats from 'refs/notes/buildstats/perf-debian11/master/qemux86' but I did notice:  Stdev: nan whereas in your linked charts, and in my first run with commit: 663f1805742ff6fb6955719d0ab7846a425debcf the correct numerical standard deviation is displayed. I can debug this is you can't reproduce it yourself. Again. these reports and beautiful interactive charts are great to see. Thanks! ../Randy > > > Ninette Adhikari (5): > oe-build-perf-report: Add apache echarts to make report interactive > oe-build-perf-report: Display more than 300 commits and date instead > of commit number > oe-build-perf-report: Improve report styling and add descriptions > oe-build-perf-report: Update chart tooltip and chart type > oe-build-perf-report: Add dark mode > > .../build_perf/html/measurement_chart.html | 140 ++++++++++++------ > scripts/lib/build_perf/html/report.html | 124 +++++++++++----- > scripts/lib/build_perf/report.py | 5 +- > scripts/oe-build-perf-report | 6 +- > 4 files changed, 193 insertions(+), 82 deletions(-) > -- # Randy MacLeod # Wind River Linux