On 2024-05-07 10:17 a.m., Ninette Adhikari wrote: > Hi Randy, > > Thank you so much for the detailed feedback. This is very helpful:) > Here are my comments: > > * whitespace warning: Thanks for the catch. I'll make sure to check > the patches next time to make sure there are no warnings. > * local setup: You are right this can be updated. Do I need to > update this in the cover letter and resend it? Let me know, then I > can also include the update on --commit argument. > Since this is only in the cover letter and not in a commit and, since I was able to make use of the work and document how I did that on the email thread, so I won't insist on a v3. > > * stdev as nan: I was able to reproduce the issue and I think this > is where the nan error is being generated. > https://git.yoctoproject.org/poky/tree/scripts/lib/build_perf/report.py#n331 > . > This error persists in the current master branch. Unfortunately > I'm not exactly sure on the logic of how this is generated and was > not able to resolve this issue. If you have any suggestions on how > to proceed do let me know. > I'm not sure off-hand. Would you open a defect in bugzilla.yoctoproject.org to record the problem and what you know about it? > > * vertical dashed line (VDL) behavior: I tried to troubleshoot this > but it seems like Apache echarts tooltip > > is only triggered when there is a data point in the x axis. And in > your example the data points are on the edges of the bar. Some of > the charts do have continuous data points and hence seems > smoother. Hope this is not too much of a holdup. > Well, now that I've seen it, I can't unsee it and my OCD is triggered! ;-) I don't think it's a big problem for anyone else so let's not worry about it unless you'd like to report it in the YP bugzilla as an enhancement. Thanks again, ../Randy > * tooltip.gif > > Let me know if you have any other questions. Thanks again! > > Ninette > > On Fri, May 3, 2024 at 8:10 PM Randy MacLeod > wrote: > > On 2024-05-03 1:22 p.m., Randy MacLeod via lists.openembedded.org > > wrote: >> On 2024-05-03 10:43 a.m., Ninette Adhikari wrote: >>> This work is done according to "Milestone 9: Build performance test report view" as stated in the Scope of Work with Sovereign Tech Fund (STF) (https://www.sovereigntechfund.de/). >>> The current report can be accessed here: >>> Performance test report HTML (https://autobuilder.yocto.io/pub/non-release/20240117-15/testresults/buildperf-alma8/perf-alma8_master_20240117090048_663f180574.html) >>> The report is created using the `oe-build-perf-report` script in the poky repository. This script generates a performance test report in HTML format using the data from the yocto-buildstats (https://git.yoctoproject.org/yocto-buildstats/) repository. >>> The report displays the performance test results in line chart format. The chart x-axis represents the commit numbers, and the y-axis represents the test duration in minutes. >>> The report also includes a table that displays the measurement statistics data for each test. The report is interactive and allows users to zoom in on specific sections of the line chart. >>> >>> The current report format required some updates to make it more interactive and user-friendly. And this patch addresses such improvements: >>> >>> - Add [Apache echart](https://echarts.apache.org/en/index.html) library to create oe build performance report charts and make them interactive. >>> - Restructure data to time and value array format to be used by echarts. It also converts test duration to minutes and adds zoom to the charts. >>> - Update measurement statistics data to include `start_time` so that time can be displayed instead of commit numbers on the chart. It also updates default commit history length to 300. >>> - Add styling updates including page margin, labels for x and y axis, tooltip, and section descriptions. >> The tooltips are very nice and useful so >>> - The charts are created as step charts instead of plain line charts. >>> - Add start time, size, and commit number to tooltip. >>> - Add dark mode view >>> >>> Updated report screenshots: >>> https://github.com/neighbourhoodie/poky/assets/13760198/65a1890c-fd2a-40d4-ac90-f13055735e53 >>> https://github.com/neighbourhoodie/poky/assets/13760198/b40c326b-17d2-42e2-8e41-72e52ed2c003 >>> https://github.com/neighbourhoodie/poky/assets/13760198/cc7ec996-9dab-435a-8fdc-82a2a4193c0a >>> https://github.com/neighbourhoodie/poky/assets/13760198/6e0fe09d-50e5-4b0b-b70b-6943f71b5208 >> >> >> Very nice! Thanks for the work and the v2 improvements. >> >> >> I applied the patches by saving them from email and there was a >> minor whitespace warning which >> is really nothing to worry about but if you want to avoid such >> noise, you could send yourself >> patches by email and then apply them in a different branch if you >> want to check for such problems. >> >> >> ❯ git am /tmp/ninette-v2/* >> Applying: oe-build-perf-report: Add apache echarts to make report >> interactive >> Applying: oe-build-perf-report: Display more than 300 commits and >> date instead of commit number >> Applying: oe-build-perf-report: Improve report styling and add >> descriptions >> .git/rebase-apply/patch:240: trailing whitespace. >>     start_time = time >> warning: 1 line adds whitespace errors. >> Applying: oe-build-perf-report: Update chart tooltip and chart type >> Applying: oe-build-perf-report: Add dark mode >> >> poky.git on ninette-v2 [$?] >> ❯ ls /tmp/ninette-v2/* >> '/tmp/ninette-v2/[OE-core] [PATCH v2 1_5] oe-build-perf-report: >> Add apache echarts to make report interactive - "Ninette Adhikari >> via lists.openembedded.org >> " >> >> - >> 2024-05-03 1043.eml' >> '/tmp/ninette-v2/[PATCH v2 2_5] oe-build-perf-report: Display >> more than 300 commits and date instead of commit number - Ninette >> Adhikari >> - 2024-05-03 1043.eml' >> '/tmp/ninette-v2/[PATCH v2 3_5] oe-build-perf-report: Improve >> report styling and add descriptions - Ninette Adhikari >> - >> 2024-05-03 1043.eml' >> '/tmp/ninette-v2/[PATCH v2 4_5] oe-build-perf-report: Update >> chart tooltip and chart type - Ninette Adhikari >> - >> 2024-05-03 1043.eml' >> '/tmp/ninette-v2/[PATCH v2 5_5] oe-build-perf-report: Add dark >> mode - Ninette Adhikari >> - 2024-05-03 1043.eml' >> >>> For local setup, you can do the following: >>> >>> 1. Clone the yocto-buildstats (https://git.yoctoproject.org/yocto-buildstats/) and the poky repository (https://git.yoctoproject.org/poky/) >>> >>> 2. In the poky repository run the following to build the report HTML: >>> ```bash >>> ./scripts/oe-build-perf-report -r "LOCAL_PATH_TO_YOCTO_BUILDSTATS" --branch "master" --commit "663f1805742ff6fb6955719d0ab7846a425debcf" --branch2 "master" --html > test.html >> >> These are the same setup step as last time I think and again, >> they aren't quite right but >> the script tells you do do: >> >> ❯ git fetch origin >> refs/notes/buildstats/perf-debian11/master/qemux86:refs/notes/buildstats/perf-debian11/master/qemux86 >> >> >> and that works. >> >> >>> ``` >>> Note: >>> - Add your local path to the yocto-buildstats repo >>> - The above command builds the report in a file called `test.html`. You can access it in the root directory in poky. >>> - This exmaple report uses the commit `663f1805742ff6fb6955719d0ab7846a425debcf` from `master` branch. >> >> Hmmm, ah, that's a poky commit id. I (stupidly!) expected it to >> be a commit ID in the yocto-buildstats repo. >> >> >> One thing that you can think about fixing or just ignore is that >> the --commit arg needs to be the >> full commit hash not the shortened version. i.e. this doesn't work: >> >> ❯ ./scripts/oe-build-perf-report -r >> "/media/rmacleod/gitter/rmacleod/src/distro/yocto/yocto-buildstats.git" >> --branch "master" --commit "663f180574" --branch2 "master" --html >> > /tmp/ninette-v2-test-663f180574.html >> Traceback (most recent call last): >>   File >> "/media/rmacleod/gitter/rmacleod/src/distro/yocto/poky.git/./scripts/oe-build-perf-report", >> line 617, in >>     sys.exit(main()) >>   File >> "/media/rmacleod/gitter/rmacleod/src/distro/yocto/poky.git/./scripts/oe-build-perf-report", >> line 532, in main >>     index1 = gitarchive.rev_find(revs, 'commit', args.commit) >>   File >> "/media/rmacleod/gitter/rmacleod/src/distro/yocto/poky.git/meta/lib/oeqa/utils/gitarchive.py", >> line 282, in rev_find >>     raise ValueError("Unable to find '{}' value >> '{}'".format(attr, val)) >> ValueError: Unable to find 'commit' value '663f180574' >> >> Since the script is going to be used in other scripts run on the >> autobuilder, this is a low priority enhancement. >> >> >> I was happy to see that a newer poky commit, worked fine: >> >> ❯ ./scripts/oe-build-perf-report -r >> "/media/rmacleod/gitter/rmacleod/src/distro/yocto/yocto-buildstats.git" >> --branch "master" --commit >> "632e3170595bb32717c0471a55a619b4b33fe787" --branch2 "master" >> --html > >> /tmp/ninette-v2-test-632e3170595bb32717c0471a55a619b4b33fe787.html >> INFO: Parsing buildstats from >> 'refs/notes/buildstats/perf-debian11/master/qemux86' >> >> >> but I did notice:  Stdev: nan >> >> >> whereas in your linked charts, and in my first run with commit: >> 663f1805742ff6fb6955719d0ab7846a425debcf >> >> the correct numerical standard deviation is displayed. >> >> I can debug this is you can't reproduce it yourself. >> > * if you can't  ... > > > Also, I wanted to draw your attention to how the vertical dashed > line (VDL) behaves > to see if it's something that can (or should) be changed. > > If you zoom in on a graph the VDL is placed at the left edge of  > bar until the pointer passes the mid-point of the bar: > > > at which time the VDL moves to the next bar/data point: > > > > It's a small thing but it would be nice if: >   1) the VDL was placed in the centre of the bar and >   2) the commit number /tooltip data only changed when the pointer > move to a different bar >       so the same data would be displayed as one swept across a > given bar. > > What do you think? > > > Picky, eh! ;-) > > > That's all for now. > > ../Randy > > >> >> Again. these reports and beautiful interactive charts are great >> to see. >> >> Thanks! >> >> ../Randy >> >> >>> Ninette Adhikari (5): >>> oe-build-perf-report: Add apache echarts to make report interactive >>> oe-build-perf-report: Display more than 300 commits and date instead >>> of commit number >>> oe-build-perf-report: Improve report styling and add descriptions >>> oe-build-perf-report: Update chart tooltip and chart type >>> oe-build-perf-report: Add dark mode >>> >>> .../build_perf/html/measurement_chart.html | 140 ++++++++++++------ >>> scripts/lib/build_perf/html/report.html | 124 +++++++++++----- >>> scripts/lib/build_perf/report.py | 5 +- >>> scripts/oe-build-perf-report | 6 +- >>> 4 files changed, 193 insertions(+), 82 deletions(-) >>> >> >> -- >> # Randy MacLeod >> # Wind River Linux >> >> -=-=-=-=-=-=-=-=-=-=-=- >> Links: You receive all messages sent to this group. >> View/Reply Online (#199007):https://lists.openembedded.org/g/openembedded-core/message/199007 >> Mute This Topic:https://lists.openembedded.org/mt/105889450/3616765 >> Group Owner:openembedded-core+owner@lists.openembedded.org >> Unsubscribe:https://lists.openembedded.org/g/openembedded-core/unsub [randy.macleod@windriver.com] >> -=-=-=-=-=-=-=-=-=-=-=- >> > > -- > # Randy MacLeod > # Wind River Linux > > > > -- > Ninette Adhikari > Software developer > The Neighbourhoodie Software GmbH > Harzer Straße 39, 12059 Berlin > neighbourhood.ie > > > Handelsregister HRB 157851 B Amtsgericht Charlottenburg > Geschäftsführung: Jan Lehnardt, Simone Haas -- # Randy MacLeod # Wind River Linux