Re: [libevl][PATCH 3/4] evl-test: Start evl-test with load by default

["Jorge Ramirez-Ortiz, Gmail" <jorge.ramirez.ortiz@gmail.com>]

On 27/10/24 20:13:08, Jorge Ramirez-Ortiz, Gmail wrote:
> On 24/10/24 20:00:12, Schaffner, Tobias wrote:
> > Hi Philippe, hi Jorge,
> > 
> > On Wed, 2024-07-10 at 09:18 +0200, Tobias Schaffner wrote:
> > > Hey Jorge,
> > > 
> > > On 14.06.24 09:29, Jorge Ramirez-Ortiz, Gmail wrote:
> > > > On 13/06/24 19:27:10, Philippe Gerum wrote:
> > > > > 
> > > > > Philippe Gerum <rpm@xenomai.org> writes:
> > > > > 
> > > > > > Tobias Schaffner <tobias.schaffner@siemens.com> writes:
> > > > > > 
> > > > > > > Stress evl with a load command while running the tests. The
> > > > > > > load command
> > > > > > > may be set to an empty string to run tests without stressing
> > > > > > > the system.
> > > > > > > 
> > > > > > > To align with xeno-test the -l command line argument was used
> > > > > > > for the load
> > > > > > > command. Listing of the unittests can now be done with --list
> > > > > > > and --List.
> > > > > > > 
> > > > > > 
> > > > > > Nak. Compat with xeno-test is definitely not a requirement if
> > > > > > this means
> > > > > > breaking the existing evl command line usage. Besides, --List
> > > > > > looks
> > > > > > strange as an option. Let's pick a different option for the
> > > > > > load command
> > > > > > instead.
> > > > > > 
> > > > > 
> > > > > We may want to implement the stress utility as a separate evl-
> > > > > stress
> > > > > command, keeping evl-test for unit testing which has a different
> > > > > purpose. evl-stress as a script could generate the load (dohell)
> > > > > and
> > > > > possibly check the latency figures reported by latmus (using the
> > > > > -K and
> > > > > -A switches to detect weird behavior).
> > > > > 
> > > > > PS: Jorge is going to look into leveraging stress-ng for load
> > > > > generation.
> > > > 
> > > > yes I am going to start looking into it. I think that maintaining
> > > > Linux
> > > > stressors should not be Xenomai's business. If we can't replicate
> > > > the
> > > > load with todays stress-ng perhaps we could just usptream what is
> > > > needed (stress-ng is well maintained)
> > > 
> > 
> > may I bring this up again? Any new developments or opinions on this?
> > I would like to see a solution that allows to compare xenomai 3 and
> > EVL. Reliable numbers are necessary to make an informed decision if EVL
> > is on par or even better than xenomai 3. Imo we are comparing apples
> > and oranges as long as we are not using the same tooling on both
> > platforms.
> 
> Sorry about this, it is being a bit of a crazy year for me at a personal
> levely. I am start on it this week.
> 
> My setup uses some Zephyr code executing on frdmk64 I wrote years ago to
> benchmark evl
> 
> https://github.com/ldts/libevl/commit/905d1946ca117e3889c53d1150a1334cc91033a8
> 
> I will load my evl system using stress-ng and measure latency figures
> and then replicate with dohell loading tools. I should have something to
> share by next friday.
> 
> feel free to ping me anytime. In IRC you can find me in LiberaChat
> #u-boot (nick ldts) and since recently in OFTC #edk2

I was finally able to get some numbers on imx8mm (an Arduino Pro with
the Portenta breakout board recently contributed to my lab by Wim
Taymans for audio testing of the Xenomai4 Pipewire-plugin[0] for
realtime audio)

I am using dohell/rt-tools/ltp and stress-ng on EVL - havent tested
Xenomai3.

My test bench runs with the help of an external FRDMK64F running Zephyr
very similar to what is described in the latmus/latmon combo
documentation [1]. Given how cheap this board is perhaps it should be
more widely used (?).

[RFC]
I would like to update latmon[2] to the tip of Zephyr - it has fallen
behind - I'd like then to propose it upstream (if accepted I would like
to propose that it is removed from EVL). Anyone disagrees?

IMO We should also take latmus to its own separate tree dedicated to RT
benchmarking and extend it to support scheduling latency metrics based
on GPIO data.
[\]

So:

* stress-ng --vm 2 --vm-bytes 1G --mmap 2 --mmap-bytes 1G --page-in \
  --matrix 0 --matrix-size 64 --tz -t 60

This simple command (VM and cpu load for a 2GB DDR system) hits the
system very quick and my externally measured GPIO IRQ latency peaks
within seconds.

stress-ng has FPU, VM, CPU all sort of stressors but I didnt need to go
into those just yet.

I found some info that RH [3] is providing interesting - still the
stress-ng code is simple enough to work with if needed (really good
encapsulation and code base)

* dohell -l /opt/ltp -b /bin/hackbench -m /home/root

After five minutes of run the latency is still contained (far from
peaking, 50% below the peak) but I assume this is because LTP takes a
while to run all tests. Without going into more details, this difference
in the invested time seems enough for me to switch to stress-ng.


Jorge

[0] https://gitlab.freedesktop.org/pipewire/pipewire/-/commit/ffaa365beff0e7f1d1f395800c14b1be26d8d007
[1] https://evlproject.org/core/benchmarks/#latmus-gpio-response-time
[2] https://source.denx.de/Xenomai/xenomai4/libevl/-/tree/master/benchmarks/zephyr/latmon/ 
[3] https://docs.redhat.com/en/documentation/red_hat_enterprise_linux_for_real_time/9/html/optimizing_rhel_9_for_real_time_for_low_latency_operation/assembly_stress-testing-real-time-systems-with-stress-ng_optimizing-rhel9-for-real-time-for-low-latency-operation#proc_testing-cpu-with-multiple-stress-mechanisms_assembly_stress-testing-real-time-systems-with-stress-ng