Some issues running rteval on arm64, arm and i386

Some issues running rteval on arm64, arm and i386

[Punit Agrawal]

[-- Attachment #1: Type: text/plain, Size: 1015 bytes --]

Hi John,

We ran into a few issues when trying to run rteval on arm64, arm and
i386.

A few of the assumptions in rteval don't hold true on these systems.

    1. On arm64, there is no 'model name' in /proc/cpuinfo. See attached
       sample output from qemu arm64. I verified the same behaviour on
       an arm64 board we have as well.

    2. Also, the build target for the kernel on arm64 is "Image" of
       "bzImage". Rather than use different targets per-architecture it
       maybe better to drop it all together. Do you see any downsides?

The attached patches[1][2] gets things moving with regards to Issues 1
and 2 locally but I am not sure that's the best solution - especially
for "cpuinfo".

    3. Both arm and i386 do not provide "numa" nodes in sysfs. This
       causes rteval to complain "No valid nodes found in
       /sys/devices/system/node"

       Is this requirement planned to be relaxed - it'll be really
       useful to be able to use rteval on these architectures.

Thanks,
Punit


[-- Attachment #2: arm64 cpuinfo --]
[-- Type: text/plain, Size: 752 bytes --]

processor	: 0
BogoMIPS	: 125.00
Features	: fp asimd evtstrm aes pmull sha1 sha2 crc32 cpuid
CPU implementer	: 0x41
CPU architecture: 8
CPU variant	: 0x0
CPU part	: 0xd08
CPU revision	: 3

processor	: 1
BogoMIPS	: 125.00
Features	: fp asimd evtstrm aes pmull sha1 sha2 crc32 cpuid
CPU implementer	: 0x41
CPU architecture: 8
CPU variant	: 0x0
CPU part	: 0xd08
CPU revision	: 3

processor	: 2
BogoMIPS	: 125.00
Features	: fp asimd evtstrm aes pmull sha1 sha2 crc32 cpuid
CPU implementer	: 0x41
CPU architecture: 8
CPU variant	: 0x0
CPU part	: 0xd08
CPU revision	: 3

processor	: 3
BogoMIPS	: 125.00
Features	: fp asimd evtstrm aes pmull sha1 sha2 crc32 cpuid
CPU implementer	: 0x41
CPU architecture: 8
CPU variant	: 0x0
CPU part	: 0xd08
CPU revision	: 3


[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #3: Make model name optional --]
[-- Type: text/x-diff, Size: 1816 bytes --]

From 684b785cd0d1e75d922200fb93cd83bcd225097e Mon Sep 17 00:00:00 2001
From: Punit Agrawal <punit1.agrawal@toshiba.co.jp>
Date: Mon, 18 Jan 2021 17:51:12 +0900
Subject: [PATCH 1/2] rteval: cyclictest.py: Make 'model name' optional

Certain architectures such as arm64 don't have a "model name" in
/proc/cpuinfo. Relax the requirement to include the model name in the
description to allow running rteval on such machines.

Signed-off-by: Punit Agrawal <punit1.agrawal@toshiba.co.jp>
---
 rteval/modules/measurement/cyclictest.py | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/rteval/modules/measurement/cyclictest.py b/rteval/modules/measurement/cyclictest.py
index 232bd6b..afe87f7 100644
--- a/rteval/modules/measurement/cyclictest.py
+++ b/rteval/modules/measurement/cyclictest.py
@@ -217,13 +217,13 @@ class Cyclictest(rtevalModulePrototype):
         for core in self.__cpus:
             self.__cyclicdata[core] = RunData(core, 'core', self.__priority,
                                               logfnc=self._log)
-            self.__cyclicdata[core].description = info[core]['model name']
+            self.__cyclicdata[core].description = info[core].get('model name', '')
 
         # Create a RunData object for the overall system
         self.__cyclicdata['system'] = RunData('system',
                                               'system', self.__priority,
                                               logfnc=self._log)
-        self.__cyclicdata['system'].description = ("(%d cores) " % self.__numcores) + info['0']['model name']
+        self.__cyclicdata['system'].description = ("(%d cores) " % self.__numcores) + info['0'].get('model name', '')
 
         if self.__sparse:
             self._log(Log.DEBUG, "system using %d cpu cores" % self.__numcores)
-- 
2.29.2


[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #4: Drop kernel build targets --]
[-- Type: text/x-diff, Size: 1442 bytes --]

From 8095a27755bcad4542ae98bc3fd80d2ec2ef46fe Mon Sep 17 00:00:00 2001
From: Punit Agrawal <punit1.agrawal@toshiba.co.jp>
Date: Mon, 18 Jan 2021 17:53:52 +0900
Subject: [PATCH 2/2] rteval: cyclictest.py: Make build targets architecture
 independent

Not all kernel archiectures provide the "bzImage" target, e.g., arm64
provides "Image" which generates an uncompressed kernel binary.

Instead of going down the path of customizing build targets
per-architecture, let's drop them altogether. The default kernel
target should build the kernel and modules on all architectures
anyways.

Signed-off-by: Punit Agrawal <punit1.agrawal@toshiba.co.jp>
---
 rteval/modules/loads/kcompile.py | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/rteval/modules/loads/kcompile.py b/rteval/modules/loads/kcompile.py
index 326f1ae..e747b9f 100644
--- a/rteval/modules/loads/kcompile.py
+++ b/rteval/modules/loads/kcompile.py
@@ -58,7 +58,7 @@ class KBuildJob:
         else:
             self.jobs = self.calc_jobs_per_cpu() * len(self.node)
         self.log(Log.DEBUG, "node %d: jobs == %d" % (int(node), self.jobs))
-        self.runcmd = "%s make O=%s -C %s -j%d bzImage modules" \
+        self.runcmd = "%s make O=%s -C %s -j%d" \
                 % (self.binder, self.objdir, self.kdir, self.jobs)
         self.cleancmd = "%s make O=%s -C %s clean allmodconfig" \
                 % (self.binder, self.objdir, self.kdir)
-- 
2.29.2

Re: Some issues running rteval on arm64, arm and i386

[Ahmed S. Darwish]

On Mon, Jan 18, 2021 at 06:00:46PM +0900, Punit Agrawal wrote:
>
> We ran into a few issues when trying to run rteval on arm64, arm and
> i386.
>
> A few of the assumptions in rteval don't hold true on these systems.
>

For some embedded devices, it can be tough to use rteval with it. In
general, it requires, *on the target*:

  1. a full development toolchain (GCC, make, flex, bison, etc.)
  2. a full kernel source tree
  3. a full Python environment

You can use a combination of stress-ng and cyclictest to properly
evaluate your preempt_rt system latencies. Just make sure to exclude the
stress-ng stressors which allocate real-time threads, which can (and do)
conflict with the cyclictest ones.

Kind regards,

--
Ahmed S. Darwish

Re: Some issues running rteval on arm64, arm and i386

[John Kacur]

On Mon, 18 Jan 2021, Punit Agrawal wrote:

> Hi John,
> 
> We ran into a few issues when trying to run rteval on arm64, arm and
> i386.
> 
> A few of the assumptions in rteval don't hold true on these systems.
> 
>     1. On arm64, there is no 'model name' in /proc/cpuinfo. See attached
>        sample output from qemu arm64. I verified the same behaviour on
>        an arm64 board we have as well.
> 
>     2. Also, the build target for the kernel on arm64 is "Image" of
>        "bzImage". Rather than use different targets per-architecture it
>        maybe better to drop it all together. Do you see any downsides?
> 
> The attached patches[1][2] gets things moving with regards to Issues 1
> and 2 locally but I am not sure that's the best solution - especially
> for "cpuinfo".
> 
>     3. Both arm and i386 do not provide "numa" nodes in sysfs. This
>        causes rteval to complain "No valid nodes found in
>        /sys/devices/system/node"
> 
>        Is this requirement planned to be relaxed - it'll be really
>        useful to be able to use rteval on these architectures.
> 
> Thanks,
> Punit
> 
> 

In the past we targetted x86_64 only with rteval, but I definitely am 
interested in patches that would make it work on other architectures where 
the rt-kernel runs.

Could you resend the patches inline in separate mails, as you would when
sending patches for the linux kernel? The reason for this is it makes it
easier to comment on the code.

Thanks

John

Re: Some issues running rteval on arm64, arm and i386

[John Kacur]

On Fri, 22 Jan 2021, Ahmed S. Darwish wrote:

> On Mon, Jan 18, 2021 at 06:00:46PM +0900, Punit Agrawal wrote:
> >
> > We ran into a few issues when trying to run rteval on arm64, arm and
> > i386.
> >
> > A few of the assumptions in rteval don't hold true on these systems.
> >
> 
> For some embedded devices, it can be tough to use rteval with it. In
> general, it requires, *on the target*:
> 
>   1. a full development toolchain (GCC, make, flex, bison, etc.)
>   2. a full kernel source tree
>   3. a full Python environment
> 
> You can use a combination of stress-ng and cyclictest to properly
> evaluate your preempt_rt system latencies. Just make sure to exclude the
> stress-ng stressors which allocate real-time threads, which can (and do)
> conflict with the cyclictest ones.
> 
> Kind regards,
> 
> --
> Ahmed S. Darwish
> 

I agree, rteval wasn't designed for the embedded world. The reason for the
full development tool chain is to compile the kernel as a load, so if you 
are compiling the kernel on a machine other than the device it is 
targetted for, then rteval won't work. Also, yes it runs with a python3
envirnoment, we still maintain a version for python2, but no new 
development is occuring there.

note that rteval can now run stress-ng too, but the default is without it.

John

Re: Some issues running rteval on arm64, arm and i386

[Punit Agrawal]

Hi John,

John Kacur <jkacur@redhat.com> writes:

[...]

> In the past we targetted x86_64 only with rteval, but I definitely am 
> interested in patches that would make it work on other architectures where 
> the rt-kernel runs.

Thanks for confirming.

> Could you resend the patches inline in separate mails, as you would when
> sending patches for the linux kernel? The reason for this is it makes it
> easier to comment on the code.

Sure, no problem. I'll resend the patches inline in a separate thread
shortly.

Thanks,
Punit

[...]

Re: Some issues running rteval on arm64, arm and i386

[Punit Agrawal]

Hi John, Ahmed,

John Kacur <jkacur@redhat.com> writes:

> On Fri, 22 Jan 2021, Ahmed S. Darwish wrote:
>
>> On Mon, Jan 18, 2021 at 06:00:46PM +0900, Punit Agrawal wrote:
>> >
>> > We ran into a few issues when trying to run rteval on arm64, arm and
>> > i386.
>> >
>> > A few of the assumptions in rteval don't hold true on these systems.
>> >
>> 
>> For some embedded devices, it can be tough to use rteval with it. In
>> general, it requires, *on the target*:
>> 
>>   1. a full development toolchain (GCC, make, flex, bison, etc.)
>>   2. a full kernel source tree
>>   3. a full Python environment
>> 
>> You can use a combination of stress-ng and cyclictest to properly
>> evaluate your preempt_rt system latencies. Just make sure to exclude the
>> stress-ng stressors which allocate real-time threads, which can (and do)
>> conflict with the cyclictest ones.

[...]

> I agree, rteval wasn't designed for the embedded world. The reason for the
> full development tool chain is to compile the kernel as a load, so if you 
> are compiling the kernel on a machine other than the device it is 
> targetted for, then rteval won't work. Also, yes it runs with a python3
> envirnoment, we still maintain a version for python2, but no new 
> development is occuring there.
>
> note that rteval can now run stress-ng too, but the default is without it.

Thanks for your comments.

Fortunately, the device filesystems can provide the dependencies
(toolchain, python3, etc.) of rteval. But I'll keep in mind alternative
ways to evaluate the system latencies when we can't.

Thanks,
Punit

[...]