Re: [EVL] Problems writing to physical memory with mmap on oob context

[Philippe Gerum <rpm@xenomai.org>]

Jesus Villena <j.villena@ecler.com> writes:

> Hello,
>
> We are using an ARM64 based platform which includes a small Cortex M4 processor to run some algorithms, and both subsystems use an internal shared SRAM memory to implement the inter-processor communication.
>
> In mainline Linux, we open a "/dev/mem" file in the ARM64 core to get access to physical memory, and mmap the shared area to read and write directly to the SRAM from userland.
>
> With EVL, it doesn't work: after opening the file and doing mmap, we call evl_attach_self to switch to the oob context, and then, reading on the shared area works, but writing aborts the program with the message:
>
> EVL: evl_test switching in-band [pid=363, excpt=0, user_pc=0x404b5c]
> oop...
> CPU time limit exceeded
>
> I don't understand why EVL is switching to in-band context.
>
> Do you know where is the problem? Why does reading work but writing doesn't?
>

Looking at the output you get, it most likely behaves the same and does
work in both cases, w/ and w/o EVL involved. In both cases you receive a
so-called major fault when writing to this area: the only difference is
that EVL reports such kind of event which requires heavyweight fixups
when running oob by sending a SIGXCPU signal. You definitely want to
read this doc [1] about these notifications.

Those major faults cannot be handled directly from the oob stage, so EVL
downgrades the thread issuing the write access request to in-band
context first, then notifies the issuer about it via SIGXCPU. You could
disable this signal by clearing the EVL_T_HMSIG flag for that thread
using evl_set_thread_mode(), or set a dummy handler for this signal. But
that would obviously only paper over the problem, which is that the MMU
wants some help from the kernel for dealing with this write access in
particular.

If not already set, you may want to turn CONFIG_STRICT_DEVMEM on,
checking whether mmap() still succeeds or fails with EPERM. That would
give you a first hint about how the kernel views this SRAM range,
i.e. dev exclusive or other. In any case, figuring out why the CPU traps
for that access may require some debug work with the kernel code (I
would start looking at arch/arm64/mm/fault.c, do_page_fault()
typically). You may also want to include the KERN_ALERT log level in the
console output, in order to get a number of low-level memory management
traces on arm64.

PS: Since /dev/mem is a liability when it comes to security, you may
want to write your own (small) driver only for the purpose of creating
memory mappings specifically from the SRAM to your application,
disabling /dev/mem entirely next. EVL would still have no part in this.

[1] https://v4.xenomai.org/core/user-api/thread/#hm-notification-methods

-- 
Philippe.