Re: Relax tracing not working for me (solved)

[Philippe Gerum <rpm@xenomai.org>]

"Rosenow, James" <jim.rosenow@mts.com> writes:

> Thank you, Philippe.
>
> This helped; I have located the call source of the MSW and now I am in need some guidance on how to go about fixing it.
>
> The offending calls look like this:
>
> Userspace call:
>
>   int dma_move( struct ioctl_dma_req *dr_p)
>   {
>     return( ioctl( fd, IOCTL_DMA_MOVE, dr_p));
>   }
>
> Kernel driver call:
>
> int 
> ioctl_dma( struct dma_device *dev,
> 	   unsigned int cmd,
> 	   unsigned long arg)
> {
>   int retval;
>
>   retval = 0;
>   switch ( cmd)
>   {
>     case IOCTL_DMA_MOVE:
>     {
>       struct ioctl_dma_req dma_req;
>
>       if( copy_from_user(&dma_req, (void *)arg, sizeof(dma_req)))
>       {
>         return( -EFAULT);
>       }
>       retval = dma_move( dev, &dma_req);
>       if( retval < 0)
>       {
>         return( retval);
>       }
>       if( copy_to_user( (void *)arg, &dma_req, sizeof( dma_req)))
>       {
>         return -EFAULT;
>       }
>       break;
>     }
>
> So my questions are these:
> 1) Is the MSW occurring due to the ioctl() call from userspace?

Yes.

> 2) Or is the MSW occurring in the kernel module?
>

There can be no _implicit_ MSW from kernel space, because that one may
only happen on crossing the user->kernel boundary when handling a system
call, depending on the syscall type. This means that calling in-band
code from oob context past this crossing is always unchecked, unsafe,
and the fastest path to train wreck.

> And finally,
> 3) In this scenario, which must be somewhat common for real-time systems, i.e., a device driver is needed to facilitate some sort of real-time hardware operation, how does one 'port' this properly into Xenomai such that it does incur MSW?
>

The answer mostly depends on the device, so let's focus on DMA
ops:

a) the DMA hardware is known from - and managed by - the regular
   'dmaengine' framework available from the kernel. IOW, this peripheral
   may be providing data transfer services to some (inband) driver
   already. In this case, one would have to provide oob support to the
   corresponding DMA back-end driver which deals with the nitty-gritty
   details of the DMA hardware. This kind of support code is currently
   provided by Dovetail [1] for the bcm2835-dma and imx-sdma [2]
   back-ends. So, in order to share a DMA device between inband and oob
   activities seamlessly for (kernel-based) users (like dma_move() in
   your driver), one has to implement the oob extension briefly
   described in the doc I mentioned. There is no other sane option for
   addressing this case. Actually, there is no other working option,
   period. Also, I'm assuming a Dovetail-based setup, which provides the
   required infrastructure for implementing such extension. Legacy
   I-pipe is not an option in this case either.

b) the DMA engine is hosted on some custom hardware of your own, which
   is not available to common kernel users. IOW, only your driver would
   know about it. This is the simplest case: you would implement the DMA
   transfer ops using oob-compatible constructs (e.g. hard spinlocks for
   serialization), according to your DMA interface specs, easy
   peasy. That would even be compatible with concurrent inband use with
   proper serialization. In that case, some driver of yours would
   provide the DMA management code for that particular DMA engine, and
   dma_move() would call that code (or implement such code, for that
   matter).

c) the two options above assume that you have real-time requirements for
   DMA ops, since you mentioned 'real-time hardware operations'. If the
   use case is actually more relaxed, i.e. you need to kick the DMA
   transfer from some oob thread, but you could live with some unbounded
   delay until the transfer completes, then there is the option for any
   oob code to offload the DMA transfer to a routine which would run
   inband "asap". That routine would use the regular dmaengine calls to
   manage the transfer, no need for oob extension of the DMA back-end
   involved then.

[1] https://v4.xenomai.org/core/drivers/dma/
[2]
https://source.denx.de/Xenomai/linux-dovetail/-/blob/v6.1.y-cip-dovetail/drivers/dma/imx-sdma.c?ref_type=heads
(see CONFIG_IMX_SDMA_OOB)

-- 
Philippe.