[Xenomai-help] dma in rtdm

[Xenomai-help] dma in rtdm

[Harkema, G.A.]

[-- Attachment #1: Type: text/plain, Size: 1122 bytes --]

Hallo,

 

We are currently trying to develop a realtime usb2.0 stack based on
xenomai rtdm. In non-realtime mode we use first the kmalloc() function
to allocate a dma buffer and use the dma_map_single() function to get
the bus address for the dma engine of the ehci hostcontroller. This
works fine. For use in a realtime context we replaced the kmalloc()
function by the rtdm_malloc() function. In our logs we see that the data
transfer take place, but the dma buffer was not filled with the
(received)data. We don't know where the data in the memory is stored. Is
this the right way for using dma in realtime context ? I guess not., but
what is the right way. Who can help us with this problem.

 

Thanks.

 

Kindest Regards,

 

Gerard

 

G.A. Harkema

Senior Hardware & Software Engineer

Eindhoven University of Technology

/department of physics

Measurement and Computer Science

Den Dolech 2

P.O.Box 513

5600 MB  Eindhoven

The Netherlands

tel: +31 40 247 4607

fax: +31 40 247 2144

www.tuedacs.nl

P Before you print think about the ENVIRONMENT

 


[-- Attachment #2: Type: text/html, Size: 8044 bytes --]

Re: [Xenomai-help] dma in rtdm

[Jan Kiszka]

[-- Attachment #1: Type: text/plain, Size: 1591 bytes --]

Harkema, G.A. wrote:
> Hallo,
> 
>  
> 
> We are currently trying to develop a realtime usb2.0 stack based on
> xenomai rtdm. In non-realtime mode we use first the kmalloc() function
> to allocate a dma buffer and use the dma_map_single() function to get
> the bus address for the dma engine of the ehci hostcontroller. This
> works fine. For use in a realtime context we replaced the kmalloc()
> function by the rtdm_malloc() function. In our logs we see that the data

rtdm_malloc does not return memory guaranteed to be usable for DMA (we
should document this somewhere). Depending on the size of your system
heap, the memory /may/ come from a pre-allocated kmalloc'ed (heap <=
128K) or from a vmalloc'ed block.

> transfer take place, but the dma buffer was not filled with the
> (received)data. We don't know where the data in the memory is stored. Is
> this the right way for using dma in realtime context ? I guess not., but
> what is the right way. Who can help us with this problem.

I'm convinced that ahead-of-time allocation of URBs, transfer
descriptors, and other buffers is the way to go. The application /
high-level driver requesting some service from the UHCI controller
should take care of providing them. That way you will gain full
determinism, because even rtdm_malloc is not guaranteed to work always.
Low memory or fragmentation caused by other users of this service may
bite you in complex scenarios.

Jan


PS: Is your design now based on usb4rt or do you work on a separate
solution? I haven't heard any news from you on this.


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 250 bytes --]

[Xenomai-help] Xenomai task (beginner)

[GUARDIOLA-FALCO Sebastien 204282]

Hello,

I'm waiting for Xenomai to be included in the ElinOS Linux package, so I can't try it before next week. 
My question is simple : how to port a linux program to Xenomai so this program is recognized as a Xenomai task? (and has low latencies?). Is it done when launching the program? like "prio 255 myprogram" in LynxOS? or are there lines of code to include?
My linux task doesn't have any system call, except mmap(). Next I'm using the pointer I get from mmap() and that's it.

Thanks a lot,
Sébastien.

Re: [Xenomai-help] Xenomai task (beginner)

[Gilles Chanteperdrix]

GUARDIOLA-FALCO Sebastien 204282 wrote:
 > Hello,
 > 
 > I'm waiting for Xenomai to be included in the ElinOS Linux package, so I can't try it before next week. 
 > My question is simple : how to port a linux program to Xenomai so this program is recognized as a Xenomai task? (and has low latencies?). Is it done when launching the program? like "prio 255 myprogram" in LynxOS? or are there lines of code to include?
 > My linux task doesn't have any system call, except mmap(). Next I'm using the pointer I get from mmap() and that's it.
 > 

If you are talking about latencies, then your task has to suspend,
waiting for some event using some other system call than mmap. I assume
here that you are using the POSIX interface for that purpose. 

To have a Linux program "recognized" by Xenomai, you have to compile it
with special compilation flags, provided by the xeno-config script. The
advantage of this approach is that the regular Linux services remain
accessible by prefixing their names with the __real_ prefix. For example
Linux regular mmap remain accessible but is called __real_mmap.

If your program is called hello.c, you would compile it for Xenomai
POSIX skin by calling:

gcc -o hello.o  -c hello.c `xeno-config --posix-cflags`

gcc -o hello hello.o `xeno-config --posix-ldflags`

-- 


					    Gilles Chanteperdrix.

RE: [Xenomai-help] Xenomai task (beginner)

[GUARDIOLA-FALCO Sebastien 204282]

[-- Attachment #1: Type: text/plain, Size: 2033 bytes --]

GUARDIOLA-FALCO Sebastien 204282 wrote:
 >> Hello,
 >> 
 >> I'm waiting for Xenomai to be included in the ElinOS Linux package, so I can't try it before next week. 
 >> My question is simple : how to port a linux program to Xenomai so this program is recognized as a Xenomai task? (and has low latencies?). Is it done when launching the program? like "prio 255 myprogram" in LynxOS? or are there lines of code to include?
 >> My linux task doesn't have any system call, except mmap(). Next I'm using the pointer I get from mmap() and that's it.
 >> 

Gilles Chanteperdrix wrote:
>If you are talking about latencies, then your task has to suspend,
>waiting for some event using some other system call than mmap. I assume
>here that you are using the POSIX interface for that purpose. 

Actually my program (see attached file) doesn't wait, it's a "polling" task (on a VME bus) that reads a pointer.
And if data read shows that we missed an event/data (means that data refresh is too fast, 500 microseconds for Linux for example), then I save/display the error.
So there's no blocking call here, only polling.

>To have a Linux program "recognized" by Xenomai, you have to compile it
>with special compilation flags, provided by the xeno-config script. The
>advantage of this approach is that the regular Linux services remain
>accessible by prefixing their names with the __real_ prefix. For example
>Linux regular mmap remain accessible but is called __real_mmap.

Interesting. About the compilation, does it means that when I launch such a task the Xenomai scheduler will do all the job? only because I include this 'xeno-config' option?
If it's the case, I'm wondering if that scheduler will increase performance for my polling thing... not sure.
What is certain is that task priority handling must be greater in Xenomai than Linux/premmptive.
About that, does my "sched_get_priority_max(SCHED_FIFO)" of my code will have any effect on Xenomai? a Xenomai equivalent for this call?

Thanks,
Sébastien.

[-- Attachment #2: polling.c --]
[-- Type: application/octet-stream, Size: 3099 bytes --]

/*
**   FILE   : pooling.c
**
**   AUTHOR : Sébastien GUARDIOLA FALCO
**
**   PRODUCT : Reading Test program on VME bus (polling an acquisition board)
**
**   VERSION: 1.0 LINUX
**
**   Created : 03 Juillet 2006
**
**   Comments :
**
**   
**
*/
#include <sched.h>
#include <sys/mman.h>
#include <sys/stat.h>
#include "vmedrv.h"
#include <stdlib.h>
#include <stddef.h>
#include <sys/mman.h>
#include <unistd.h> 
#include <errno.h>
#include <sys/file.h>
#include <sys/ioctl.h>
#include <sys/types.h>
#include <fcntl.h>
#include <stdio.h>
#include <strings.h>
#include <signal.h>
#include "chrono.h"

static int ITER=1;
static FILE *output;
static unsigned long tab[1];


int main(int argc, char *argv[])
{
   struct timeval start, elapsed;
   int k=0,fdOut,n;
   long int time_e;
   unsigned long msb,lsb=0,dateprec,evt;
   char * ptrA32;
   char devnode[20];
   struct sched_param mysched;

   output = fopen("Result_file","w");

/*** SET TASK PRIORITY ***/
    mysched.sched_priority = sched_get_priority_max(SCHED_FIFO) - 1;
    if( sched_setscheduler( 0, SCHED_FIFO, &mysched ) == -1 ) {
      printf("ERROR IN SETTING THE SCHEDULER");
      perror("errno");
      exit(1);
    }

/*** OPEN VME WINDOW ***/
    sprintf(devnode, "/dev/vme_m%d", 0);
    fdOut = open(devnode, O_RDWR);
    if (fdOut < 0) {
 	printf("Ouverture de la fenetre #%d impossible.  Errno = %d\n", 0, errno);
	_exit(1);
    }

/****** MMAP OF THAT WINDOW ********/
    ptrA32 = mmap(0,0x10000000, PROT_READ|PROT_WRITE, MAP_SHARED, fdOut, (off_t)0x00000000);
    if(ptrA32 == MAP_FAILED)
    {
	printf("Echec lors du mmap().\n");
	return -1;
    }

/****** INIT OF ACQUISITION BOARD *******/
   chroVmeInit(ptrA32);
   chroVmeSetMode(PROG);
   chroVmeSetRAZDate(0x20,ENRAZDATE,RAZDATENIV0,4);
   chroVmeSetCpt(DATE_H_MHZ);
   chroVmeRAZCpt(CPT_DATE);
   chroVmeSetMode(UTIL);
   for(k=0;k<ITER;k++)
	tab[k]=0;
   printf("init ok\n");
   chroVmeSetSensEvtDate(1, NIV0);

/*** LOCK MEMORY ****/ 
   mlockall(MCL_CURRENT | MCL_FUTURE);

/*** SAVE START TIME ***/
   gettimeofday(&start,(struct timeval*)0);
   
/*** READ EVENT (NON-BLOCKING) ***/
   chroVmeGetEvtDate(&evt);

/*** READ DATA ASSOCIATED TO THAT EVENT ***/
   chroVmeGetDateEvtDate( 1, &msb, &dateprec);

/*** DO THIS READ FOR A WHILE... ***/
   for(k=0;k<ITER;)
   {
     chroVmeGetEvtDate(&evt);
     if (evt&EVDATE1)
     {
	
      chroVmeGetDateEvtDate( 1, &msb, &lsb);

	/*** PROCESSING DATA (ERROR IF THE PROGRAM IS MISSING AN EVENT) ***/
	if((lsb-dateprec)>600) 
	{
	    if(lsb!=0) {
	        tab[k]=lsb-dateprec;
                ++k;
	    }
	}
	dateprec=lsb;
     }
 
   }


   gettimeofday(&elapsed,(struct timeval*)0);
   time_e=(elapsed.tv_sec-start.tv_sec)*1000000 + (elapsed.tv_usec-start.tv_usec);
   printf("Error found in %d microseconds.\n",time_e);

/*** WRITING RESULTS ***/
   for(k=0;k<ITER;k++)
	fprintf(output,"%d\n",tab[k]);


  munmap(ptrA32,0x10000000);
  close(fdOut);
  fclose(output);
  return 0;
}

RE: [Xenomai-help] Xenomai task (beginner)

[Gilles Chanteperdrix]

GUARDIOLA-FALCO Sebastien 204282 wrote:
 > GUARDIOLA-FALCO Sebastien 204282 wrote:
 >  >> Hello,
 >  >> 
 >  >> I'm waiting for Xenomai to be included in the ElinOS Linux package, so I can't try it before next week. 
 >  >> My question is simple : how to port a linux program to Xenomai so this program is recognized as a Xenomai task? (and has low latencies?). Is it done when launching the program? like "prio 255 myprogram" in LynxOS? or are there lines of code to include?
 >  >> My linux task doesn't have any system call, except mmap(). Next I'm using the pointer I get from mmap() and that's it.
 >  >> 
 > 
 > Gilles Chanteperdrix wrote:
 > >If you are talking about latencies, then your task has to suspend,
 > >waiting for some event using some other system call than mmap. I assume
 > >here that you are using the POSIX interface for that purpose. 
 > 
 > Actually my program (see attached file) doesn't wait, it's a "polling" task (on a VME bus) that reads a pointer.
 > And if data read shows that we missed an event/data (means that data refresh is too fast, 500 microseconds for Linux for example), then I save/display the error.
 > So there's no blocking call here, only polling.
 > 
 > >To have a Linux program "recognized" by Xenomai, you have to compile it
 > >with special compilation flags, provided by the xeno-config script. The
 > >advantage of this approach is that the regular Linux services remain
 > >accessible by prefixing their names with the __real_ prefix. For example
 > >Linux regular mmap remain accessible but is called __real_mmap.
 > 
 > Interesting. About the compilation, does it means that when I launch such a task the Xenomai scheduler will do all the job? only because I include this 'xeno-config' option?
 > If it's the case, I'm wondering if that scheduler will increase performance for my polling thing... not sure.

The worst case latency of your polling program is given by the longest
time during which Linux will preempt your polling task. If this polling
program is the only program running, then the longest preemption will be
given by the execution time of the longest interruption handler, likely
the timer interrupt.

You will get similar behaviour with Xenomai, with the additional
constraint that Linux will not be running at all as long as your
program runs. If we assume that Xenomai timer interrupt handler is
shorter than Linux interrupt handler, then you will get a shorter
worst case latency with Xenomai.

Note however that either with Xenomai or with Linux using the SCHED_FIFO
policy, you can not be running your polling program all the time,
because everything else that is not running with the SCHED_FIFO policy
will be blocked.

Using only polling is generally considered a poor design. If the
activity on your VME bus is such that you can not tolerate one interrupt
by event, then you may adopt a design similar to Linux network drivers
using NAPI, that is:
- enable generation of interruptions upon VME bus event
- in the interrupt handler, disable generation of interruptions and wake
  up a polling thread
- in the polling thread, poll the VME bus for a limited amount of time,
  then re-enable generation of interruptions and go to sleep.

This way, the number of interruptions will be limited, and you will
leave some time for the system to run other activities than the
acquisition loop.

 > What is certain is that task priority handling must be greater in Xenomai than Linux/premmptive.
 > About that, does my "sched_get_priority_max(SCHED_FIFO)" of my code will have any effect on Xenomai? a Xenomai equivalent for this call?
 > 

Xenomai posix skin uses the same priority range as Linux, so setting the
priority to sched_get_priority_max(SCHED_FIFO) will have the same effect
on Xenomai. Note however that with Xenomai, you will have to use
pthread_setschedparam in order to set the thread priority, instead of
sched_setscheduler.

 > Thanks,
 > Sébastien.


-- 


					    Gilles Chanteperdrix.

RE: [Xenomai-help] Xenomai task (beginner)

[GUARDIOLA-FALCO Sebastien 204282]

Gilles Chanteperdrix wrote :
>The worst case latency of your polling program is given by the longest
>time during which Linux will preempt your polling task. If this polling
>program is the only program running, then the longest preemption will be
>given by the execution time of the longest interruption handler, likely
>the timer interrupt.

Ok so if I understand, during a polling cycle interrupted by preemption of the timer interrupt, we are in the worst case latency. Could you explain a little what is this timer interrupt please? As far as I know, this soft irq is how the kernel stops doing whatever it's currently doing to check if it needs to be doing something else. I'm using Linux 2.4 with the preemptible kernel option. Do you have an idea of the frequency of the timer interrupt or how to know it? My results shows weird things like a quite long latency (a value between 100 and 200 microseconds) EVERY 60 seconds! May be it's this timer we are talking about?


>You will get similar behaviour with Xenomai, with the additional
>constraint that Linux will not be running at all as long as your
>program runs. If we assume that Xenomai timer interrupt handler is
>shorter than Linux interrupt handler, then you will get a shorter
>worst case latency with Xenomai.

By "if we assume", you may say that it depends on kernel version. On this point, do you know the differences of my linux 2.4 alone, versus Xenomai patched to 2.6?

>Note however that either with Xenomai or with Linux using the SCHED_FIFO
>policy, you can not be running your polling program all the time,
>because everything else that is not running with the SCHED_FIFO policy
>will be blocked.

I disabled this policy since today, precisely trying to run other processes in order to load cpu usage. But it's obvious that even with a normal policy, my prompt is completely blocked! Same result on serial port console and with a telnet...

>Using only polling is generally considered a poor design. If the
>activity on your VME bus is such that you can not tolerate one interrupt
>by event, then you may adopt a design similar to Linux network drivers
>using NAPI, that is:
>- enable generation of interruptions upon VME bus event
>- in the interrupt handler, disable generation of interruptions and wake
>  up a polling thread
>- in the polling thread, poll the VME bus for a limited amount of time,
>  then re-enable generation of interruptions and go to sleep.
>This way, the number of interruptions will be limited, and you will
>leave some time for the system to run other activities than the
>acquisition loop.

Ok, for the moment VME interruptions are not used in our context (see below). So enabling or disabling interrupt handling in the driver should lead to the same results. In other words, if there is no interruption to handle it would be a surprise that the VME driver executes some code in the interrupt handler.
In fact, we intend to use polling not every cycle like in my test program, but regularly when there is more data to pick on the bus. This test is designed to know more easily the worst case, because we do not tolerate any missing data/event.
We use interruptions too, in other acquisition programs... but this is another story.

Thanks again Gilles,
Sébastien.

RE: [Xenomai-help] Xenomai task (beginner)

[Gilles Chanteperdrix]

GUARDIOLA-FALCO Sebastien 204282 wrote:
 > Gilles Chanteperdrix wrote :
 > >The worst case latency of your polling program is given by the longest
 > >time during which Linux will preempt your polling task. If this polling
 > >program is the only program running, then the longest preemption will be
 > >given by the execution time of the longest interruption handler, likely
 > >the timer interrupt.
 > 
 > Ok so if I understand, during a polling cycle interrupted by
 > preemption of the timer interrupt, we are in the worst case
 > latency. 

Only if there are no other activities with higher priority. And in
this case you observe that the computer is blocked during the polling.

 > Could you explain a little what is this timer interrupt
 > please? As far as I know, this soft irq is how the kernel stops doing
 > whatever it's currently doing to check if it needs to be doing
 > something else. I'm using Linux 2.4 with the preemptible kernel
 > option. Do you have an idea of the frequency of the timer interrupt
 > or how to know it? My results shows weird things like a quite long
 > latency (a value between 100 and 200 microseconds) EVERY 60 seconds! 
 > May be it's this timer we are talking about?

The timer interrupt frequency is 100 Hz on 2.4 and either 100 Hz, 250 Hz
or 1000 Hz on 2.6, it is configurable at compilation time.

 > 
 > 
 > >You will get similar behaviour with Xenomai, with the additional
 > >constraint that Linux will not be running at all as long as your
 > >program runs. If we assume that Xenomai timer interrupt handler is
 > >shorter than Linux interrupt handler, then you will get a shorter
 > >worst case latency with Xenomai.
 > 
 > By "if we assume", you may say that it depends on kernel version. On this point, do you know the differences of my linux 2.4 alone, versus Xenomai patched to 2.6?

No, I meant that it is not directly a measure that matters in typical
use cases, because Xenomai anticipates, the hardware timer ticks a
little bit before the next software timer release point in order to
account for the interrupt duration. The measure that matters in typical
use cases is the scheduling latency. This said, Xenomai timer interrupt
should have much less work to do than Linux timer interrupt, so should
be shorter.

 > 
 > >Note however that either with Xenomai or with Linux using the SCHED_FIFO
 > >policy, you can not be running your polling program all the time,
 > >because everything else that is not running with the SCHED_FIFO policy
 > >will be blocked.
 > 
 > I disabled this policy since today, precisely trying to run other processes in order to load cpu usage. But it's obvious that even with a normal policy, my prompt is completely blocked! Same result on serial port console and with a telnet...

If you do not use SCHED_FIFO policy, then your worst case latency will
be much higher than if you were using interruptions, because Linux will
forcibly suspend your task from time to time in order to let other tasks
run during tens of milliseconds.

-- 


					    Gilles Chanteperdrix.

Re: [Xenomai-help] Xenomai task (beginner)

[Jan Kiszka]

[-- Attachment #1: Type: text/plain, Size: 363 bytes --]

GUARDIOLA-FALCO Sebastien 204282 wrote:
> Hello,
> 
> I'm waiting for Xenomai to be included in the ElinOS Linux package, so I can't try it before next week. 

Oh, good to hear that this long announced update finally takes place! I
think there are quite a few ElinOS users out there happily awaiting it.
Let's see what version they will deliver.

Jan


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 250 bytes --]

Re: [Xenomai-help] Xenomai task (beginner)

[Ulrich Schwab]

On Tuesday 25 July 2006 19:41, Jan Kiszka wrote:
> GUARDIOLA-FALCO Sebastien 204282 wrote:
> > Hello,
> >
> > I'm waiting for Xenomai to be included in the ElinOS Linux package, so I
> > can't try it before next week.
>
> Oh, good to hear that this long announced update finally takes place! I
> think there are quite a few ElinOS users out there happily awaiting it.
> Let's see what version they will deliver.
I doubt this, since they have their own realtime implementation (PikeOS).
According to them, it is better than any other, of course ! :-)

I never saw  xenomai support announced, by the way.


Ulrich Schwab
-- 
====================================================
  inmess GmbH
  Frankfurter Str. 74
  D - 64521 Gross-Gerau
  Phone: +49 6152 97790
  Fax  : +49 6152 977920
  mail : info@domain.hid
  web:   www.inmess.de
====================================================

RE: [Xenomai-help] Xenomai task (beginner)

[GUARDIOLA-FALCO Sebastien 204282]

On Tuesday 25 July 2006 19:41, Jan Kiszka wrote:
> GUARDIOLA-FALCO Sebastien 204282 wrote:
> > I'm waiting for Xenomai to be included in the ElinOS Linux package, so I
> > can't try it before next week.
>
> Oh, good to hear that this long announced update finally takes place! I
> think there are quite a few ElinOS users out there happily awaiting it.
> Let's see what version they will deliver.
>I doubt this, since they have their own realtime implementation (PikeOS).
>According to them, it is better than any other, of course ! :-)

>I never saw  xenomai support announced, by the way.

This is not announced on their website, I saw it. You know, sometimes commercials forget things... or not. 
But there is no doubt since I am in contact with their support and also checked my member ftp area ;)

Sébastien.

Re: [Xenomai-help] Xenomai task (beginner)

[Jan Kiszka]

[-- Attachment #1: Type: text/plain, Size: 1410 bytes --]

Ulrich Schwab wrote:
> On Tuesday 25 July 2006 19:41, Jan Kiszka wrote:
>> GUARDIOLA-FALCO Sebastien 204282 wrote:
>>> Hello,
>>>
>>> I'm waiting for Xenomai to be included in the ElinOS Linux package, so I
>>> can't try it before next week.
>> Oh, good to hear that this long announced update finally takes place! I
>> think there are quite a few ElinOS users out there happily awaiting it.
>> Let's see what version they will deliver.
> I doubt this, since they have their own realtime implementation (PikeOS).
> According to them, it is better than any other, of course ! :-)

Not all of their customers "buy" this. PikeOS is a different league.
It's a microkernel-based full virtualisation of Linux, just like Jaluna,
Integrity, or what they are all called. It has its very specific
application domain (nothing comes for free, you know...). And you have
to consider that its proprietary software by vendor X - not really
popular anymore. :)

But ElinOS also has RTAI support, I know some of its users. One told me
that they will not migrate to Xenomai before Sysgo supports it -
political decision.

> 
> I never saw  xenomai support announced, by the way.

Not officially, but I talked to them twice at some fairs, and they
confirmed to work on it. For me it appears logical when looking at their
board portfolio and given that RTAI is practically bound to x86 now.

Jan


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 250 bytes --]