RT : 2.6.12rc5 + realtime-preempt-2.6.12-rc5-V0.7.47-15

RT : 2.6.12rc5 + realtime-preempt-2.6.12-rc5-V0.7.47-15

[Serge Noiraud]

[-- Attachment #1: Type: text/plain, Size: 496 bytes --]

I have a test program which made a loop in RT to mesure the system
perturbation.
It works finely in a tty environment.
When I run it in an X environment ( xterm ), I get something like if I
click the Enter key in the active window.
If I open a new xterm, this is the new active window which receive these
events.
These events stop when the program stop.

I tried with X in RT and no RT : I have the problem.

I send you the program in copy to reproduce.
I have this problem in all version of RT.

[-- Attachment #2: test_tsc.c --]
[-- Type: text/x-c, Size: 2401 bytes --]

#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include "fonctions.h"

#include <stdio.h>
#include <unistd.h>
#include <sys/mman.h> /* for mmap and MCL_CURRENT */
#include <stdlib.h> /* for atoi */
#include <asm/msr.h> /* for rdtsc */

#define __USE_GNU 
#include <sched.h> /* for sched_xxxx */

//#define NB_TEST 60000000000ULL
#define NB_TEST 60000000ULL

float get_cpu_clock_speed()
{
  FILE *fp;
  char buffer[1024];
  size_t bytes_read;
  char *match;
  float clock_speed;

  fp = fopen ("/proc/cpuinfo", "r");
  bytes_read = fread (buffer, 1, sizeof (buffer), fp);
  fclose (fp);
  if (bytes_read == 0 || bytes_read == sizeof (buffer))
    return 0;
  buffer[bytes_read] = '\0';
  match = strstr (buffer, "cpu MHz");
  if (match == NULL)
    return 0;
  sscanf (match, "cpu MHz : %f", &clock_speed);
  return clock_speed;
}

int main(int argc,char **argv)
{
 unsigned long long max,dt;
 float frequency;
 int cptr;
 long long i;
 struct sched_param sched_param;
 double duree;
 union {
   unsigned long long total;
   struct {
     unsigned long MSL;
     unsigned long LSL;
   };
 } tempsdeb,tempsfin,t1,t2,tmax1,tmax2;
 cpu_set_t new_mask;
 pid_t p = 0;
 int ret;

  if (argc!=2)
  {
     fprintf(stderr,"Usage:%s <NUM_CPU>\n",argv[0]);
     return -1;
  }
  CPU_ZERO(&new_mask);
  CPU_SET(atoi(argv[1]),&new_mask);
  ret = sched_setaffinity(p,sizeof(cpu_set_t) , &new_mask);
  rdtsc(tempsdeb.MSL,tempsdeb.LSL);
  printf(" Start time %Lx \n",tempsdeb.total);
  sched_param.sched_priority =99;
  cptr = sched_setscheduler(getpid(), SCHED_FIFO,&sched_param );
  cptr = mlockall(MCL_CURRENT);
  printf("mlockall cptr %d\n",cptr);
 
 max=0;
 rdtsc(t1.MSL,t1.LSL);
 for (i = 0; i < NB_TEST; i++)
   {
	 rdtsc(t2.MSL,t2.LSL);
	 dt = t2.total - t1.total;
	 if ( dt > max) 
	 { max = dt;
	  tmax1.total = t1.total;
	  tmax2.total = t2.total;
	  }
	 t1.total = t2.total;
   }
 rdtsc(tempsfin.MSL,tempsfin.LSL);
 printf(" End time  %Lx \n",tempsfin.total);
 printf(" Max time 1  %Lx \n",tmax1.total);
 printf(" Max time 2  %Lx \n",tmax2.total);
 frequency = get_cpu_clock_speed();
  if (frequency == 0)
	  return -1;
 duree =((double) (tempsfin.total - tempsdeb.total))/(frequency*1000000);
 printf("Test duration is %f s max detected %.0f µs\n",duree,((double)max)/(frequency));

 return 0;
}

Re: RT : 2.6.12rc5 + realtime-preempt-2.6.12-rc5-V0.7.47-15

[Lee Revell]

On Tue, 2005-05-31 at 16:53 +0200, Serge Noiraud wrote:
> I have a test program which made a loop in RT to mesure the system
> perturbation.
> It works finely in a tty environment.
> When I run it in an X environment ( xterm ), I get something like if I
> click the Enter key in the active window.
> If I open a new xterm, this is the new active window which receive these
> events.
> These events stop when the program stop.
> 
> I tried with X in RT and no RT : I have the problem.

Try adding:

Option "NoAccel"

to the Driver section of your X config.

Some buggy video drivers can stall the PCI bus for tens or hundreds of
*milliseconds*.  The "via" driver had this problem until I identified
the problem and the Unichrome guys fixed it.  If it goes away with
"NoAccel", then you are having the same problem.

For details search the unichrome-devel archives for "losing interrupts".

Lee

Re: RT : 2.6.12rc5 + realtime-preempt-2.6.12-rc5-V0.7.47-15

[Serge Noiraud]

Le mar 31/05/2005 à 21:47, Lee Revell a écrit :
> On Tue, 2005-05-31 at 16:53 +0200, Serge Noiraud wrote:
> > I have a test program which made a loop in RT to mesure the system
> > perturbation.
> > It works finely in a tty environment.
> > When I run it in an X environment ( xterm ), I get something like if I
> > click the Enter key in the active window.
> > If I open a new xterm, this is the new active window which receive these
> > events.
> > These events stop when the program stop.
> > 
> > I tried with X in RT and no RT : I have the problem.
> 
> Try adding:
> 
> Option "NoAccel"
Same problem.
I have this problem with two graphic card on three different machines :
On the fisrt one  ( IBM Intellistation Mpro ) :
#lspci
...
01:00.0 VGA compatible controller: nVidia Corporation NV28GL [Quadro4
980 XGL] (rev a1)
...
#lshw
...
        *-pci:0
             description: PCI bridge (Normal decode)
             product: 82875P Processor to AGP Controller
             vendor: Intel Corp.
             physical id: 1
             bus info: pci@00:01.0
             version: 02
             clock: 66MHz
             capabilities: pci normal_decode bus_master
           *-display UNCLAIMED
                description: VGA compatible controller (VGA)
                product: NV28GL [Quadro4 980 XGL]
                vendor: nVidia Corporation
                physical id: 0
                bus info: pci@01:00.0
                version: a1
                size: 128MB
                clock: 66MHz
                capabilities: vga bus_master cap_list
                resources: iomemory:f8000000-f8ffffff
iomemory:f0000000-f7ffffff irq:145
...

On the second machine ( IBM Intellistation Zpro ) :
#lspci
01:00.0 VGA compatible controller: nVidia Corporation NV28GL [Quadro4
980 XGL] (rev a1)
#lshw
        *-pci:0
             description: PCI bridge (Normal decode)
             product: E7505/E7205 PCI-to-AGP Bridge
             vendor: Intel Corp.
             physical id: 1
             bus info: pci@00:01.0
             version: 03
             clock: 66MHz
             capabilities: pci normal_decode bus_master cap_list
             resources: iomemory:f0000000-f3ffffff
           *-display
                description: VGA compatible controller (VGA)
                product: NV28GL [Quadro4 980 XGL]
                vendor: nVidia Corporation
                physical id: 0
                bus info: pci@01:00.0
                version: a1
                size: 128MB
                clock: 66MHz
                capabilities: vga bus_master cap_list
                configuration: driver=nvidia
                resources: iomemory:f8000000-f8ffffff
iomemory:e8000000-efffffff irq:201
...

On the third machine ( Force Computers - 2 ways ):
#lspci
...
00:05.0 VGA compatible controller: ATI Technologies Inc Rage XL (rev 27)
...
#lshw
...
     *-pci:0
          description: Host bridge
          product: CMIC-WS Host Bridge (GC-LE chipset)
          vendor: ServerWorks
          physical id: 100
          bus info: pci@00:00.0
          version: 13
          clock: 33MHz
        *-display UNCLAIMED
             description: VGA compatible controller (VGA)
             product: Rage XL
             vendor: ATI Technologies Inc
             physical id: 5
             bus info: pci@00:05.0
             version: 27
             size: 16MB
             clock: 33MHz
             capabilities: vga bus_master cap_list
             resources: iomemory:fb000000-fbffffff ioport:1000-10ff
iomemory:fc000000-fc000fff irq:9
...
> 
> to the Driver section of your X config.
> 
> Some buggy video drivers can stall the PCI bus for tens or hundreds of
> *milliseconds*.  The "via" driver had this problem until I identified
> the problem and the Unichrome guys fixed it.  If it goes away with
> "NoAccel", then you are having the same problem.
> 
> For details search the unichrome-devel archives for "losing interrupts".
I'll look at this :
Dragging window in X causes soundcard interrupts to be lost

They play with the mouse.
In my case, I do nothing. I launch the test then wait. I have <Enter
Key> events in my active window when I look at it the hand in the back!

> 
> Lee

Re: RT : 2.6.12rc5 + realtime-preempt-2.6.12-rc5-V0.7.47-15

[Ingo Molnar]

* Serge Noiraud <serge.noiraud@bull.net> wrote:

> Le mar 31/05/2005 à 21:47, Lee Revell a écrit :
> > On Tue, 2005-05-31 at 16:53 +0200, Serge Noiraud wrote:
> > > I have a test program which made a loop in RT to mesure the system
> > > perturbation.
> > > It works finely in a tty environment.
> > > When I run it in an X environment ( xterm ), I get something like if I
> > > click the Enter key in the active window.
> > > If I open a new xterm, this is the new active window which receive these
> > > events.
> > > These events stop when the program stop.
> > > 
> > > I tried with X in RT and no RT : I have the problem.
> > 
> > Try adding:
> > 
> > Option "NoAccel"
> Same problem.

could you enable latency timing and tracing in the .config:

 CONFIG_CRITICAL_PREEMPT_TIMING=y
 CONFIG_CRITICAL_IRQSOFF_TIMING=y
 CONFIG_LATENCY_TIMING=y
 CONFIG_LATENCY_TRACE=y

and start a new search for a maximum latency via:

 echo 0 > /proc/sys/kernel/preempt_max_latency

and then do the X test - what is the largest latency reported in 
'dmesg'? Also, please send me a (bzip2 -9 compressed, if too large) 
/proc/latency_trace trace output of the largest incident.

	Ingo

Re: RT : 2.6.12rc5 + realtime-preempt-2.6.12-rc5-V0.7.47-15

[Serge Noiraud]

[-- Attachment #1: Type: text/plain, Size: 1677 bytes --]

Le mer 01/06/2005 à 10:23, Ingo Molnar a écrit :
> * Serge Noiraud <serge.noiraud@bull.net> wrote:
> 
> > Le mar 31/05/2005 à 21:47, Lee Revell a écrit :
> > > On Tue, 2005-05-31 at 16:53 +0200, Serge Noiraud wrote:
> > > > I have a test program which made a loop in RT to mesure the system
> > > > perturbation.
> > > > It works finely in a tty environment.
> > > > When I run it in an X environment ( xterm ), I get something like if I
> > > > click the Enter key in the active window.
> > > > If I open a new xterm, this is the new active window which receive these
> > > > events.
> > > > These events stop when the program stop.
> > > > 
> > > > I tried with X in RT and no RT : I have the problem.
> > > 
> > > Try adding:
> > > 
> > > Option "NoAccel"
> > Same problem.
> 
> could you enable latency timing and tracing in the .config:
> 
>  CONFIG_CRITICAL_PREEMPT_TIMING=y
>  CONFIG_CRITICAL_IRQSOFF_TIMING=y
>  CONFIG_LATENCY_TIMING=y
>  CONFIG_LATENCY_TRACE=y
Already in the config. I have the following too :
CONFIG_CRITICAL_TIMING=y
> 
> and start a new search for a maximum latency via:
> 
>  echo 0 > /proc/sys/kernel/preempt_max_latency
> 
> and then do the X test - what is the largest latency reported in 
> 'dmesg'? Also, please send me a (bzip2 -9 compressed, if too large) 
> /proc/latency_trace trace output of the largest incident.
The max latency reported are normal.

I have the problem with another program which mesure latencies about
semaphore only in X environment too.
        
I tried to ssh this machine from another X environment and the problem
does not exist. It's only on the console.
> 
> 	Ingo

[-- Attachment #2: dmesg.txt.bz2 --]
[-- Type: application/x-bzip, Size: 5632 bytes --]

[-- Attachment #3: latency_trace.txt.bz2 --]
[-- Type: application/x-bzip, Size: 985 bytes --]

Re: RT : 2.6.12rc5 + realtime-preempt-2.6.12-rc5-V0.7.47-15

[Ingo Molnar]

* Serge Noiraud <serge.noiraud@bull.net> wrote:

> > and start a new search for a maximum latency via:
> > 
> >  echo 0 > /proc/sys/kernel/preempt_max_latency
> > 
> > and then do the X test - what is the largest latency reported in 
> > 'dmesg'? Also, please send me a (bzip2 -9 compressed, if too large) 
> > /proc/latency_trace trace output of the largest incident.
> The max latency reported are normal.

yeah. (btw., enable CONFIG_KALLSYMS to get a much more readable trace 
output)

> I have the problem with another program which mesure latencies about 
> semaphore only in X environment too.
>         
> I tried to ssh this machine from another X environment and the problem 
> does not exist. It's only on the console.

does your measurement program do any tty IO during the measurement? That 
could delay the measurement code artificially. Generally, latency 
measurement must be done very carefully (userspace and kernelspace 
alike).

E.g. rtc_wakeup uses a FIFO between two threads to isolate the 
measurement thread as much as possible. rtc_wakeup can be found at:

   http://www.affenbande.org/~tapas/wiki/index.php?rtc_wakeup

and should be run like:

  chrt -f 95 -p `pidof 'IRQ 8'`
  ./rtc_wakeup -f 1024 -t 100000

if you only trust your own measurement code then you can use a hybrid 
tracing method as well: switch the kernel's trace into 'user triggered':

  echo 1 > /proc/sys/kernel/trace_user_triggered

then you can turn tracing on in your code via a syscall hack:

	gettimeofday(0,1);

(yes - gettimeofday. Has nothing to do with tracing.) You can turn 
tracing off via:

	gettimeofday(0,0);

and the kernel will do a maximum search for you, and you should have the 
highest-latency trace available in /proc/latency_trace.

	Ingo

RT : 2.6.12rc5 + realtime-preempt-2.6.12-rc5-V0.7.47-15

[Serge Noiraud]

Could someone explains me the following error with : hda: lost interrupt
What can I do ?
When hda error occurs, the was one RT process doing nothing with disks.

following the end od dmesg

 (     ksoftirqd/0-3    |#0): new 15 us maximum-latency wakeup.
 (     ksoftirqd/1-6    |#1): new 16 us maximum-latency wakeup.
 (     ksoftirqd/1-6    |#1): new 20 us maximum-latency wakeup.
 (     ksoftirqd/0-3    |#0): new 22 us maximum-latency wakeup.
 (            bash-11002|#0): new 30 us maximum-latency wakeup.
 (             gpm-5508 |#0): new 31 us maximum-latency wakeup.
 hda: dma_timer_expiry: dma status == 0x24
 hda: DMA interrupt recovery
 hda: lost interrupt
 (     ksoftirqd/0-3    |#0): new 33 us maximum-latency wakeup.
 (       desched/0-4    |#0): new 37 us maximum-latency wakeup.
 (     ksoftirqd/0-3    |#0): new 48 us maximum-latency wakeup.