Re: [Xenomai-help] gatekeeper/0

Re: [Xenomai-help] gatekeeper/0

[Gilles Chanteperdrix]

henning.richter@domain.hid wrote:
> So I will configure the kernel once more. This time with the switchtest
> option.
> But what I am still wondering about is that the kernel oops is not
> regularly when I am running
> my application. Sometimes there is no problem running the application with
> prio 99 a few times (e.g. 10 times)
> and gathering information from /xenomai/stat and/or /sched etc. (also in
> graphical mode)
> (I also run my application in console mode and nothing went wrong.)

You mean that you get crashes only in graphic mode ?

-- 
                                                 Gilles.

[Xenomai-help] gatekeeper/0

[Waldemar Valdas Bancewicz]

Hi,

I am getting an abnormally high cpu utilization (30 - 85%) for the 
gatekeeper/0 thread. I'm using Xenomai 2.4.10 with a Debian linux kernel 
2.6.32. The problem appears after upgrading the kernel with newer debian 
patches. Any ideas?

Thanks.

Re: [Xenomai-help] gatekeeper/0

[Philippe Gerum]

On Tue, 2011-01-18 at 15:23 -0500, Waldemar Valdas Bancewicz wrote:
> Hi,
> 
> I am getting an abnormally high cpu utilization (30 - 85%) for the 
> gatekeeper/0 thread. I'm using Xenomai 2.4.10 with a Debian linux kernel 
> 2.6.32. The problem appears after upgrading the kernel with newer debian 
> patches. Any ideas?

That is an interesting question, but I'm unsure that you will get a
definite answer, at least because of the following reasons:

- we have absolutely no clue about what those "newer debian patches"
are, we track the mainline kernel only.

- we don't know what your application does, what were the CPU figures
before you changed your configuration, etc.

- you are running 2.4.10, which is seriously outdated, and you did not
tell anything about the I-pipe release you are running, albeit this is
likely crucial for your issue. Actually we don't even know what
architecture you are running.

Wild guess:

- your RT code is probably doing a massive number of mode switches to
secondary mode. You may want to check whether the value of the CSW
counter from /proc/xenomai/stat increases at the same pace depending on
the version you are testing.

- those patches somehow broke the host clock tick accounting code of the
I-pipe, and the gatekeeper is unduly charged with all Linux timer ticks
which happened while your system was running in primary mode, as soon as
it returns to secondary mode. In such a case, the longer the time spent
in primary mode, the higher the likeliness to have deferred Linux ticks
charged this way.

> 
> Thanks.
> 
> _______________________________________________
> Xenomai-help mailing list
> Xenomai-help@domain.hid
> https://mail.gna.org/listinfo/xenomai-help

-- 
Philippe.

Re: [Xenomai-help] gatekeeper/0

[Gilles Chanteperdrix]

henning.richter@domain.hid wrote:
> Hi Gilles, Hi list,
> 
> nearly everything is working fine. I rebuild the kernel again without the
> given options and it seems to work.
> Even the latencies look quite much better. There is only one problem left
> that bothers me.
> 
> Every time I load the rt_eepro100 for my real-time device I get the
> message:
> 
> [   95.753152]   *** Receiver lock-up bug detected ***
> [   95.753154]   Your device may not work reliably!
> 
> and sometimes the systems hangs (and can not be woken up) at this point.
> 
> I know that this is a hint for a irq problem but I don't know where this
> should be.
> My eth0 device (irq6) is not sharing this irq with an other deveice.
> 
> IRQs after boot:

Hi,

when your question is new, please post a new message instead of replying
to an old thread. The person likely to answer your question probably did
not even read your message because it is buried so deep in a thread
which I handled until now.

Regards.

-- 
                                                 Gilles.

Re: [Xenomai-help] gatekeeper/0

[Gilles Chanteperdrix]

Somehow, the xenomai-help mailing list does not receive your mails. So,
I am resending your mail to the list.


Hi Gilles, Hi list,

nearly everything is working fine. I rebuild the kernel again without the
given options and it seems to work.
Even the latencies look quite much better. There is only one problem left
that bothers me.

Every time I load the rt_eepro100 for my real-time device I get the
message:

[   95.753152]   *** Receiver lock-up bug detected ***
[   95.753154]   Your device may not work reliably!

and sometimes the systems hangs (and can not be woken up) at this point.

I know that this is a hint for a irq problem but I don't know where this
should be.
My eth0 device (irq6) is not sharing this irq with an other deveice.

IRQs after boot:

         CPU0
  0:         40    XT-PIC-XT        timer
  1:         52    XT-PIC-XT        i8042
  2:          0    XT-PIC-XT        cascade
  3:          2    XT-PIC-XT
  4:        280    XT-PIC-XT        serial
  5:       6179    XT-PIC-XT        uhci_hcd:usb2, eth3,
i915@domain.hid
  6:         44    XT-PIC-XT        eth0
  7:       1293    XT-PIC-XT        ehci_hcd:usb1
  8:          2    XT-PIC-XT        rtc0
  9:         11    XT-PIC-XT        acpi
 10:          2    XT-PIC-XT
 11:          2    XT-PIC-XT
 12:        951    XT-PIC-XT        i8042
 14:       7734    XT-PIC-XT        ata_piix
 15:         40    XT-PIC-XT        uhci_hcd:usb3, ata_piix, eth1, eth2

   IRQs after I added the real-time device:

          CPU0
  0:         40    XT-PIC-XT        timer
  1:        131    XT-PIC-XT        i8042
  2:          0    XT-PIC-XT        cascade
  3:          2    XT-PIC-XT
  4:        280    XT-PIC-XT        serial
  5:      21773    XT-PIC-XT        uhci_hcd:usb2, i915@domain.hid,
eth3
  6:         71    XT-PIC-XT
  7:       2334    XT-PIC-XT        ehci_hcd:usb1
  8:          2    XT-PIC-XT        rtc0
  9:         17    XT-PIC-XT        acpi
 10:          2    XT-PIC-XT
 11:          2    XT-PIC-XT
 12:       3771    XT-PIC-XT        i8042
 14:      11960    XT-PIC-XT        ata_piix
 15:         66    XT-PIC-XT        uhci_hcd:usb3, ata_piix, eth1, eth2

cat /proc/xenomai/irq
IRQ         CPU0
  6:           1         rt_eepro100
230:           0         [IPI]
233:      170615         [timer]
234:           0         [critical sync]
258:           2         [virtual]

Re: [Xenomai-help] gatekeeper/0

[Gilles Chanteperdrix]

henning.richter@domain.hid wrote:
>> And then tries to access the same room on the stack believing that the
>> frame pointer or pc is stored there. Game over.
>>
>> Modify profile_pc function declaration to add the "notrace" qualifier.
>>
>> This bug is a red herring, this is a simple effect of enabling the
> tracing.
> 
> --
>                                                  Gilles.
> 
> Can you give me a hint how to do that or is there sth. to read about?
> 
> Henning

Try:

diff --git a/arch/x86/include/asm/ptrace.h b/arch/x86/include/asm/ptrace.h
index 6d34d95..abfd422 100644
--- a/arch/x86/include/asm/ptrace.h
+++ b/arch/x86/include/asm/ptrace.h
@@ -134,7 +134,7 @@ struct pt_regs {
 struct cpuinfo_x86;
 struct task_struct;

-extern unsigned long profile_pc(struct pt_regs *regs);
+extern unsigned long notrace profile_pc(struct pt_regs *regs);

 extern unsigned long
 convert_ip_to_linear(struct task_struct *child, struct pt_regs *regs);

If that is not enough, add notrace where profile_pc is implemented too.

-- 
                                                 Gilles.

Re: [Xenomai-help] gatekeeper/0

[Gilles Chanteperdrix]

henning.richter@domain.hid wrote:
>> And then tries to access the same room on the stack believing that the
>> frame pointer or pc is stored there. Game over.
>>
>> Modify profile_pc function declaration to add the "notrace" qualifier.
>>
>> This bug is a red herring, this is a simple effect of enabling the
> tracing.
> 
> --
>                                                  Gilles.
> 
> Can you give me a hint how to do that or is there sth. to read about?

But you can also try to disable all the tracing and profiling options I
gave you in two different mails and that slow down your system anyway.

-- 
                                                 Gilles.

Re: [Xenomai-help] gatekeeper/0

[Gilles Chanteperdrix]

henning.richter@domain.hid wrote:
> 
> It really is the first bug.
> 
> [ 2510.572237] BUG: unable to handle kernel paging request at b807a7fc
> [ 2510.576044] IP: [<c01070c6>] profile_pc+0x46/0x50
> [ 2510.576044] Oops: 0000 [#1] SMP
> [ 2510.576044] Modules linked in: nls_iso8859_1 nls_cp437 vfat fat e100
> rt_eepro100 rtpacket rtnet af_packet i915 drm bridge stp bnep rfcomm sco
> l2cap bluetooth ppdev ipv6 iptable_filter ip_tables x_tables parport_pc lp
> parport sr_mod cdrom serio_raw evdev psmouse com20020_pci com20020 arcnet
> iTCO_wdt iTCO_vendor_support shpchp intel_agp pci_hotplug agpgart ext3 jbd
> mbcache sd_mod crc_t10dif sg usb_storage libusual ata_piix ata_generic
> libata mii uhci_hcd ehci_hcd scsi_mod dock usbcore fuse [last unloaded:
> e100]
> 
> 
> here the corresponding disassembly:
> 
> c0107080 <profile_pc>:
> c0107080:      55                      push   %ebp
> c0107081:      89 e5                   mov    %esp,%ebp
> c0107083:      83 ec 08                sub    $0x8,%esp
> c0107086:      89 1c 24                mov    %ebx,(%esp)
> c0107089:      89 74 24 04             mov    %esi,0x4(%esp)

Ok. So profile_pc reserves some room on the stack for mcount arguments...

> c010708d:      e8 da 24 01 00          call   c011956c <mcount>
> c0107092:      f6 40 36 02             testb  $0x2,0x36(%eax)
> c0107096:      8b 70 2c                mov    0x2c(%eax),%esi
> c0107099:      89 c3                   mov    %eax,%ebx
> c010709b:      75 0d                   jne    c01070aa <profile_pc+0x2a>
> c010709d:      8b 40 30                mov    0x30(%eax),%eax
> c01070a0:      25 fc 00 00 00          and    $0xfc,%eax
> c01070a5:      83 f8 60                cmp    $0x60,%eax
> c01070a8:      74 0e                   je     c01070b8 <profile_pc+0x38>
> c01070aa:      89 f0                   mov    %esi,%eax
> c01070ac:      8b 1c 24                mov    (%esp),%ebx
> c01070af:      8b 74 24 04             mov    0x4(%esp),%esi

And then tries to access the same room on the stack believing that the
frame pointer or pc is stored there. Game over.

Modify profile_pc function declaration to add the "notrace" qualifier.

This bug is a red herring, this is a simple effect of enabling the tracing.

However, it would be nice if you could recompile the kernel without the
following configuration options:
CONFIG_PROFILING
CONFIG_MARKERS
CONFIG_OPROFILE
CONFIG_KPROBES
CONFIG_KRETPROBES


-- 
                                                 Gilles.

Re: [Xenomai-help] gatekeeper/0

[Gilles Chanteperdrix]

Gilles Chanteperdrix wrote:
> henning.richter@domain.hid wrote:
>> It really is the first bug.
>>
>> [ 2510.572237] BUG: unable to handle kernel paging request at b807a7fc
>> [ 2510.576044] IP: [<c01070c6>] profile_pc+0x46/0x50
>> [ 2510.576044] Oops: 0000 [#1] SMP
>> [ 2510.576044] Modules linked in: nls_iso8859_1 nls_cp437 vfat fat e100
>> rt_eepro100 rtpacket rtnet af_packet i915 drm bridge stp bnep rfcomm sco
>> l2cap bluetooth ppdev ipv6 iptable_filter ip_tables x_tables parport_pc lp
>> parport sr_mod cdrom serio_raw evdev psmouse com20020_pci com20020 arcnet
>> iTCO_wdt iTCO_vendor_support shpchp intel_agp pci_hotplug agpgart ext3 jbd
>> mbcache sd_mod crc_t10dif sg usb_storage libusual ata_piix ata_generic
>> libata mii uhci_hcd ehci_hcd scsi_mod dock usbcore fuse [last unloaded:
>> e100]
>>
>>
>> here the corresponding disassembly:
>>
>> c0107080 <profile_pc>:
>> c0107080:      55                      push   %ebp
>> c0107081:      89 e5                   mov    %esp,%ebp
>> c0107083:      83 ec 08                sub    $0x8,%esp
>> c0107086:      89 1c 24                mov    %ebx,(%esp)
>> c0107089:      89 74 24 04             mov    %esi,0x4(%esp)
> 
> Ok. So profile_pc reserves some room on the stack for mcount arguments...
> 
>> c010708d:      e8 da 24 01 00          call   c011956c <mcount>
>> c0107092:      f6 40 36 02             testb  $0x2,0x36(%eax)
>> c0107096:      8b 70 2c                mov    0x2c(%eax),%esi
>> c0107099:      89 c3                   mov    %eax,%ebx
>> c010709b:      75 0d                   jne    c01070aa <profile_pc+0x2a>
>> c010709d:      8b 40 30                mov    0x30(%eax),%eax
>> c01070a0:      25 fc 00 00 00          and    $0xfc,%eax
>> c01070a5:      83 f8 60                cmp    $0x60,%eax
>> c01070a8:      74 0e                   je     c01070b8 <profile_pc+0x38>
>> c01070aa:      89 f0                   mov    %esi,%eax
>> c01070ac:      8b 1c 24                mov    (%esp),%ebx
>> c01070af:      8b 74 24 04             mov    0x4(%esp),%esi
> 
> And then tries to access the same room on the stack believing that the
> frame pointer or pc is stored there. Game over.

This is bullsh*t, I tried to give an answer too fast, so please try to
disable the options I gave you.

-- 
                                                 Gilles.

Re: [Xenomai-help] gatekeeper/0

[Gilles Chanteperdrix]

henning.richter@domain.hid wrote:
> So here is what I got via serial console when my application causes that
> freeze:

Ok. Is it the first bug? If yes, could you send us a disassembly of the
profile_pc function? (run objdump vmlinux, then look for profile_pc, and
only copy this function in the mail).

-- 
                                                 Gilles.

Re: [Xenomai-help] gatekeeper/0

[Gilles Chanteperdrix]

henning.richter@domain.hid wrote:
> So here are some more results.
> The switchtest is now working with the enabled kernel option.
> My adeos Ipipe version is 2.2-06.
> I run both, latency and switchtest, in graphical mode.
> I send you my .config in a seperate mail, as there where problem the last
> time.

Ok. The working switchtest is a good news.
The latency results are a bit high, if they are taken on an idle system
(only latencies taken on a busy system are meaningful), do not you get a
message in kernel console talking about SMIs ? In any case, we do not
see the same results as your application, so, the cause of the latencies
observed by your application is not Xenomai latencies.

and:
> Here the .config as plain text.

Nope, your mailer still sends your mail with twice its body: once in
plain text, once in HTML. This is pure bandwith waste for the few
hundred people subscribed to the list, and this makes people think that
your mail is spam, because spam is usually HTML too. Please switch your
mailer to plain text instead.

As for the config itself you should disable CONFIG_X86_PM_TIMER.


and also:
> Things are still strange.
> The 'freeze' still appears sporadically, but only in graphical mode.
> When I login via console I don't get any freeze.

Ok. As I already told you, if when you see freezes the keyboard leds
blink, it means that what you have is a BUG, an oops, or a kernel panic.
In any case, the kernel console will tell us what happens. Try to get
the kernel console through a serial cable, or if not possible using the
less reliable netconsole.

-- 
					    Gilles.

Re: [Xenomai-help] gatekeeper/0

[Gilles Chanteperdrix]

henning.richter@domain.hid wrote:
> Things are strange. Now my application works without crashing. I haven't
> changed anything.
> Btw. what should be a "normal" value of /proc/xenomai/latency ?
> I've got 1676

/proc/xenomai/latency is the anticipation of the Xenomai timer. It is
actually a value for you to tweak with a default value that does not
cause any pathological behaviour.

> 
> Back to my main problem:
> Sending / Receiving Frames is still too slow. At a framesize of around 700
> byte it takes around
> 350 us before the next frame is send. And this at a given cycle time of 250
> us at a priority of 90.
> All printf's and other logs are removed. No Modeswitches are done during
> run of my application.
> Btw. what counts CSW at /proc/xenomai/stat ?

CSW is context switches.
But IMO, first things should be done first, and you should investigate
the crashes. And it would be still be interesting to know if switchtest
works on your platform.

As for the latency, as I think I already told you, you should run the
latency test at the same frequency as the one of your application with
which you see problems. If you see problems with the latency test, then
Xenomai has a problem on your platform, otherwise, the problem is in
your application.

> 
> 
>    ethercat@domain.hid:~$ cat /proc/xenomai/sched
> 
>    CPU         PID    PRI         PERIOD           TIMEOUT    TIMEBASE
> STAT       NAME
>          0     0           -1          0           0           master    R
> ROOT/0
>          0     0           98          0           0           master    W
> rtnet-stack
>          0     0           0                 0           0           master
> W          rtnet-rtpc
>          0     5259     0              0                 0
> master    X          master_test
>          0     5260    90              249752     1109638            master
> w          myRtTask
> 
> 
> 
>    ethercat@domain.hid:~$ cat /proc/xenomai/stat
>    CPU  PID          MSW        CSW        PF    STAT       %CPU  NAME
>      0         0           0          436897       0     00500080   74.9
> ROOT/0
>      0   0           0          435422       0     00000082    1.0
> rtnet-stack
>      0         0           0          1            0     00000082    0.0
> rtnet-rtpc
>      0   5259   1          1                       0     00300380    0.0
> master_test
>      0         5260   24         59775             0     00300186    0.1
> myRtTask
>      0         0           0          870784       0     00000000    3.6
> IRQ6: rt_eepro100
>      0         0           0          527744       0     00000000    2.6
> IRQ233: [timer]
> 
> I think I will start tracing with the ipipe tracer.

The I-pipe tracer will not help you understand what happens in
user-space. For this, you are probably better of measuring things with
rt_timer_tsc() or clock_gettime().

-- 
                                                 Gilles.

Re: [Xenomai-help] gatekeeper/0

[Gilles Chanteperdrix]

henning.richter@domain.hid wrote:
> Beside I actually don't use xconfig because QT3 is not installed
> my application runs well in console mode and I am able to gather
> information from /proc/xenomai/stat etc.

In a previous mail you told us that you were running your application in
graphical mode. If you are running in console mode, you should see the
oops (though it may not be easy to capture, and you may need to actually
increase the console resolution through kernel command line to see the
message entirely, some people in the past send us numerical photos of
the screen to show us their oops...).

As for not being able to use switchtest, sorry, but if you do not want
to use the graphical tool, you have to learn how to search in text mode,
this means find . -name 'Kconfig*' | xargs grep switchtest...

-- 
                                                 Gilles.

Re: [Xenomai-help] gatekeeper/0

[Gilles Chanteperdrix]

henning.richter@domain.hid wrote:
> In which submenu is this option hidden?

It is not hidden. Besides xconfig has a "search" functionality.

-- 
                                                 Gilles.

Re: [Xenomai-help] gatekeeper/0

[Gilles Chanteperdrix]

henning.richter@domain.hid wrote:
> Outputs of the switchtest:
> 
> == FPU check routines: OK!.
> switchtest: Unabale to open switchtest device.
> (modprobe xeno_switchtest ? )
> 
> modeprobe xeno_switchtest leads to FATAL: Module xeno_switchtest not found.

I do not see anything in this message related to the number of CPUs in
your system. You should simply enable the switchtest module in Xenomai
configuration (in the kernel configuration menus).

-- 
                                                 Gilles.

Re: [Xenomai-help] gatekeeper/0

[Gilles Chanteperdrix]

henning.richter@domain.hid wrote:
> Running latency is fine.

And switchtest ?

-- 
                                                 Gilles.

Re: [Xenomai-help] gatekeeper/0

[henning.richter]

[-- Attachment #1: Type: text/plain, Size: 208 bytes --]


My machine only has one CPU.
So switchtest doesn't work.

Henning

henning.richter@domain.hid wrote:
> Running latency is fine.

And switchtest ?

--
                                                 Gilles.

[-- Attachment #2: Type: text/html, Size: 423 bytes --]

Re: [Xenomai-help] gatekeeper/0

[Gilles Chanteperdrix]

henning.richter@domain.hid wrote:
> My machine only has one CPU.
> So switchtest doesn't work.

switchtest works on machine with only one CPU. What is the exact error
message you get ?

-- 
                                                 Gilles.

Re: [Xenomai-help] gatekeeper/0

[Gilles Chanteperdrix]

henning.richter@domain.hid wrote:
> Now I'm using xenomai 2.4.7 on kernel 2.6.27.19.
> 
> The sigxcpu example from xenomai is now working correctly.
> 
> After removing some outputs there are no more modeswitches in my task,
> apart from some debugging output while initilization.
> 
> But now my system is no longer stable. It freezes.

Show us /proc/interrupts, /proc/xenomai/irq, your .config if it changed.

What kind of freeze is this? Do you run in graphical or in text mode? If
in graphical mode, try and get the console output using serial console
or netconsole.

Does the system restart when you hit the keyboard?

Does the system freeze when using the latency test alone? Same question
with switchtest?

-- 
                                                 Gilles.

Re: [Xenomai-help] gatekeeper/0

[henning.richter]

[-- Attachment #1: Type: text/plain, Size: 2137 bytes --]


I'm running in graphical mode (Kubuntu 8.10 with Kde4).
Now it freezes while 'cat /proc/xenomai/sched'
The Scroll/Lock -LED and the Caps-Lock-Led were flashing.

cat /proc/xenomai/irq         (while running my rt task)
IRQ   CPU0
6:    19144 rt_eepro_100
230:  0     [IPI]
233:  68798 [timer]
234:  0     [critical sync]
258:  83    [virtual]

cat /proc/interrupts          (while running my rt task)
      CPU0
0:    40    XT-PIC-XT   timer
1:    411   XT-PIC-XT   i8042
2:    0     XT-PIC-XT   cascade
3:    2     XT-PIC-XT
4:    2     XT-PIC-XT
5:    54863 XT-PIC-XT   uhci_hcd:usb2, i915@domain.hid, eth3
6:    140   XT-PIC-XT
7:    6380  XT-PIC-XT   ehci_hcd:usb1
8:    2     XT-PIC-XT   rtc0
9:    20    XT-PIC-XT   acpi
10:   2     XT-PIC-XT
11:   2     XT-PIC-XT
12:   4403  XT-PIC-XT   i8042
14:   9713  XT-PIC-XT   ata_piix
15:   27142 XT-PIC-XT   uhci_hcd:usb3, ata_piix, eth1, eth2
NMI:  0     Non-maskable interrupts
LOC:  61661 Local timer interrupts
RES:  0     Rescheduling interrupts
CAL:  0     function call interrupts
TLB:  0     TLB shootdowns
SPU   0     Spurious interrupts
ERR:  0
MIS:  0


A few seconds after i gathered this information the system freezes again.
The .config is the same as yesterday with one change -> ACPI is now enabled
and the
ACPI_PROCESSOR disabled.
I now try to gather more information via netconsole.


Henning




henning.richter@domain.hid wrote:
> Now I'm using xenomai 2.4.7 on kernel 2.6.27.19.
>
> The sigxcpu example from xenomai is now working correctly.
>
> After removing some outputs there are no more modeswitches in my task,
> apart from some debugging output while initilization.
>
> But now my system is no longer stable. It freezes.

Show us /proc/interrupts, /proc/xenomai/irq, your .config if it changed.

What kind of freeze is this? Do you run in graphical or in text mode? If
in graphical mode, try and get the console output using serial console
or netconsole.

Does the system restart when you hit the keyboard?

Does the system freeze when using the latency test alone? Same question
with switchtest?

--
                                                 Gilles.

[-- Attachment #2: Type: text/html, Size: 2509 bytes --]

Re: [Xenomai-help] gatekeeper/0

[Gilles Chanteperdrix]

henning.richter@domain.hid wrote:
> I'm running in graphical mode (Kubuntu 8.10 with Kde4).
> Now it freezes while 'cat /proc/xenomai/sched'
> The Scroll/Lock -LED and the Caps-Lock-Led were flashing.

Ok. It is a kernel oops then. Do you get it when simply running the
latency or switchtest test instead of your application ?

-- 
                                                 Gilles.

[Xenomai-help] gatekeeper/0

[henning.richter]

[-- Attachment #1: Type: text/plain, Size: 333 bytes --]


Hi,

running a xenomai task in which I am sending a single Frame (size ~500
byte) from my EtherCAT Master to some Beckhoff slaves has latencies around
200 us.
While running that task, top shows that gatekeeper/0 (with RT priority) has
30% CPU usage.
What is the task of this gatekeeper and can this cause my high latencies?

Henning

[-- Attachment #2: Type: text/html, Size: 517 bytes --]

Re: [Xenomai-help] gatekeeper/0

[Gilles Chanteperdrix]

henning.richter@domain.hid wrote:
> Hi,
> 
> running a xenomai task in which I am sending a single Frame (size ~500
> byte) from my EtherCAT Master to some Beckhoff slaves has latencies around
> 200 us.
> While running that task, top shows that gatekeeper/0 (with RT priority) has
> 30% CPU usage.
> What is the task of this gatekeeper and can this cause my high latencies?

The gatekeeper is used when a taks is migrating from secondary mode to
primary mode. The gatekeeper itself can not cause the high latencies,
but the fact that a task runs in secondary mode can.

To check if this is what happens, check the MSW column in
/proc/xenomai/stat for the thread which has latencies issues. If MSW is
increasing then you have a problem.

To find where your thread switches to secondary mode, use the technique
demonstrated in examples/native/sigxcpu.c.

-- 
                                                 Gilles.

Re: [Xenomai-help] gatekeeper/0

[henning.richter]

[-- Attachment #1: Type: text/plain, Size: 1256 bytes --]




henning.richter@domain.hid wrote:
>> Hi,
>>
>> running a xenomai task in which I am sending a single Frame (size ~500
>> byte) from my EtherCAT Master to some Beckhoff slaves has latencies
around
>> 200 us.
>> While running that task, top shows that gatekeeper/0 (with RT priority)
has
>> 30% CPU usage.
>> What is the task of this gatekeeper and can this cause my high
latencies?
>
> The gatekeeper is used when a taks is migrating from secondary mode to
> primary mode. The gatekeeper itself can not cause the high latencies,
> but the fact that a task runs in secondary mode can.
>
> To check if this is what happens, check the MSW column in
> /proc/xenomai/stat for the thread which has latencies issues. If MSW is
> increasing then you have a problem.

> To find where your thread switches to secondary mode, use the technique
> demonstrated in examples/native/sigxcpu.c.

--
                                                 Gilles.

So thank your for defining my problem.
Now I am sure that there my thread is switching to secondary mode
as the MSW column increases rapidly while running my application.
Can I also use the ipipe tracer or LTTng to find the place where it
switches the modes as I don't have the
xenomai examples installed.

Henning

[-- Attachment #2: Type: text/html, Size: 1613 bytes --]

Re: [Xenomai-help] gatekeeper/0

[Gilles Chanteperdrix]

henning.richter@domain.hid wrote:
> 
> 
> henning.richter@domain.hid wrote:
>>> Hi,
>>>
>>> running a xenomai task in which I am sending a single Frame (size ~500
>>> byte) from my EtherCAT Master to some Beckhoff slaves has latencies
> around
>>> 200 us.
>>> While running that task, top shows that gatekeeper/0 (with RT priority)
> has
>>> 30% CPU usage.
>>> What is the task of this gatekeeper and can this cause my high
> latencies?
>> The gatekeeper is used when a taks is migrating from secondary mode to
>> primary mode. The gatekeeper itself can not cause the high latencies,
>> but the fact that a task runs in secondary mode can.
>>
>> To check if this is what happens, check the MSW column in
>> /proc/xenomai/stat for the thread which has latencies issues. If MSW is
>> increasing then you have a problem.
> 
>> To find where your thread switches to secondary mode, use the technique
>> demonstrated in examples/native/sigxcpu.c.
> 
> --
>                                                  Gilles.
> 
> So thank your for defining my problem.
> Now I am sure that there my thread is switching to secondary mode
> as the MSW column increases rapidly while running my application.
> Can I also use the ipipe tracer or LTTng to find the place where it
> switches the modes as I don't have the
> xenomai examples installed.

Yes, you can use the I-pipe tracer, but reading the output might be a
bit complicated. Examples are part of Xenomai sources freely available
on Xenomai web site.

-- 
                                                 Gilles.

Re: [Xenomai-help] gatekeeper/0

[henning.richter]

[-- Attachment #1: Type: text/plain, Size: 279 bytes --]

Hi Gilles,

I tried to run, after a simple make, sigxcpu.
The result:
./sigxcpu[0x80488337]
[0xb800c400]
/lib/tls/i686/cmov/libphtread.so.0[0xb7fcc50f]
/lib/tls/i686/cmov/libc.so.6(clone+0x5e)[0xb7f417ee]

This is where my system freezes or simply reboots.

Am I doing sth wrong?

[-- Attachment #2: Type: text/html, Size: 588 bytes --]

Re: [Xenomai-help] gatekeeper/0

[Gilles Chanteperdrix]

henning.richter@domain.hid wrote:
> Hi Gilles,
> 
> I tried to run, after a simple make, sigxcpu.
> The result:
> ./sigxcpu[0x80488337]
> [0xb800c400]
> /lib/tls/i686/cmov/libphtread.so.0[0xb7fcc50f]
> /lib/tls/i686/cmov/libc.so.6(clone+0x5e)[0xb7f417ee]
> 
> This is where my system freezes or simply reboots.
> 
> Am I doing sth wrong?

You are probably setting the warning bit too early and/or for the wrong
thread. You should set the bit only for the thread which is expected to
run in primary mode, and only after it has done its non primary mode
duties (like creating new threads).

However, you should not get freezes or reboots. Any trace on the console ?

-- 
                                                 Gilles.

Re: [Xenomai-help] gatekeeper/0

[Gilles Chanteperdrix]

henning.richter@domain.hid wrote:
> Hi Gilles,
> 
> I tried to run, after a simple make, sigxcpu.
> The result:
> ./sigxcpu[0x80488337]
> [0xb800c400]
> /lib/tls/i686/cmov/libphtread.so.0[0xb7fcc50f]
> /lib/tls/i686/cmov/libc.so.6(clone+0x5e)[0xb7f417ee]
> 
> This is where my system freezes or simply reboots.
> 
> Am I doing sth wrong?

What version of Xenomai are you using ?

-- 
                                                 Gilles.