* [Xenomai] Xenomai/Cobalt on i7-3770S CPU
@ 2016-06-08 9:42 Heinick, J Michael
2016-06-08 19:39 ` Gilles Chanteperdrix
0 siblings, 1 reply; 8+ messages in thread
From: Heinick, J Michael @ 2016-06-08 9:42 UTC (permalink / raw)
To: xenomai@xenomai.org
We currently have an RTDM driver that is running well on Xenomai/Cobalt 3.0.1 on 2 Dell computers with Core2 processors, but will hang (unresponsive mouse and keyboard, no discernable activity) an entire SuperLogics i7 computer with a Core i7-3770S processor (4 physical cores, 4 logical cores). The hang occurs in the ioctl function at an rtdm_sem_down call that waits for the interrupt handler to signal the handling of an interrupt. We suspect that we have a problem with our kernel build/installation configuration options. We have attempted to configure the Core i7-3770S system so that Xenomai/Cobalt only uses 2 cores like the other 2 working Core2 computers, but the system still hangs (more detail on the results of our attempt is included beow). Eventually, we would like to configure Xenomai/Cobalt to run on 4 cores of the i7 computer if possible.
Any suggestions to help us make/install/configure Xenomai/Cobalt to run on the SuperLogics computer with the i7-3770S processor so that the rtdm_sem_down call in the RTDM driver does not hang the entire system would be appreciated.
Thank you for any help,
Mike
<<<<<<< Output from /lib/xenomai/bin/xeno-config -info: >>>>>>>>
Xenomai version: Xenomai/cobalt v3.0.1
Linux sgslrtest 3.14.44-RTneti10X301SomeD-kMod2cpu #6 SMP Fri Jun 3 11:06:41 EDT 2016 x86_64 x86_64 x86_64 GNU/Linux
Kernel parameters: BOOT_IMAGE=/vmlinuz-3.14.44-RTneti10X301SomeD-kMod2cpu root=UUID=2c093713-cc31-4dd0-9f1b-8fc31992825a ro quiet splash vt.handoff=7
I-pipe release #10 detected
Cobalt core 3.0.1 detected
Compiler: gcc version 4.9.1 (Ubuntu 4.9.1-16ubuntu6)
Build args: --with-core=cobalt --enable-smp --enable-pshared --host=x86_64-linux host_alias=x86_64-linux
Contents of selected files in the /proc directory:
cat /proc/cpuinfo >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
processor : 0
vendor_id : GenuineIntel
cpu family : 6
model : 58
model name : Intel(R) Core(TM) i7-3770S CPU @ 3.10GHz
stepping : 9
microcode : 0x1b
cpu MHz : 3100.102
cache size : 8192 KB
physical id : 0
siblings : 2
core id : 0
cpu cores : 2
apicid : 0
initial apicid : 0
fpu : yes
fpu_exception : yes
cpuid level : 13
wp : yes
flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf pni pclmulqdq dtes64 monitor ds_cpl vmx smx est tm2 ssse3 cx16 xtpr pdcm pcid sse4_1 sse4_2 x2apic popcnt tsc_deadline_timer aes xsave avx f16c rdrand lahf_lm ida arat xsaveopt pln pts dtherm tpr_shadow vnmi flexpriority ept vpid fsgsbase smep erms
bogomips : 6200.20
clflush size : 64
cache_alignment : 64
address sizes : 36 bits physical, 48 bits virtual
power management:
processor : 1
vendor_id : GenuineIntel
cpu family : 6
model : 58
model name : Intel(R) Core(TM) i7-3770S CPU @ 3.10GHz
stepping : 9
microcode : 0x1b
cpu MHz : 3100.102
cache size : 8192 KB
physical id : 0
siblings : 2
core id : 1
cpu cores : 2
apicid : 2
initial apicid : 2
fpu : yes
fpu_exception : yes
cpuid level : 13
wp : yes
flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf pni pclmulqdq dtes64 monitor ds_cpl vmx smx est tm2 ssse3 cx16 xtpr pdcm pcid sse4_1 sse4_2 x2apic popcnt tsc_deadline_timer aes xsave avx f16c rdrand lahf_lm ida arat xsaveopt pln pts dtherm tpr_shadow vnmi flexpriority ept vpid fsgsbase smep erms
bogomips : 6200.20
clflush size : 64
cache_alignment : 64
address sizes : 36 bits physical, 48 bits virtual
power management:
<<<<< cat /proc/sys/kernel/sem >>>>>>>>>>>>>>>>>>>>>>
250 32000 32 128
<<<<< ls /proc/xenomai >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
affinity
apc
clock
debug
faults
heap
irq
latency
registry
sched
timer
version
<<<<< cat /proc/xenomai/affinity >>>>>>>>>>>>>>>>>>>>
00000003
<<<<< cat /proc/xenomai/faults >>>>>>>>>>>>>>>>>>>>>>
TRAP CPU0 CPU1
0: 0 0 (Divide error)
1: 0 0 (Debug)
3: 0 0 (Int3)
4: 0 0 (Overflow)
5: 0 0 (Bounds)
6: 0 0 (Invalid opcode)
7: 0 0 (FPU not available)
8: 0 0 (Double fault)
9: 0 0 (FPU segment overrun)
10: 0 0 (Invalid TSS)
11: 0 0 (Segment not present)
12: 0 0 (Stack segment)
13: 0 0 (General protection)
14: 0 0 (Page fault)
15: 0 0 (Spurious interrupt)
16: 0 0 (FPU error)
17: 0 0 (Alignment check)
18: 0 0 (Machine check)
19: 0 0 (SIMD error)
<<<<< cat /proc/xenomai/irq >>>>>>>>>>>>>>>>>>>>>>>>>
IRQ CPU0 CPU1
4352: 0 1 [reschedule]
4353: 28214 22346 [timer/0]
4354: 1 0 [timer-ipi]
4355: 0 0 [sync]
4419: 3 1 [virtual]
<<<<< cat /proc/xenomai/apc >>>>>>>>>>>>>>>>>>>>>>>>>
APC CPU0 CPU1
0: 0 0 (selector_list_destroy)
1: 0 0 (registry_export)
<<<<< cat /proc/xenomai/heap >>>>>>>>>>>>>>>>>>>>>>>>
TOTAL FREE NAME
262144 262144 system heap
16384 16384 debug log
65536 65408 shared heap
<<<<< cat /proc/xenomai/latency >>>>>>>>>>>>>>>>>>>>>
1162
<<<<< cat /proc/xenomai/version >>>>>>>>>>>>>>>>>>>>>
3.0.1
<<<<< ls /proc/xenomai/clock >>>>>>>>>>>>>>>>>>>>>>>>
coreclk
<<<<< cat /proc/xenomai/clock/coreclk >>>>>>>>>>>>>>>
gravity: irq=81 kernel=1162 user=1162
devices: timer=lapic, clock=tsc
status: on
setup: 81
ticks: 2592839844596
<<<<< ls /proc/xenomai/registry >>>>>>>>>>>>>>>>>>>>>
usage
<<<<< cat /proc/xenomai/registry/usage >>>>>>>>>>>>>>
9/512
<<<<< ls /proc/xenomai/timer >>>>>>>>>>>>>>>>>>>>>>>>
coreclk
<<<<< cat /proc/xenomai/timer/coreclk >>>>>>>>>>>>>>
CPU SCHED/SHOT TIMEOUT INTERVAL NAME
0 151861/26621 170us - [host-timer/0]
1 156824/21589 94us - [host-timer/1]
<<<<< ls /proc/xenomai/debug >>>>>>>>>>>>>>>>>>>>>>>>
relax
<<<<< cat /proc/xenomai/debug/relax >>>>>>>>>>>>>>>>>
<<<<< ls /proc/xenomai/sched >>>>>>>>>>>>>>>>>>>>>>>>
acct
rt
stat
threads
<<<<< cat /proc/xenomai/sched/acct >>>>>>>>>>>>>>>>>>
0 0 0 2 0 0 00018000 706537604072 706535605162 836373274151 ROOT/0 idle -1 0
1 0 0 2 0 0 00018000 706535604243 706534275732 836373921556 ROOT/1 idle -1 0
1 429 0 2 0 0 00000042 706535604243 0 2923 rtnet-stack rt 98 0
0 431 0 2 0 0 00020042 706537604072 0 2299 rtnet-rtpc rt 0 0
1 0 0 22354 0 0 00000000 706535604243 1057211 2559533 IRQ4353: [timer] idle 0 0
<<<<< cat /proc/xenomai/sched/stat >>>>>>>>>>>>>>>>>>
CPU PID MSW CSW XSC PF STAT %CPU NAME
0 0 0 2 0 0 00018000 100.0 [ROOT/0]
1 0 0 2 0 0 00018000 100.0 [ROOT/1]
1 429 0 2 0 0 00000042 0.0 [rtnet-stack]
0 431 0 2 0 0 00020042 0.0 [rtnet-rtpc]
1 0 0 22354 0 0 00000000 0.0 [IRQ4353: [timer]]
<<<<< cat /proc/xenomai/sched/threads >>>>>>>>>>>>>>>
CPU PID CLASS TYPE PRI TIMEOUT STAT NAME
0 0 idle core -1 - R [ROOT/0]
1 0 idle core -1 - R [ROOT/1]
1 429 rt core 98 - W [rtnet-stack]
0 431 rt core 0 - W [rtnet-rtpc]
<<<<< ls /proc/xenomai/sched/rt >>>>>>>>>>>>>>>>>>>>>
threads
<<<<< cat /proc/xenomai/sched/rt/threads >>>>>>>>>>>>
CPU PID PRI PERIOD NAME
1 429 98 - rtnet-stack
<<<<<<<<<<<<<<<<<<<<<<< End of Message >>>>>>>>>>>
^ permalink raw reply [flat|nested] 8+ messages in thread* Re: [Xenomai] Xenomai/Cobalt on i7-3770S CPU 2016-06-08 9:42 [Xenomai] Xenomai/Cobalt on i7-3770S CPU Heinick, J Michael @ 2016-06-08 19:39 ` Gilles Chanteperdrix 2016-06-09 17:32 ` Heinick, J Michael 0 siblings, 1 reply; 8+ messages in thread From: Gilles Chanteperdrix @ 2016-06-08 19:39 UTC (permalink / raw) To: Heinick, J Michael; +Cc: xenomai@xenomai.org On Wed, Jun 08, 2016 at 09:42:26AM +0000, Heinick, J Michael wrote: > > We currently have an RTDM driver that is running well on Xenomai/Cobalt 3.0.1 on 2 Dell computers with Core2 processors, but will hang (unresponsive mouse and keyboard, no discernable activity) an entire SuperLogics i7 computer with a Core i7-3770S processor (4 physical cores, 4 logical cores). The hang occurs in the ioctl function at an rtdm_sem_down call that waits for the interrupt handler to signal the handling of an interrupt. We suspect that we have a problem with our kernel build/installation configuration options. We have attempted to configure the Core i7-3770S system so that Xenomai/Cobalt only uses 2 cores like the other 2 working Core2 computers, but the system still hangs (more detail on the results of our attempt is included beow). Eventually, we would like to configure Xenomai/Cobalt to run on 4 cores of the i7 computer if possible. > > Any suggestions to help us make/install/configure Xenomai/Cobalt to run on the SuperLogics computer with the i7-3770S processor so that the rtdm_sem_down call in the RTDM driver does not hang the entire system would be appreciated. This sounds like an irq conflict: a device handled by an RTDM driver can not use the same irq line as a device handled by a plain Linux driver without modifying the plain Linux driver. See FAQ for solutions to that problem. -- Gilles. https://click-hack.org ^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: [Xenomai] Xenomai/Cobalt on i7-3770S CPU 2016-06-08 19:39 ` Gilles Chanteperdrix @ 2016-06-09 17:32 ` Heinick, J Michael 2016-06-09 17:41 ` Gilles Chanteperdrix 0 siblings, 1 reply; 8+ messages in thread From: Heinick, J Michael @ 2016-06-09 17:32 UTC (permalink / raw) To: xenomai@xenomai.org -----Original Message----- From: Gilles Chanteperdrix [mailto:gilles.chanteperdrix@xenomai.org] Sent: Wednesday, June 08, 2016 3:39 PM To: Heinick, J Michael Cc: xenomai@xenomai.org Subject: Re: [Xenomai] Xenomai/Cobalt on i7-3770S CPU On Wed, Jun 08, 2016 at 09:42:26AM +0000, Heinick, J Michael wrote: > > We currently have an RTDM driver that is running well on Xenomai/Cobalt 3.0.1 on 2 Dell computers with Core2 processors, but will hang (unresponsive mouse and keyboard, no discernable activity) an entire SuperLogics i7 computer with a Core i7-3770S processor (4 physical cores, 4 logical cores). The hang occurs in the ioctl function at an rtdm_sem_down call that waits for the interrupt handler to signal the handling of an interrupt. We suspect that we have a problem with our kernel build/installation configuration options. We have attempted to configure the Core i7-3770S system so that Xenomai/Cobalt only uses 2 cores like the other 2 working Core2 computers, but the system still hangs (more detail on the results of our attempt is included beow). Eventually, we would like to configure Xenomai/Cobalt to run on 4 cores of the i7 computer if possible. > > Any suggestions to help us make/install/configure Xenomai/Cobalt to run on the SuperLogics computer with the i7-3770S processor so that the rtdm_sem_down call in the RTDM driver does not hang the entire system would be appreciated. This sounds like an irq conflict: a device handled by an RTDM driver can not use the same irq line as a device handled by a plain Linux driver without modifying the plain Linux driver. See FAQ for solutions to that problem. -- Gilles. https://click-hack.org Thanks for the reply, Gilles. Yes, there was a conflict on irq 16. We disabled the conflicting component so that now our driver is the only one on irq 16, but the i7-3770S system is still hanging. We know the interrupt handler is receiving the interrupts and handling them without hanging the i7 system. With the interrupts running at 20Hz, the handler stores counts of the second and sub-second interrupts that we can retrieve with a different ioctl call that does not wait for an interrupt. The hang only occurs in the ioctl call at the wait with the rtdm_sem_down call. The contents of selected files from the /proc directory are included below. Thanks for the help so far and any additional help to come, Mike <<<<< cat /proc/interrupts >>>>>>>>>>>>>>>>>>>>>>>>>> CPU0 CPU1 0: 23 0 IO-APIC-edge timer 1: 1403 0 IO-APIC-edge i8042 8: 1 0 IO-APIC-edge rtc0 9: 3 0 IO-APIC-fasteoi acpi 12: 5472 0 IO-APIC-edge i8042 40: 0 0 PCI-MSI-edge xhci_hcd 41: 0 0 PCI-MSI-edge xhci_hcd 42: 0 0 PCI-MSI-edge xhci_hcd 43: 180 15 PCI-MSI-edge eth2 44: 14186 148 PCI-MSI-edge ahci 45: 179 15 PCI-MSI-edge eth3-rx-0 46: 0 0 PCI-MSI-edge eth3-tx-0 47: 1 0 PCI-MSI-edge eth3 48: 0 0 PCI-MSI-edge ahci 49: 13 0 PCI-MSI-edge mei_me 50: 21185 0 PCI-MSI-edge i915 NMI: 1 0 Non-maskable interrupts LOC: 18135 15491 Local timer interrupts SPU: 0 0 Spurious interrupts PMI: 1 0 Performance monitoring interrupts IWI: 800 447 IRQ work interrupts RTR: 0 0 APIC ICR read retries RES: 11895 12632 Rescheduling interrupts CAL: 456 7273 Function call interrupts TLB: 389 447 TLB shootdowns TRM: 0 0 Thermal event interrupts THR: 0 0 Threshold APIC interrupts MCE: 0 0 Machine check exceptions MCP: 3 3 Machine check polls ERR: 0 MIS: 0 <<<<< cat /proc/xenomai/irq >>>>>>>>>>>>>>>>>>>>>>>>> IRQ CPU0 CPU1 16: 0 0 tcgrtdm 4352: 0 0 [reschedule] 4353: 18866 18260 [timer/0] 4354: 1 0 [timer-ipi] 4355: 0 0 [sync] 4419: 0 368 [virtual] <<<<< End of Message >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: [Xenomai] Xenomai/Cobalt on i7-3770S CPU 2016-06-09 17:32 ` Heinick, J Michael @ 2016-06-09 17:41 ` Gilles Chanteperdrix 2016-06-10 17:03 ` Jan Kiszka 0 siblings, 1 reply; 8+ messages in thread From: Gilles Chanteperdrix @ 2016-06-09 17:41 UTC (permalink / raw) To: Heinick, J Michael; +Cc: xenomai@xenomai.org On Thu, Jun 09, 2016 at 05:32:51PM +0000, Heinick, J Michael wrote: > > -----Original Message----- > From: Gilles Chanteperdrix [mailto:gilles.chanteperdrix@xenomai.org] > Sent: Wednesday, June 08, 2016 3:39 PM > To: Heinick, J Michael > Cc: xenomai@xenomai.org > Subject: Re: [Xenomai] Xenomai/Cobalt on i7-3770S CPU > > On Wed, Jun 08, 2016 at 09:42:26AM +0000, Heinick, J Michael wrote: > > > > We currently have an RTDM driver that is running well on Xenomai/Cobalt 3.0.1 on 2 Dell computers with Core2 processors, but will hang (unresponsive mouse and keyboard, no discernable activity) an entire SuperLogics i7 computer with a Core i7-3770S processor (4 physical cores, 4 logical cores). The hang occurs in the ioctl function at an rtdm_sem_down call that waits for the interrupt handler to signal the handling of an interrupt. We suspect that we have a problem with our kernel build/installation configuration options. We have attempted to configure the Core i7-3770S system so that Xenomai/Cobalt only uses 2 cores like the other 2 working Core2 computers, but the system still hangs (more detail on the results of our attempt is included beow). Eventually, we would like to configure Xenomai/Cobalt to run on 4 cores of the i7 computer if possible. > > > > Any suggestions to help us make/install/configure Xenomai/Cobalt to run on the SuperLogics computer with the i7-3770S processor so that the rtdm_sem_down call in the RTDM driver does not hang the entire system would be appreciated. > > This sounds like an irq conflict: a device handled by an RTDM driver can not use the same irq line as a device handled by a plain Linux driver without modifying the plain Linux driver. See FAQ for solutions to that problem. > > -- > Gilles. > https://click-hack.org > > > Thanks for the reply, Gilles. > > Yes, there was a conflict on irq 16. We disabled the conflicting > component so that now our driver is the only one on irq 16, but > the i7-3770S system is still hanging. We know the interrupt > handler is receiving the interrupts and handling them without > hanging the i7 system. With the interrupts running at 20Hz, the > handler stores counts of the second and sub-second interrupts that > we can retrieve with a different ioctl call that does not wait for > an interrupt. The hang only occurs in the ioctl call at the wait > with the rtdm_sem_down call. The contents of selected files from > the /proc directory are included below. Could you post the simplest driver which generates this issue? -- Gilles. https://click-hack.org ^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: [Xenomai] Xenomai/Cobalt on i7-3770S CPU 2016-06-09 17:41 ` Gilles Chanteperdrix @ 2016-06-10 17:03 ` Jan Kiszka 2016-06-16 15:51 ` Heinick, J Michael 0 siblings, 1 reply; 8+ messages in thread From: Jan Kiszka @ 2016-06-10 17:03 UTC (permalink / raw) To: Gilles Chanteperdrix, Heinick, J Michael; +Cc: xenomai@xenomai.org On 2016-06-09 19:41, Gilles Chanteperdrix wrote: > On Thu, Jun 09, 2016 at 05:32:51PM +0000, Heinick, J Michael wrote: >> >> -----Original Message----- >> From: Gilles Chanteperdrix [mailto:gilles.chanteperdrix@xenomai.org] >> Sent: Wednesday, June 08, 2016 3:39 PM >> To: Heinick, J Michael >> Cc: xenomai@xenomai.org >> Subject: Re: [Xenomai] Xenomai/Cobalt on i7-3770S CPU >> >> On Wed, Jun 08, 2016 at 09:42:26AM +0000, Heinick, J Michael wrote: >>> >>> We currently have an RTDM driver that is running well on Xenomai/Cobalt 3.0.1 on 2 Dell computers with Core2 processors, but will hang (unresponsive mouse and keyboard, no discernable activity) an entire SuperLogics i7 computer with a Core i7-3770S processor (4 physical cores, 4 logical cores). The hang occurs in the ioctl function at an rtdm_sem_down call that waits for the interrupt handler to signal the handling of an interrupt. We suspect that we have a problem with our kernel build/installation configuration options. We have attempted to configure the Core i7-3770S system so that Xenomai/Cobalt only uses 2 cores like the other 2 working Core2 computers, but the system still hangs (more detail on the results of our attempt is included beow). Eventually, we would like to configu > re Xenomai/Cobalt to run on 4 cores of the i7 computer if possible. >>> >>> Any suggestions to help us make/install/configure Xenomai/Cobalt to run on the SuperLogics computer with the i7-3770S processor so that the rtdm_sem_down call in the RTDM driver does not hang the entire system would be appreciated. >> >> This sounds like an irq conflict: a device handled by an RTDM driver can not use the same irq line as a device handled by a plain Linux driver without modifying the plain Linux driver. See FAQ for solutions to that problem. >> >> -- >> Gilles. >> https://click-hack.org >> >> >> Thanks for the reply, Gilles. >> >> Yes, there was a conflict on irq 16. We disabled the conflicting >> component so that now our driver is the only one on irq 16, but >> the i7-3770S system is still hanging. We know the interrupt >> handler is receiving the interrupts and handling them without >> hanging the i7 system. With the interrupts running at 20Hz, the >> handler stores counts of the second and sub-second interrupts that >> we can retrieve with a different ioctl call that does not wait for >> an interrupt. The hang only occurs in the ioctl call at the wait >> with the rtdm_sem_down call. The contents of selected files from >> the /proc directory are included below. > > > Could you post the simplest driver which generates this issue? > And do you have CONFIG_XENO_OPT_DEBUG_COBALT enabled? That may reveal sleeping issues in the driver design (common source of troubles). Jan -- Siemens AG, Corporate Technology, CT RDA ITP SES-DE Corporate Competence Center Embedded Linux ^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: [Xenomai] Xenomai/Cobalt on i7-3770S CPU 2016-06-10 17:03 ` Jan Kiszka @ 2016-06-16 15:51 ` Heinick, J Michael 2016-06-16 17:16 ` Gilles Chanteperdrix 0 siblings, 1 reply; 8+ messages in thread From: Heinick, J Michael @ 2016-06-16 15:51 UTC (permalink / raw) To: xenomai@xenomai.org -----Original Message----- From: Jan Kiszka [mailto:jan.kiszka@siemens.com] Sent: Friday, June 10, 2016 1:03 PM To: Gilles Chanteperdrix; Heinick, J Michael Cc: xenomai@xenomai.org Subject: Re: Xenomai/Cobalt on i7-3770S CPU On 2016-06-09 19:41, Gilles Chanteperdrix wrote: > On Thu, Jun 09, 2016 at 05:32:51PM +0000, Heinick, J Michael wrote: >> >> -----Original Message----- >> From: Gilles Chanteperdrix [mailto:gilles.chanteperdrix@xenomai.org] >> Sent: Wednesday, June 08, 2016 3:39 PM >> To: Heinick, J Michael >> Cc: xenomai@xenomai.org >> Subject: Re: [Xenomai] Xenomai/Cobalt on i7-3770S CPU >> >> On Wed, Jun 08, 2016 at 09:42:26AM +0000, Heinick, J Michael wrote: >>> >>> We currently have an RTDM driver that is running well on >>> Xenomai/Cobalt 3.0.1 on 2 Dell computers with Core2 processors, but >>> will hang (unresponsive mouse and keyboard, no discernable activity) >>> an entire SuperLogics i7 computer with a Core i7-3770S processor (4 >>> physical cores, 4 logical cores). The hang occurs in the ioctl >>> function at an rtdm_sem_down call that waits for the interrupt >>> handler to signal the handling of an interrupt. We suspect that we >>> have a problem with our kernel build/installation configuration >>> options. We have attempted to configure the Core i7-3770S system so >>> that Xenomai/Cobalt only uses 2 cores like the other 2 working Core2 >>> computers, but the system still hangs (more detail on the results of >>> our attempt is included beow). Eventually, we would like to configu > re Xenomai/Cobalt to run on 4 cores of the i7 computer if possible. >>> >>> Any suggestions to help us make/install/configure Xenomai/Cobalt to run on the SuperLogics computer with the i7-3770S processor so that the rtdm_sem_down call in the RTDM driver does not hang the entire system would be appreciated. >> >> This sounds like an irq conflict: a device handled by an RTDM driver can not use the same irq line as a device handled by a plain Linux driver without modifying the plain Linux driver. See FAQ for solutions to that problem. >> >> -- >> Gilles. >> https://click-hack.org >> >> >> Thanks for the reply, Gilles. >> >> Yes, there was a conflict on irq 16. We disabled the conflicting >> component so that now our driver is the only one on irq 16, but the >> i7-3770S system is still hanging. We know the interrupt handler is >> receiving the interrupts and handling them without hanging the i7 >> system. With the interrupts running at 20Hz, the handler stores >> counts of the second and sub-second interrupts that we can retrieve >> with a different ioctl call that does not wait for an interrupt. The >> hang only occurs in the ioctl call at the wait with the rtdm_sem_down >> call. The contents of selected files from the /proc directory are >> included below. > > > Could you post the simplest driver which generates this issue? > And do you have CONFIG_XENO_OPT_DEBUG_COBALT enabled? That may reveal sleeping issues in the driver design (common source of troubles). Jan -- Siemens AG, Corporate Technology, CT RDA ITP SES-DE Corporate Competence Center Embedded Linux ------------------ Thanks Gilles and Jan for your help last week. We have apparently found the bug in my code that caused the problem. This is our first RTDM driver, and early in its development only the ioctl_nrt handler was specified in the ops structure of the rtdm_driver structure, and not the ioctl_rt handler. During the process this past week of stripping down the driver to generate some code to post that would demonstrate the problem, the stripped down driver would hang the core2 computer just like the i7 computer. Eventually, I noticed that only nrt handlers were specified in the ops structure. After moving our ioctl functions to .ioctl_rt, both the stripped down driver and the full driver run on both the core2 and i7 computers. Why the core2 computers appeared to work with only the ioctl_nrt handler specified in the full driver and hang the i7 computer is still a mystery to us. Thanks again for your help, Mike ^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: [Xenomai] Xenomai/Cobalt on i7-3770S CPU 2016-06-16 15:51 ` Heinick, J Michael @ 2016-06-16 17:16 ` Gilles Chanteperdrix 2016-06-17 13:25 ` Heinick, J Michael 0 siblings, 1 reply; 8+ messages in thread From: Gilles Chanteperdrix @ 2016-06-16 17:16 UTC (permalink / raw) To: Heinick, J Michael; +Cc: xenomai@xenomai.org On Thu, Jun 16, 2016 at 03:51:17PM +0000, Heinick, J Michael wrote: > problem, the stripped down driver would hang the core2 computer > just like the i7 computer. Are you running a graphic system? If yes, have you tried to plug a serial console or netconsole to try and retrieve the kernel console when the hang happens? Because if the hang is in fact a kernel oops, the oops message may indicate what the problem is. > Eventually, I noticed that only nrt > handlers were specified in the ops structure. After moving our > ioctl functions to .ioctl_rt, both the stripped down driver and > the full driver run on both the core2 and i7 computers. You can not call "sleeping" services from an _nrt handler. If that is what you were doing. > Why the core2 computers appeared to work with only the ioctl_nrt > handler specified in the full driver and hang the i7 computer is > still a mystery to us. Did the two machines run with the same kernel? Or were there differences in the kernel configuration? Like I-pipe checks disabled/enabled? -- Gilles. https://click-hack.org ^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: [Xenomai] Xenomai/Cobalt on i7-3770S CPU 2016-06-16 17:16 ` Gilles Chanteperdrix @ 2016-06-17 13:25 ` Heinick, J Michael 0 siblings, 0 replies; 8+ messages in thread From: Heinick, J Michael @ 2016-06-17 13:25 UTC (permalink / raw) To: xenomai@xenomai.org -----Original Message----- From: Gilles Chanteperdrix [mailto:gilles.chanteperdrix@xenomai.org] Sent: Thursday, June 16, 2016 1:16 PM To: Heinick, J Michael Cc: xenomai@xenomai.org Subject: Re: [Xenomai] Xenomai/Cobalt on i7-3770S CPU On Thu, Jun 16, 2016 at 03:51:17PM +0000, Heinick, J Michael wrote: > problem, the stripped down driver would hang the core2 computer just > like the i7 computer. Are you running a graphic system? If yes, have you tried to plug a serial console or netconsole to try and retrieve the kernel console when the hang happens? Because if the hang is in fact a kernel oops, the oops message may indicate what the problem is. > Eventually, I noticed that only nrt > handlers were specified in the ops structure. After moving our ioctl > functions to .ioctl_rt, both the stripped down driver and the full > driver run on both the core2 and i7 computers. You can not call "sleeping" services from an _nrt handler. If that is what you were doing. > Why the core2 computers appeared to work with only the ioctl_nrt > handler specified in the full driver and hang the i7 computer is still > a mystery to us. Did the two machines run with the same kernel? Or were there differences in the kernel configuration? Like I-pipe checks disabled/enabled? -- Gilles. https://click-hack.org ---------------------------------------------------------- Thanks Gilles, Yes, I was making an rtdm_sem_down call in the ioctl_nrt to wait for the rtdm_sem_up in the interrupt handler. I probably set it that way months ago when I was developing calls in the full driver to configure the device, and did not remember it when I implemented the interrupts and things still appeared to work on the core2 machine where development was taking place. The stripped down driver eliminated the interrupt handler and put the rtdm_sem_up call in the write handler that was called by the user program to fake an interrupt. The stripped down driver with the ioctl_nrt would hang the computer on both machines. Changing ioctl_nrt to ioctl_rt enabled both the stripped down driver and the full driver to work on all machines. All machines were running xenomai 3.0.1 with ipipe release 10 on linux 3.14.44. There were probably some differences in kernel configuration, but anything involving things like i-pipe just used defaults. Since the fix the Super Logics i7 computer has been updated to Xenomai 3.0.2. I suppose now I should separate my ioctl_nrt stuff to a separate function from the ioctl_rt stuff, and then I will I have both functions specified in the ops structure of the rtdm_driver structure. I will have to look up how xenomai is going to determine which one to use. Thanks for the help, Mike ^ permalink raw reply [flat|nested] 8+ messages in thread
end of thread, other threads:[~2016-06-17 13:25 UTC | newest] Thread overview: 8+ messages (download: mbox.gz follow: Atom feed -- links below jump to the message on this page -- 2016-06-08 9:42 [Xenomai] Xenomai/Cobalt on i7-3770S CPU Heinick, J Michael 2016-06-08 19:39 ` Gilles Chanteperdrix 2016-06-09 17:32 ` Heinick, J Michael 2016-06-09 17:41 ` Gilles Chanteperdrix 2016-06-10 17:03 ` Jan Kiszka 2016-06-16 15:51 ` Heinick, J Michael 2016-06-16 17:16 ` Gilles Chanteperdrix 2016-06-17 13:25 ` Heinick, J Michael
This is an external index of several public inboxes, see mirroring instructions on how to clone and mirror all data and code used by this external index.