AW: AW: [Xenomai-help] resubmission : memset of heap crashes Xenomai-Task

AW: AW: [Xenomai-help] resubmission : memset of heap crashes Xenomai-Task

[Roderik_Wildenburg]

jan.kiszka@domain.hid [mailto:jan.kiszka@domain.hid] :
 
> First of all, do we know where the demo precisely faults? In 
> which heap's memset? We could then analyse how this heap is 
> being set up in the kernel, if all mappings are done as 
> expected, if reasonable address are passed to the user, etc.
> 

It crashs when the last heap is touched (please see programs output below).
Astonishingly it does not crash every time !? In fact I had to reboot to produce this output (after running the program several times without a crash; don´t ask me why) :
 
Start Heap
root_thread_init :
timer startedt
display task created
root_thread_init beendet
Available Memory 113377280 (Pagesize 4096)
heap 0 of size 16000000 created
Heap 0 allocated size 16000000
heap 1 of size 16000000 created
Heap 1 allocated size 16000000
heap 2 of size 16000000 created
Heap 2 allocated size 16000000
heap 3 of size 16000000 created
Heap 3 allocated size 16000000
heap 4 of size 16000000 created
Heap 4 allocated size 16000000
heap 5 of size 16000000 created
Heap 5 allocated size 16000000
heap 6 of size 16000000 created
Heap 6 allocated size 16000000
Available Memory after allocation 4317184
Heap 7 created with size 1505280
Heap 7 allocated size 1505280
Memory allocated in total : 113505280
Setting heap 0. Size 16000000
Setting heap 1. Size 16000000
Setting heap 2. Size 16000000
Setting heap 3. Size 16000000
Setting heap 4. Size 16000000
Setting heap 5. Size 16000000
Setting heap 6. Size 16000000
Setting heap 7. Size 1505280
Segmentation fault 

> > I may have my hands on a 405 board on Friday, but I'm not 
> sure if I can do much more testing than a simple run of the 
> program. Of course, testing will primarily take place over 
> latest Xenomai...
> 

This certainly would be helpful, especially if you could reproduce the problem (so I know I don´t hunt a phantasm).

> Roderik, you said you tested under Qemu as well? For PPC or x86 arch?

I thought, you tested with Qemu without success (crash). But perhaps my rememberence mixes something up. When I said, that the crash could not be reproduced with Qemu I thought to reference your test. Isn´t this correct ?  

Roderik

Re: AW: AW: [Xenomai-help] resubmission : memset of heap crashes Xenomai-Task

[Jan Kiszka]

[-- Attachment #1: Type: text/plain, Size: 2706 bytes --]

Roderik_Wildenburg@domain.hid wrote:
> jan.kiszka@domain.hid [mailto:jan.kiszka@domain.hid] :
>  
>> First of all, do we know where the demo precisely faults? In 
>> which heap's memset? We could then analyse how this heap is 
>> being set up in the kernel, if all mappings are done as 
>> expected, if reasonable address are passed to the user, etc.
>>
> 
> It crashs when the last heap is touched (please see programs output below).

Where precisely? What address is touched? Already tried to catch the
SEGV with a debugger?

> Astonishingly it does not crash every time !? In fact I had to reboot to produce this output (after running the program several times without a crash; don´t ask me why) :
>  
> Start Heap
> root_thread_init :
> timer startedt
> display task created
> root_thread_init beendet
> Available Memory 113377280 (Pagesize 4096)
> heap 0 of size 16000000 created
> Heap 0 allocated size 16000000
> heap 1 of size 16000000 created
> Heap 1 allocated size 16000000
> heap 2 of size 16000000 created
> Heap 2 allocated size 16000000
> heap 3 of size 16000000 created
> Heap 3 allocated size 16000000
> heap 4 of size 16000000 created
> Heap 4 allocated size 16000000
> heap 5 of size 16000000 created
> Heap 5 allocated size 16000000
> heap 6 of size 16000000 created
> Heap 6 allocated size 16000000
> Available Memory after allocation 4317184
> Heap 7 created with size 1505280
> Heap 7 allocated size 1505280
> Memory allocated in total : 113505280
> Setting heap 0. Size 16000000
> Setting heap 1. Size 16000000
> Setting heap 2. Size 16000000
> Setting heap 3. Size 16000000
> Setting heap 4. Size 16000000
> Setting heap 5. Size 16000000
> Setting heap 6. Size 16000000
> Setting heap 7. Size 1505280
> Segmentation fault 
> 
>>> I may have my hands on a 405 board on Friday, but I'm not 
>> sure if I can do much more testing than a simple run of the 
>> program. Of course, testing will primarily take place over 
>> latest Xenomai...
>>
> 
> This certainly would be helpful, especially if you could reproduce the problem (so I know I don´t hunt a phantasm).
> 
>> Roderik, you said you tested under Qemu as well? For PPC or x86 arch?
> 
> I thought, you tested with Qemu without success (crash). But perhaps my rememberence mixes something up. When I said, that the crash could not be reproduced with Qemu I thought to reference your test. Isn´t this correct ?  

Ok, misunderstanding. I tried with Qemu and real x86 hardware, but only
over 2.6 so far. Will have to check with 2.4 as well because of the
over-commitment 2.6 does when requesting memory + the OOM killer
shooting the test immediately.

Jan


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 250 bytes --]

AW: AW: [Xenomai-help] resubmission : memset of heap crashes Xenomai-Task

[Roderik_Wildenburg]

Gilles Chanteperdrix [mailto:gilles.chanteperdrix@xenomai.org] :
> 
> Ok. Now the question remains: does this problem still exist 
> with version
> 2.2.2 ? The heaps overhead computations have changed recently.
> 

Good news. I can´t reproduce the "seg. fault" with 2.2.2 although I tried hard (restarted at least 20 times). Obviously the new "heap overhead computations" eliminated the problem. 
Wolgang could you be so nice to try to reproduce the problem with 2.2.2, just to be sure ?

Many thanks to everybody who spend his precious time 
Roderik

P.s.: with 2.2.2 I noticed, that in /proc/xenomai/interfaces vxWorks is not listed although I compiled the kernel with support for vxworks. Is this o.K. ?

Re: AW: AW: [Xenomai-help] resubmission : memset of heap crashes Xenomai-Task

[Wolfgang Grandegger]

Roderik_Wildenburg@domain.hid wrote:
> Gilles Chanteperdrix [mailto:gilles.chanteperdrix@xenomai.org] :
>> Ok. Now the question remains: does this problem still exist 
>> with version
>> 2.2.2 ? The heaps overhead computations have changed recently.
>>
> 
> Good news. I can´t reproduce the "seg. fault" with 2.2.2 although I tried hard (restarted at least 20 times). Obviously the new "heap overhead computations" eliminated the problem. 
> Wolgang could you be so nice to try to reproduce the problem with 2.2.2, just to be sure ?

Good news, indeed. I will do so a.s.a.p. (= not before next week).

Wolfgang.

> Many thanks to everybody who spend his precious time 
> Roderik
> 
> P.s.: with 2.2.2 I noticed, that in /proc/xenomai/interfaces vxWorks is not listed although I compiled the kernel with support for vxworks. Is this o.K. ?
> 
> 

Re: AW: AW: [Xenomai-help] resubmission : memset of heap crashes Xenomai-Task

[Gilles Chanteperdrix]

Roderik_Wildenburg@domain.hid wrote:
 > P.s.: with 2.2.2 I noticed, that in /proc/xenomai/interfaces vxWorks is not listed although I compiled the kernel with support for vxworks. Is this o.K. ?

The vxworks interface probably failed to start, likely because it needs
periodic timer and you load the native interface that set the hardware
timer to aperiodic mode.

-- 


					    Gilles Chanteperdrix.

Re: AW: AW: [Xenomai-help] resubmission : memset of heap crashes Xenomai-Task

[Wolfgang Grandegger]

[-- Attachment #1: Type: text/plain, Size: 1186 bytes --]

Roderik_Wildenburg@domain.hid wrote:
> Gilles Chanteperdrix [mailto:gilles.chanteperdrix@xenomai.org] :
>> Ok. Now the question remains: does this problem still exist 
>> with version
>> 2.2.2 ? The heaps overhead computations have changed recently.
>>
> 
> Good news. I can´t reproduce the "seg. fault" with 2.2.2 although I tried hard (restarted at least 20 times). Obviously the new "heap overhead computations" eliminated the problem. 
> Wolgang could you be so nice to try to reproduce the problem with 2.2.2, just to be sure ?

I ran heap.c plenty of times on my MPC5200 setup under Linux 2.4.25 and 
Xenomai (head of SVN) and was unable to reproduce the segfault error 
:-). But I got an Oops when I aborted the program via ^C _once_ after 
reboot. All subsequent executions worked fine. I have attached the trace 
and a manual backtrace. Seems like a crash in strlen() called from 
remove_proc_entry().

Wolfgang.

> 
> Many thanks to everybody who spend his precious time 
> Roderik
> 
> P.s.: with 2.2.2 I noticed, that in /proc/xenomai/interfaces vxWorks is not listed although I compiled the kernel with support for vxworks. Is this o.K. ?
> 
> 


[-- Attachment #2: heap-exit.log --]
[-- Type: text/x-log, Size: 3962 bytes --]

bash-2.05b# ./heap
Start Heap
root_thread_init :
timer startedt
display task created
root_thread_init beendet
Available Memory 45674496 (Pagesize 4096)
heap 0 of size 16000000 created
Heap 0 allocated size 16000000
heap 1 of size 16000000 created
Heap 1 allocated size 16000000
heap 2 of size 16000000 created
Heap 2 allocated size 16000000
__alloc_pages: 0-order allocation failed (gfp=0x1f2/0)
Available Memory after allocation 8888320
__alloc_pages: 0-order allocation failed (gfp=0x1f2/0)
Heap 3 created with size 4172800
Heap 3 allocated size 4172800
Memory allocated in total : 52172800
1 loops done
11 loops done
UDP->root_thread_exit
*** long delay ***
Machine check in kernel mode.
Caused by (from SRR1=41000): Transfer error ack signal
Oops: machine check, sig: 7
NIP: C00117A0 XER: 20000000 LR: C0060018 SP: C03D7F00 REGS: c03d7e50 TRAP: 0200    Not tainted
MSR: 00041000 EE: 0 PR: 0 FP: 0 ME: 1 IR/DR: 00
TASK = c03d6000[2] 'keventd' Last syscall: -1
last math 00000000 last altivec 00000000
GPR00: 00000000 C03D7F00 C03D6000 F0927EB7 F0927EB7 00000008 00000000 C024540C
GPR08: 00000000 C02042B4 C01F0000 C01DBC80 84044442 1001AA80 03FBA000 00000000
GPR16: 00000001 00000001 FFFFFFFF 007FFF00 003FF000 C0240000 C0240000 C0210000
GPR24: C02453CC C31919C0 C01C5DB8 C01C6C30 C3E1A420 C3E1A620 C01E731C 22004042
Call backtrace:
C00232F4 C01115E4 C0019A34 C0022E44 C0008598
UDP->root_thread_exit
UDP->root_thread_exit

(gdb) l *0xC00117A0
No source file for address 0xc00117a0. *** in strlen ***
(gdb) l *0xC00232F4
0xc00232f4 is in ipipe_unstall_pipeline_from (core.c:279).
274             else
275                     pos = __ipipe_pipeline.next;
276
277             __ipipe_walk_pipeline(pos, cpuid);
278
279             if (__ipipe_pipeline_head_p(ipd))
280                     local_irq_enable_hw();
281             else
282                     ipipe_unlock_cpu(flags);
283     }
(gdb) l *0xC01115E4
0xc01115e4 is in registry_proc_callback (registry.c:369).
364
365                     xnlock_put_irqrestore(&nklock, s);
366
367                     remove_proc_entry(entry->name, dir);
368
369                     if (entries <= 0) {
370                             remove_proc_entry(type, rdir);
371
372                             if (pnode->root->entries <= 0)
373                                     remove_proc_entry(root, registry_proc_root);
(gdb) l *0xC0019A34
0xc0019a34 is in __run_task_queue (softirq.c:357).
352                     f = p->routine;
353                     data = p->data;
354                     wmb();
355                     p->sync = 0;
356                     if (f)
357                             f(data);
358             }
359     }
360
361     static int ksoftirqd(void * __bind_cpu)
(gdb) l *0xC0022E44
0xc0022e44 is in context_thread (context.c:103).
98                      if (TQ_ACTIVE(tq_context))
99                              set_task_state(curtask, TASK_RUNNING);
100                     schedule();
101                     remove_wait_queue(&context_task_wq, &wait);
102                     run_task_queue(&tq_context);
103                     wake_up(&context_task_done);
104                     if (signal_pending(curtask)) {
105                             while (waitpid(-1, (unsigned int *)0, __WALL|WNOHANG) > 0)
106                                     ;
107                             spin_lock_irq(&curtask->sigmask_lock);
(gdb) l *0xC0008598
No source file for address 0xc0008598. *** in arch_kernel_thread ***
(gdb) l *0xC0060018
0xc0060018 is in remove_proc_entry (generic.c:565).
560             int len;
561
562             if (!parent && xlate_proc_name(name, &parent, &fn) != 0)
563                     goto out;
564             len = strlen(fn);
565             for (p = &parent->subdir; *p; p=&(*p)->next ) {
566                     if (!proc_match(len, fn, *p))
567                             continue;
568                     de = *p;
569                     *p = de->next;
(gdb)

AW: AW: [Xenomai-help] resubmission : memset of heap crashes Xenomai-Task

[Roderik_Wildenburg]

> 
> Ok. Now the question remains: does this problem still exist 
> with version
> 2.2.2 ? The heaps overhead computations have changed recently.
> 

I will try this tomorrow.