Re: [Question] Is it safe to call "xmalloc()" with irq disabled?

[Keir Fraser <keir.xen@gmail.com>]

We need to move dynamic allocation into CPU_UP_PREPARE context. Sadly that
will need surgery on the machine-check cruft. I'll take a look later today
and see if I can do a suitable hatchet job for 4.1.

 -- Keir

On 01/03/2011 08:22, "Haitao Shan" <maillists.shan@gmail.com> wrote:

> Hi, Keir,
> 
> Below is the log when I met the issue.
>>> ================
>>> (XEN)    ffff83023e257e40 ffff82c48019cc80 0000000100000000
>>> 0000000000000039 (XEN)    ffff82c4802c6a00 0000000000000039
>>> ffff82c4802c6a00 0000000000000039 (XEN)    0000000000000000
>>> 0000000000000000 ffff83023e257e70
>>> ffff82c48019f49a
>>> (XEN)    0000000000000039 ffff82c4802c6a00 ffff83023e257e90
>>> ffff82c48019cb8d (XEN)    0000000000000039 ffff82c4802c6a00
>>> ffff83023e257eb0 ffff82c48017815b (XEN) Xen call trace: (XEN)  
>>> [<ffff82c480177b56>] flush_area_mask+0x1b/0x127 (XEN)  
>>> [<ffff82c480115d69>] alloc_heap_pages+0x5d6/0x61b (XEN)  
>>> [<ffff82c480115e75>] alloc_domheap_pages+0xc7/0x13d (XEN)  
>>> [<ffff82c480115f3b>] alloc_xenheap_pages+0x50/0xd8 (XEN)  
>>> [<ffff82c480129e50>] xmalloc_pool_get+0x2b/0x2d (XEN)  
>>> [<ffff82c48012a674>] xmem_pool_alloc+0x26c/0x4c2 (XEN)  
>>> [<ffff82c48012a9d0>] _xmalloc+0x106/0x1b6 (XEN)  
>>> [<ffff82c48019ec25>] mcabanks_alloc+0x18/0xa4 (XEN)  
>>> [<ffff82c4801a27b6>] intel_mcheck_init+0x21/0x64e (XEN)  
>>> [<ffff82c48019f49a>] mcheck_init+0xdd/0x1b2 (XEN)  
>>> [<ffff82c48019cb8d>] identify_cpu+0x27d/0x282 (XEN)  
>>> [<ffff82c48017815b>] smp_store_cpu_info+0x3b/0xca (XEN)  
>>> [<ffff82c4801782e5>] smp_callin+0x8e/0x157 (XEN)  
>>> [<ffff82c4801799b5>] start_secondary+0xab/0x126 (XEN) (XEN)
>>> (XEN) ****************************************
>>> (XEN) Panic on CPU 57:
>>> (XEN) Assertion 'local_irq_is_enabled()' failed at smp.c:234
>>> (XEN) ****************************************
>>> (XEN)
>>> (XEN) Reboot in five seconds...^M
>>> (XEN) Resetting with ACPI MEMORY or I/O RESET_REG.
>>> ===============
> 
> 2011/3/1 Keir Fraser <keir.xen@gmail.com>
>> Haitao,
>> 
>> Both _xmalloc and xfree can only safely be called with irqs enabled. I know
>> there is a somewhat suspicious area during CPU bringup where we temporarily
>> disable spinlock debugging. It would be nice to not need this. And for this
>> particular bug you are dealing with, perhaps we can fix it now -- what is
>> the backtrace for the failing allocation?
>> 
>>  -- Keir
>> 
>> On 01/03/2011 07:42, "Haitao Shan" <maillists.shan@gmail.com> wrote:
>> 
>>> Hi, Keir,
>>> 
>>> In recent effort on debugging cpu offline/online, I met Xen panic some
>>> times.
>>> 
>>> The reason of the panic is caused by following code path:
>>> 
>>> xmalloc ---> alloc_heap_pages ---> flush_area_mask {
>>> ASSERT(local_irq_enabled)........}
>>> 
>>> This bring me the question: is it safe to call xmalloc with local irq
>>> disabled? As you can see, not all alloc_heap_pages will result in TLB
>>> flushing. But once it calls, the assertion will fail.
>>> 
>>> In my case, the xmalloc is called with starting secondary processors. Some
>>> initialization code run with local irq enabled, for example, the MCA
>>> initialization. Normally this piece of code runs when all heap pages do not
>>> have a former owner (no domain is initialized at booting time, I guess), so
>>> calling xmalloc won't be a problem. But later when this same piece of code
>>> runs as a result of cpu online operation, it has possibility to trigger the
>>> assertion failure.
>>> 
>>> What's you view on this, Keir? Is it the design that xmalloc must be called
>>> with local irq enabled? I have done a hack to remove the assertion. Every
>>> things work just fine to me. But maybe I just happened not to run into any
>>> problem with the hack.
>>> 
>>> Shan Haitao
>>> 
>> 
>> 
> 
>