Re: tmem and construct_dom0 memory allocation race

[Dulloor <dulloor@gmail.com>]

Hi Dan, I am using a pvops kernel.

Hi Keir, You are right .. there is no race. I spent some time
debugging this. The problem is that a zero-order allocation (from
alloc_chunk, for the last dom0 page) fails with tmem on (in
alloc_heap_pages), even though there are pages available in the heap.
I don't think tmem really intends to get triggered so early. What do
you think ?

Also, on an unrelated note, the number of pages estimated for dom0
(nr_pages) could be off by (opt_dom0_vcpus/16) pages, due to the
perdomain_pt_page allocation (in vcpu_initialise).

thanks
dulloor

On Mon, Jun 21, 2010 at 8:35 AM, Dan Magenheimer
<dan.magenheimer@oracle.com> wrote:
> Hi Dulloor --
>
> Intel had previously reported a failure for 2.6.18-xen
> dom0+tmem with dom0_mem unspecified.  I'm not sure if
> this is the same bug or not.
>
> The latest versions of the Linux-side tmem patch disable
> tmem by default (in Linux, not Xen!) and require a kernel
> boot option to turn it on.  Since dom0 is special and
> I've done very little testing with dom0 using tmem (as
> tmem is primarily used with guests), I think the correct
> (at least short-term) fix for this will be to not enable
> tmem for dom0 when dom0_mem is unspecified.  I haven't
> gotten around to updating 2.6.18-xen for awhile, assuming
> it is increasingly rarely used (except in products where
> dom0_mem is always specified).
>
> I'll try to submit a major update to the Linux-side
> tmem patch for the 2.6.18-xen tree soon so at least
> it is consistent with other Linux-side Xen patches.
>
> Dan
>
>> -----Original Message-----
>> From: Keir Fraser [mailto:keir.fraser@eu.citrix.com]
>> Sent: Saturday, June 19, 2010 1:27 AM
>> To: Dulloor; xen-devel@lists.xensource.com
>> Subject: Re: [Xen-devel] tmem and construct_dom0 memory allocation race
>>
>> On 19/06/2010 00:10, "Dulloor" <dulloor@gmail.com> wrote:
>>
>> > Following is the sequence :
>> > - init_tmem allocates a set of pages and sets up dstmem and workmem
>> to
>> > alloc pages in MP case (with cpu notifiers)
>> > - construct_dom0 estimates nr_pages by calling avail_domheap_pages
>> > - On other CPUs, tmem cpu_notifier gets called and allocates pages
>> > from domheap, making the construct_dom0's estimate stale.
>> > - construct_dom0 fails
>> >
>> > tmem=off or dom0_mem=xxx both solve the problem for now.
>>
>> Xen boot is pretty serialised. In particular SMP boot, and all cpu
>> notification calls, should be done before dom0 is constructed. So, have
>> you
>> actually seen this race?
>>
>>  -- Keir
>>
>>
>>
>> _______________________________________________
>> Xen-devel mailing list
>> Xen-devel@lists.xensource.com
>> http://lists.xensource.com/xen-devel
>