From mboxrd@z Thu Jan 1 00:00:00 1970 From: Dulloor Subject: Re: tmem and construct_dom0 memory allocation race Date: Tue, 22 Jun 2010 00:17:21 -0700 Message-ID: References: <8ec6725a-7211-4461-8931-d0196025fb2d@default> Mime-Version: 1.0 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable Return-path: In-Reply-To: <8ec6725a-7211-4461-8931-d0196025fb2d@default> List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Sender: xen-devel-bounces@lists.xensource.com Errors-To: xen-devel-bounces@lists.xensource.com To: Dan Magenheimer Cc: xen-devel@lists.xensource.com, Keir Fraser List-Id: xen-devel@lists.xenproject.org Hi Dan, I am using a pvops kernel. Hi Keir, You are right .. there is no race. I spent some time debugging this. The problem is that a zero-order allocation (from alloc_chunk, for the last dom0 page) fails with tmem on (in alloc_heap_pages), even though there are pages available in the heap. I don't think tmem really intends to get triggered so early. What do you think ? Also, on an unrelated note, the number of pages estimated for dom0 (nr_pages) could be off by (opt_dom0_vcpus/16) pages, due to the perdomain_pt_page allocation (in vcpu_initialise). thanks dulloor On Mon, Jun 21, 2010 at 8:35 AM, Dan Magenheimer wrote: > Hi Dulloor -- > > Intel had previously reported a failure for 2.6.18-xen > dom0+tmem with dom0_mem unspecified. =A0I'm not sure if > this is the same bug or not. > > The latest versions of the Linux-side tmem patch disable > tmem by default (in Linux, not Xen!) and require a kernel > boot option to turn it on. =A0Since dom0 is special and > I've done very little testing with dom0 using tmem (as > tmem is primarily used with guests), I think the correct > (at least short-term) fix for this will be to not enable > tmem for dom0 when dom0_mem is unspecified. =A0I haven't > gotten around to updating 2.6.18-xen for awhile, assuming > it is increasingly rarely used (except in products where > dom0_mem is always specified). > > I'll try to submit a major update to the Linux-side > tmem patch for the 2.6.18-xen tree soon so at least > it is consistent with other Linux-side Xen patches. > > Dan > >> -----Original Message----- >> From: Keir Fraser [mailto:keir.fraser@eu.citrix.com] >> Sent: Saturday, June 19, 2010 1:27 AM >> To: Dulloor; xen-devel@lists.xensource.com >> Subject: Re: [Xen-devel] tmem and construct_dom0 memory allocation race >> >> On 19/06/2010 00:10, "Dulloor" wrote: >> >> > Following is the sequence : >> > - init_tmem allocates a set of pages and sets up dstmem and workmem >> to >> > alloc pages in MP case (with cpu notifiers) >> > - construct_dom0 estimates nr_pages by calling avail_domheap_pages >> > - On other CPUs, tmem cpu_notifier gets called and allocates pages >> > from domheap, making the construct_dom0's estimate stale. >> > - construct_dom0 fails >> > >> > tmem=3Doff or dom0_mem=3Dxxx both solve the problem for now. >> >> Xen boot is pretty serialised. In particular SMP boot, and all cpu >> notification calls, should be done before dom0 is constructed. So, have >> you >> actually seen this race? >> >> =A0-- Keir >> >> >> >> _______________________________________________ >> Xen-devel mailing list >> Xen-devel@lists.xensource.com >> http://lists.xensource.com/xen-devel >