From mboxrd@z Thu Jan 1 00:00:00 1970 From: Joshua West Subject: Re: Migrate/Save of 32-bit domU Broken on Xen 3.1.2 64-bit dom0 Date: Fri, 18 Jan 2008 18:29:16 -0500 Message-ID: <4791364C.50004@brandeis.edu> References: <20080118161753234.00000001968@djm-pc> Mime-Version: 1.0 Content-Type: text/plain; charset=iso-8859-1; format=flowed Content-Transfer-Encoding: 7bit Return-path: In-Reply-To: <20080118161753234.00000001968@djm-pc> List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Sender: xen-devel-bounces@lists.xensource.com Errors-To: xen-devel-bounces@lists.xensource.com To: "dan.magenheimer@oracle.com" Cc: Kurt Hackel , "xen-devel@lists.xensource.com" List-Id: xen-devel@lists.xenproject.org Hi Dan, Dan Magenheimer wrote: > Joshua -- > > Why do you need so much memory on dom0? Are you thinking > that the total memory available to guests is limited to > what dom0 has? Or are you running some memory-hungry > applications on dom0? > Hmm, I thought that a 32-bit dom0 system can only have up to 16GB of memory available to it and its domU's -- and a 64-bit dom0 can have a max of 32GB. I only allocate ~512MB of memory to the dom0's themselves. "xm info" shows us the rest of whats available for use by domU's. Thus, even if I have 32GB of RAM in a server, the total memory that will be used in dom0 + all domU's can only add up to 16GB. Am I just plain old wrong? > Note that 64-bit Xen + 32-bit Dom0 will support both 32-bit > and 64-bit domU's. (Oracle VM uses this configuration by > default on 64-bit capable machines.) And save/restore/migration > has been tested to work for this (with the patch Kurt posted). > Interesting. For some reason I thought that the Hypervisor had to match the dom0's kernel. So you're saying that if I compile the Xen Hypervisor as 64-bit and my dom0 + domU's both as 32-bit, I can reach the full capacity of my servers with respect to RAM (32GB) instead of being limited to 16GB? (And this only works with Kurt's patches?). > Dan > > >> -----Original Message----- >> From: xen-devel-bounces@lists.xensource.com >> [mailto:xen-devel-bounces@lists.xensource.com]On Behalf Of Joshua West >> Sent: Friday, January 18, 2008 4:06 PM >> To: Keir Fraser >> Cc: Kurt Hackel; xen-devel@lists.xensource.com >> Subject: Re: [Xen-devel] Migrate/Save of 32-bit domU Broken >> on Xen 3.1.2 >> 64-bit dom0 >> >> >> Dang, doesn't work in Xen 3.1.3-rc1 (like you said/expected), with or >> without Kurt's patches. >> >> Hmm... I thought 32-bit paravirtualized domU's on 64-bit dom0's were >> supported in Xen 3.1.x? Or am I mistaken? >> >> Perhaps standard features such as start/shutdown/pause of 32-bit PV >> domU's on 64-bit dom0's are supported, but not the more advanced >> management features like save and migrate - and I was just >> unclear from >> the beginning? >> >> And of course... the final question that all developers hate... when >> s/could one _hope_ (not expect!) to see save/migrate functionality of >> 32-bit PV domU's on 64-bit dom0's? :-D >> >> Our whole reason for moving to 64-bit dom0's was to utilize 32GB of >> memory, instead of the 16GB limitation on 32-bit PAE dom0's. We can't >> convert all of our virtual machines (100+) to 64-bit linux as its too >> much work and some actually have to be 32-bit due to vendor supplied >> software (Oracle, Zimbra, etc). I suppose we'll wait to move >> to 64-bit >> until [live] migration is available, as its used quite >> frequently in our >> machine rooms. >> >> Thanks for all of your help Keir and Kurt. >> >> Keir Fraser wrote: >> >>> Looking at the issues that need to be handled to support >>> >> this, it cannot be >> >>> expected to work. >>> >>> -- Keir >>> >>> On 18/1/08 22:13, "Joshua West" wrote: >>> >>> >>> >>>> Ahh. Well, I'm building xen-3.1.3-rc1 right now and I'll >>>> >> see how that goes. >> >>>> Keir, have you seen this issue before? Where one cannot >>>> >> xm save/migrate >> >>>> a 32-bit domU on a 64-bit dom0? Or is this expected to >>>> >> work and just >> >>>> doesn't go well for me? >>>> >>>> Keir Fraser wrote: >>>> >>>> >>>>> On 18/1/08 22:05, "Joshua West" wrote: >>>>> >>>>> >>>>> >>>>> >>>>>> Just to make sure the patches aren't the cause of this new error >>>>>> message, I'm going to see what happens with Xen >>>>>> >> 3.1.3-rc1 w/o patches. >> >>>>>> Kurt, Keir, everybody else... any ideas as to why "xm >>>>>> >> save" (and thus >> >>>>>> "xm migrate") of 32-bit domU's does not work on 64-bit >>>>>> >> dom0's? Has >> >>>>>> anybody found this to be successful in the past? >>>>>> >>>>>> >>>>>> >>>>> The extent of the patches that Kurt posted indicate its >>>>> >> pretty unlikely to >> >>>>> work without considerable code modification. I guess >>>>> >> those patches aren't >> >>>>> quite fully baked yet, unfortunately, or haven't been >>>>> >> written with your >> >>>>> precise case in mind. >>>>> >>>>> -- Keir >>>>> >>>>> >>>>> >>>>> >>>>> >>> >>> >> -- >> Joshua West >> Systems Engineer >> Brandeis University >> http://www.brandeis.edu >> >> >> _______________________________________________ >> Xen-devel mailing list >> Xen-devel@lists.xensource.com >> http://lists.xensource.com/xen-devel >> >> > > -- Joshua West Systems Engineer Brandeis University http://www.brandeis.edu