Live migrations / saving fails with >32gb of ram?

Live migrations / saving fails with >32gb of ram?

[Nathan March]

Hi All,

Having an issue where I can't save or live migrate a VM once it contains
more than 32gb of ram. The hosting dom0 fails immediately with:

[2011-07-25 13:49:01 9265] INFO (XendDomainInfo:2078) Domain has
shutdown: name=migrating-nathanxenuk2 id=1 reason=suspend.
[2011-07-25 13:49:01 9265] DEBUG (XendCheckpoint:144) Written done
[2011-07-25 13:49:01 9265] INFO (XendCheckpoint:423) xc: error: Couldn't
map p2m_frame_list (22 = Invalid argument): Internal error
[2011-07-25 13:49:01 9265] INFO (XendCheckpoint:423) xc: error: Failed
to map/save the p2m frame list (22 = Invalid argument): Internal error
[2011-07-25 13:49:01 9265] ERROR (XendCheckpoint:185) Save failed on
domain nathanxenuk2 (1) - resuming.
Traceback (most recent call last):
  File "/usr/lib64/python2.6/site-packages/xen/xend/XendCheckpoint.py",
line 146, in save
    forkHelper(cmd, fd, saveInputHandler, False)
  File "/usr/lib64/python2.6/site-packages/xen/xend/XendCheckpoint.py",
line 411, in forkHelper
    raise XendError("%s failed" % string.join(cmd))
XendError: /usr/lib/xen/bin/xc_save 26 1 0 0 0 failed
[2011-07-25 13:49:01 9265] DEBUG (XendDomainInfo:3135)
XendDomainInfo.resumeDomain(1)
[2011-07-25 13:49:01 9265] DEBUG (XendDomainInfo:3176)
XendDomainInfo.resumeDomain: completed

Googled around but I can't seem to find anything related with that
error. The same vm and dom0's work fine as soon as I decrease the ram
below the 32gb mark. Anyone have any insight on what might be going on here?

Thanks,
Nathan

-- 
Nathan March <nathan@gt.net>
Gossamer Threads Inc. http://www.gossamer-threads.com/
Tel: (604) 687-5804 Fax: (604) 687-5806

Re: Live migrations / saving fails with >32gb of ram?

[Konrad Rzeszutek Wilk]

On Mon, Jul 25, 2011 at 01:59:42PM -0700, Nathan March wrote:
> Hi All,
> 
> Having an issue where I can't save or live migrate a VM once it contains
> more than 32gb of ram. The hosting dom0 fails immediately with:
> 
> [2011-07-25 13:49:01 9265] INFO (XendDomainInfo:2078) Domain has
> shutdown: name=migrating-nathanxenuk2 id=1 reason=suspend.
> [2011-07-25 13:49:01 9265] DEBUG (XendCheckpoint:144) Written done
> [2011-07-25 13:49:01 9265] INFO (XendCheckpoint:423) xc: error: Couldn't
> map p2m_frame_list (22 = Invalid argument): Internal error
> [2011-07-25 13:49:01 9265] INFO (XendCheckpoint:423) xc: error: Failed
> to map/save the p2m frame list (22 = Invalid argument): Internal error
> [2011-07-25 13:49:01 9265] ERROR (XendCheckpoint:185) Save failed on
> domain nathanxenuk2 (1) - resuming.
> Traceback (most recent call last):
>   File "/usr/lib64/python2.6/site-packages/xen/xend/XendCheckpoint.py",
> line 146, in save
>     forkHelper(cmd, fd, saveInputHandler, False)
>   File "/usr/lib64/python2.6/site-packages/xen/xend/XendCheckpoint.py",
> line 411, in forkHelper
>     raise XendError("%s failed" % string.join(cmd))
> XendError: /usr/lib/xen/bin/xc_save 26 1 0 0 0 failed
> [2011-07-25 13:49:01 9265] DEBUG (XendDomainInfo:3135)
> XendDomainInfo.resumeDomain(1)
> [2011-07-25 13:49:01 9265] DEBUG (XendDomainInfo:3176)
> XendDomainInfo.resumeDomain: completed
> 
> Googled around but I can't seem to find anything related with that
> error. The same vm and dom0's work fine as soon as I decrease the ram
> below the 32gb mark. Anyone have any insight on what might be going on here?

Does [xl|xm] save and [xl|xm] resume work?

Re: Live migrations / saving fails with >32gb of ram?

[Nathan March]

On 7/25/2011 3:05 PM, Konrad Rzeszutek Wilk wrote:
> Does [xl|xm] save and [xl|xm] resume work? 

Sorry should have been more specific, the output above is from "xm save"
but live migrations have the same problem.

xl fails similarly:

ukxen1 ~ # xl save nathanxenuk2 nx2
 Savefile will not contain xl domain config
Saving to nx2 new xl format (info 0x0/0x0/4)
xc: error: Couldn't map p2m_frame_list (22 = Invalid argument): Internal
error
xc: error: Failed to map/save the p2m frame list (22 = Invalid
argument): Internal error
libxl: error: libxl_dom.c:508:libxl__domain_suspend_common saving
domain: domain responded to suspend request: Invalid argument
xl: fatal error: xl_cmdimpl.c:2452: Unknown error 18446744073709551613:
libxl_domain_suspend(&ctx, NULL, domid, fd)

- Nathan

Re: Live migrations / saving fails with >32gb of ram?

[Olaf Hering]

On Mon, Jul 25, Nathan March wrote:

> Having an issue where I can't save or live migrate a VM once it contains
> more than 32gb of ram. The hosting dom0 fails immediately with:
> 
> [2011-07-25 13:49:01 9265] INFO (XendCheckpoint:423) xc: error: Couldn't
> map p2m_frame_list (22 = Invalid argument): Internal error

This has been fixed with this patch:

http://lists.xensource.com/archives/html/xen-devel/2011-07/msg00579.html

Olaf

Re: Live migrations / saving fails with >32gb of ram?

[Nathan March]

On 7/26/2011 3:28 AM, Olaf Hering wrote:
> On Mon, Jul 25, Nathan March wrote:
>
>> Having an issue where I can't save or live migrate a VM once it contains
>> more than 32gb of ram. The hosting dom0 fails immediately with:
>>
>> [2011-07-25 13:49:01 9265] INFO (XendCheckpoint:423) xc: error: Couldn't
>> map p2m_frame_list (22 = Invalid argument): Internal error
> This has been fixed with this patch:
>
> http://lists.xensource.com/archives/html/xen-devel/2011-07/msg00579.html
>
> Olaf
Confirmed as fixing, thanks!

- Nathan