From mboxrd@z Thu Jan 1 00:00:00 1970 From: Andrew Cooper Subject: Re: live-migration restore failed error Date: Mon, 15 Sep 2014 14:59:28 +0100 Message-ID: <5416F0C0.4090308@citrix.com> References: <983a73c3-c2db-4845-92a1-82ef8381b20a@alibaba-inc.com> Mime-Version: 1.0 Content-Type: multipart/mixed; boundary="===============4245388508801836778==" Return-path: In-Reply-To: <983a73c3-c2db-4845-92a1-82ef8381b20a@alibaba-inc.com> List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Sender: xen-devel-bounces@lists.xen.org Errors-To: xen-devel-bounces@lists.xen.org To: =?UTF-8?B?IuW4uOaAgOmRqyjkuIDmlospIg==?= , keir , "Ian.Campbell" , "stefano.stabellini" , xen-devel , "Ian.Jackson" , "george.dunlap" Cc: =?UTF-8?B?IuWImOWKsuadvijlh6/ogLMpIg==?= List-Id: xen-devel@lists.xenproject.org This is a multi-part message in MIME format. --===============4245388508801836778== Content-Type: multipart/alternative; boundary="------------050606000409000806000701" This is a multi-part message in MIME format. --------------050606000409000806000701 Content-Type: text/plain; charset=UTF-8; format=flowed Content-Length: 1997 Content-Transfer-Encoding: quoted-printable On 15/09/2014 10:41, =E5=B8=B8=E6=80=80=E9=91=AB(=E4=B8=80=E6=96=8B) wrote: > We are working on live-migration based on Xen-4.0.1(For history > reason, and meantime we are upgrading our Xen to very latest version). > Restore failed when live migrating ubuntu12.04 on xen-4.0.1. To be > more specific, error occurred when populating memory. Error messages > are as follow: > > [2014-09-12 22:40:40 7331 1189091648] DEBUG (XendCheckpoint:307) > [xc_restore]: /usr/lib64/xen/bin/xc_restore 4 2763 3 4 1 1 1 0 > [2014-09-12 22:40:40 7331 1189091648] DEBUG (XendCheckpoint:428) > Thread-40188 > [2014-09-12 22:40:40 7331 1172306240] INFO (XendCheckpoint:476) > Thread-40188:xc_domain_restore start: p2m_size =3D fefff > [2014-09-12 22:40:40 7331 1172306240] INFO (XendCheckpoint:476) > Thread-40188:Reloading memory pages: 0% > [2014-09-12 22:40:50 7331 1172306240] INFO (XendCheckpoint:476) > Thread-40188:Failed allocation for dom 2763: 128 extents of order 0 > [2014-09-12 22:40:50 7331 1172306240] INFO (XendCheckpoint:476) > Thread-40188:ERROR Internal error: Failed to allocate memory for batch.! > [2014-09-12 22:40:50 7331 1172306240] INFO (XendCheckpoint:476) > Thread-40188: > [2014-09-12 22:40:50 7331 1172306240] INFO (XendCheckpoint:476) > Thread-40188:Restore exit with rc=3D1 > [2014-09-12 22:40:50 7331 1189091648] DEBUG (XendCheckpoint:462) > /usr/lib64/xen/bin/xc_restore 4 2763 3 4 1 1 1 0 failed status 256 > [2014-09-12 22:40:50 7331 1189091648] DEBUG (XendDomainInfo:3845) > XendDomainInfo.destroy: domid=3D2763 > > In this case, populate_physmap terminated with nr_done 127. So > xc_memory_op return 127 while nr_extents equals 128. > > This problem happends once every 1770th live migration or so. As I am > debugging this issue, I'm sending this email to ask for suggestions on > this issue. > > Thanks, > Huaixin Chang Xen is unable to fulfil the allocation request. You have run out of host memory. ~Andrew --------------050606000409000806000701 Content-Type: text/html; charset=UTF-8 Content-Length: 4187 Content-Transfer-Encoding: quoted-printable On 15/09/2014 10:41, =E5=B8=B8=E6=80=80=E9=91=AB(=E4=B8=80=E6=96=8B) wrote:
We are working on live-migration based on Xen-4.0.1(For history reason, and meantime we are upgrading our Xen to very latest version). Restore failed when live migrating ubuntu12.04 on xen-4.0.1. To be more specific, error occurred when populating memory. Error messages are as follow:

[2014-09-12 22:40:40 7331 1189091648] DEBUG (XendCheckpoint:307) [xc_restore]: /usr/lib64/xen/bin/xc_restore 4 2763 3 4 1 1 1 0
[2014-09-12 22:40:40 7331 1189091648] DEBUG (XendCheckpoint:428) Thread-40188
[2014-09-12 22:40:40 7331 1172306240] INFO (XendCheckpoint:476) Thread-40188:xc_domain_restore start: p2m_size =3D fefff
[2014-09-12 22:40:40 7331 1172306240] INFO (XendCheckpoint:476) Thread-40188:Reloading memory pages:=C2=A0=C2=A0 0%
[2014-09-12 22:40:50 7331 1172306240] INFO (XendCheckpoint:476) Thread-40188:Failed allocation for dom 2763: 128 extents of order 0
[2014-09-12 22:40:50 7331 1172306240] INFO (XendCheckpoint:476) Thread-40188:ERROR Internal error: Failed to allocate memory for batch.!
[2014-09-12 22:40:50 7331 1172306240] INFO (XendCheckpoint:476) Thread-40188:
[2014-09-12 22:40:50 7331 1172306240] INFO (XendCheckpoint:476) Thread-40188:Restore exit with rc=3D1
[2014-09-12 22:40:50 7331 1189091648] DEBUG (XendCheckpoint:462) /usr/lib64/xen/bin/xc_restore 4 2763 3 4 1 1 1 0 failed status 256
[2014-09-12 22:40:50 7331 1189091648] DEBUG (XendDomainInfo:3845) XendDomainInfo.destroy: domid=3D2763

In this case, populate_physmap terminated with nr_done 127.=C2=A0 So xc_memory_op return 127 while nr_extents equals 128.

This problem happends once every 1770th live migration or so. As I am debugging this issue, I'm sending this email to ask for suggestions on this issue.

Thanks,
Huaixin Chang

Xen is unable to fulfil the allocation request.=C2=A0 You have run out of host memory.

~Andrew
--------------050606000409000806000701-- --===============4245388508801836778== Content-Type: text/plain; charset="us-ascii" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Content-Disposition: inline _______________________________________________ Xen-devel mailing list Xen-devel@lists.xen.org http://lists.xen.org/xen-devel --===============4245388508801836778==--