From mboxrd@z Thu Jan 1 00:00:00 1970 From: Shriram Rajagopalan Subject: Re: xl save -c issues with Windows 7 Ultimate Date: Thu, 5 May 2011 09:42:26 -0500 Message-ID: References: <1304427718.18845.103.camel@zakaz.uk.xensource.com> Reply-To: rshriram@cs.ubc.ca Mime-Version: 1.0 Content-Type: multipart/mixed; boundary="===============1023462677==" Return-path: In-Reply-To: List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Sender: xen-devel-bounces@lists.xensource.com Errors-To: xen-devel-bounces@lists.xensource.com To: AP Xen Cc: "xen-devel@lists.xensource.com" , Ian Campbell List-Id: xen-devel@lists.xenproject.org --===============1023462677== Content-Type: multipart/alternative; boundary=000325556532afc04004a288655a --000325556532afc04004a288655a Content-Type: text/plain; charset=ISO-8859-1 On Tue, May 3, 2011 at 2:17 PM, AP Xen wrote: > On Tue, May 3, 2011 at 7:09 AM, Shriram Rajagopalan > wrote: > > On Tue, May 3, 2011 at 6:01 AM, Ian Campbell > > wrote: > >> > >> On Fri, 2011-04-29 at 20:28 +0100, AP Xen wrote: > >> > I am trying to do an "xl save -c" on a Windows 7 Ultimate domain that > >> > will leave the domain running at the end of the save operation. > >> > >> Do you have pv drivers installed which support checkpoint suspends? I'm > >> not sure if such a thing even exists for Windows. > >> > >> I'm also not entirely sure that checkpointing was ever supported for HVM > >> domains without PV drivers (e.g. via emulated hibernation). Perhaps the > >> Remus guys know? > >> > > Remus works with HVM domains via normal xenstore based suspend/resume. > > Only PV-HVM support is "disabled" for the moment. > >> > >> [...] > >> > At the end of this the domain is frozen. Is this a known issue? Any > >> > pointers as to how to debug this? Where does xl pipe its debug > >> > messages to? > >> > >> /var/log/xen/xl-.log. You can also do "xl -vvv " to > >> get some additional debug output. > >> > > Yes. the logs would be great. Also, by frozen, do you mean the domain > > remains > > in "suspended" state? or is Windows hung? > > Not sure what the difference between Windows being suspended and hung > is. Here is the xl list output: > Name ID Mem VCPUs State > Time(s) > Domain-0 0 2914 4 r----- > 1259.0 > win7 15 1019 2 ---ss- > 0.3 > > Here is the log: > Saving to win7chk new xl format (info 0x0/0x0/255) > libxl: debug: libxl_dom.c:378:libxl__domain_suspend_common_callback > Calling xc_domain_shutdown on HVM domain > libxl: debug: libxl_dom.c:438:libxl__domain_suspend_common_callback > wait for the guest to suspend > libxl: debug: libxl_dom.c:450:libxl__domain_suspend_common_callback > guest has suspended > xc: debug: outbuf_write: 4194304 > 90092@16687124 > xc: debug: outbuf_write: 4194304 > 4169716@12607500 > xc: debug: outbuf_write: 4194304 > 4169716@12607500 > xc: debug: outbuf_write: 4194304 > 4169716@12607500 > xc: debug: outbuf_write: 4194304 > 4169716@12607500 > xc: debug: outbuf_write: 4194304 > 4169716@12607500 > xc: debug: outbuf_write: 4194304 > 4169716@12607500 > xc: debug: outbuf_write: 4194304 > 4169716@12607500 > xc: debug: outbuf_write: 4194304 > 4169716@12607500 > xc: debug: outbuf_write: 4194304 > 4169716@12607500 > xc: debug: outbuf_write: 4194304 > 4169716@12607500 > xc: debug: outbuf_write: 4194304 > 4169716@12607500 > xc: debug: outbuf_write: 4194304 > 4169716@12607500 > xc: debug: outbuf_write: 4194304 > 4169716@12607500 > xc: debug: outbuf_write: 4194304 > 4169716@12607500 > xc: debug: outbuf_write: 4194304 > 4169716@12607500 > xc: debug: outbuf_write: 4194304 > 4169716@12607500 > xc: debug: outbuf_write: 4194304 > 4169716@12607500 > xc: debug: outbuf_write: 4194304 > 4169716@12607500 > xc: debug: outbuf_write: 4194304 > 4169716@12607500 > xc: debug: outbuf_write: 4194304 > 4169716@12607500 > xc: debug: outbuf_write: 4194304 > 4169716@12607500 > xc: debug: outbuf_write: 4194304 > 4169716@12607500 > xc: debug: outbuf_write: 4194304 > 4169716@12607500 > xc: debug: outbuf_write: 4194304 > 4169716@12607500 > xc: debug: outbuf_write: 4194304 > 4169716@12607500 > xc: debug: outbuf_write: 4194304 > 4169716@12607500 > xc: debug: outbuf_write: 4194304 > 4169716@12607500 > xc: debug: outbuf_write: 4194304 > 4169716@12607500 > xc: debug: outbuf_write: 4194304 > 4169716@12607500 > xc: debug: outbuf_write: 4194304 > 4169716@12607500 > xc: debug: outbuf_write: 4194304 > 4169716@12607500 > xc: debug: outbuf_write: 4194304 > 4169716@12607500 > xc: debug: outbuf_write: 4194304 > 4169716@12607500 > xc: debug: outbuf_write: 4194304 > 4169716@12607500 > xc: debug: outbuf_write: 4194304 > 4169716@12607500 > xc: debug: outbuf_write: 4194304 > 4169716@12607500 > xc: debug: outbuf_write: 4194304 > 4169716@12607500 > xc: debug: outbuf_write: 4194304 > 4169716@12607500 > xc: debug: outbuf_write: 4194304 > 4169716@12607500 > xc: debug: outbuf_write: 4194304 > 4169716@12607500 > xc: debug: outbuf_write: 4194304 > 4169716@12607500 > xc: debug: outbuf_write: 4194304 > 4169716@12607500 > xc: debug: outbuf_write: 4194304 > 4169716@12607500 > xc: debug: outbuf_write: 4194304 > 4169716@12607500 > xc: debug: outbuf_write: 4194304 > 4169716@12607500 > xc: debug: outbuf_write: 4194304 > 4169716@12607500 > xc: debug: outbuf_write: 4194304 > 4169716@12607500 > xc: debug: outbuf_write: 4194304 > 4169716@12607500 > xc: debug: outbuf_write: 4194304 > 4169716@12607500 > xc: debug: outbuf_write: 4194304 > 4169716@12607500 > xc: debug: outbuf_write: 4194304 > 4169716@12607500 > xc: debug: outbuf_write: 4194304 > 4169716@12607500 > xc: debug: outbuf_write: 4194304 > 4169716@12607500 > xc: debug: outbuf_write: 4194304 > 4169716@12607500 > xc: debug: outbuf_write: 4194304 > 4169716@12607500 > xc: debug: outbuf_write: 4194304 > 4169716@12607500 > xc: debug: outbuf_write: 4194304 > 4169716@12607500 > xc: debug: outbuf_write: 4194304 > 4169716@12607500 > xc: debug: outbuf_write: 4194304 > 4169716@12607500 > xc: debug: outbuf_write: 4194304 > 4169716@12607500 > xc: debug: outbuf_write: 4194304 > 4169716@12607500 > xc: debug: outbuf_write: 4194304 > 4169716@12607500 > xc: debug: outbuf_write: 4194304 > 4169716@12607500 > xc: debug: outbuf_write: 4194304 > 4169716@12607500 > xc: debug: outbuf_write: 4194304 > 4169716@12607500 > xc: debug: outbuf_write: 4194304 > 4169716@12607500 > xc: debug: outbuf_write: 4194304 > 4169716@12607500 > xc: debug: outbuf_write: 4194304 > 4169716@12607500 > xc: debug: outbuf_write: 4194304 > 4169716@12607500 > xc: debug: outbuf_write: 4194304 > 4169716@12607500 > xc: debug: outbuf_write: 4194304 > 4169716@12607500 > xc: debug: outbuf_write: 4194304 > 4169716@12607500 > xc: debug: outbuf_write: 4194304 > 4169716@12607500 > xc: debug: outbuf_write: 4194304 > 4169716@12607500 > xc: debug: outbuf_write: 4194304 > 4169716@12607500 > xc: debug: outbuf_write: 4194304 > 4169716@12607500 > xc: debug: outbuf_write: 4194304 > 4169716@12607500 > xc: debug: outbuf_write: 4194304 > 4169716@12607500 > xc: debug: outbuf_write: 4194304 > 4169716@12607500 > xc: debug: outbuf_write: 4194304 > 4169716@12607500 > xc: debug: outbuf_write: 4194304 > 4169716@12607500 > xc: debug: outbuf_write: 4194304 > 4169716@12607500 > xc: debug: outbuf_write: 4194304 > 4169716@12607500 > xc: detail: delta 9991ms, dom0 27%, target 0%, sent 863Mb/s, dirtied > 0Mb/s 0 pages > xc: detail: Total pages sent= 263168 (0.25x) > xc: detail: (of which 0 were fixups) > xc: detail: All memory is saved > xc: detail: Save exit rc=0 > libxl: debug: libxl_dom.c:534:libxl__domain_save_device_model Saving > device model state to /var/lib/xen/qemu-save.15 > libxl: debug: libxl_dom.c:546:libxl__domain_save_device_model Qemu > state is 7204 bytes > > > Ok. I see a "HVM shutdown". But where is the resume? Going through the libxl code, one obvious difference I see between xl's implementation of save and the old xm implementation is, xl calls "xc_domain_unpause" while the xm implementation calls xc_domain_resume. Just in case, have you tried the same with xm save -c ? shriram > > shriram > >> > >> Ian. > >> > >> > > > > > > --000325556532afc04004a288655a Content-Type: text/html; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable
On Tue, May 3, 2011 at 2:17 PM, AP Xen <apxeng@gmail.com> wrote:
On Tue, May 3, 2011 at 7:09 AM, Shriram Rajagopalan <<= a href=3D"mailto:rshriram@cs.ubc.ca">rshriram@cs.ubc.ca> wrote:
&= gt; On Tue, May 3, 2011 at 6:01 AM, Ian Campbell <Ian.Campbell@citrix.com>
> wrote:
>>
>> On Fri, 2011-04-29 at 20:28 +0100, AP X= en wrote:
>> > I am trying to do an "xl save -c" on a= Windows 7 Ultimate domain that
>> > will leave the domain runn= ing at the end of the save operation.
>>
>> Do you have pv drivers installed which support checkpo= int suspends? I'm
>> not sure if such a thing even exists for = Windows.
>>
>> I'm also not entirely sure that checkp= ointing was ever supported for HVM
>> domains without PV drivers (e.g. via emulated hibernation). Perhap= s the
>> Remus guys know?
>>
> Remus works with HVM= domains via normal xenstore based suspend/resume.
> Only PV-HVM supp= ort is "disabled" for the moment.
>>
>> [...]
>> > At the end of this the domain i= s frozen. Is this a known issue? Any
>> > pointers as to how to= debug this? Where does xl pipe its debug
>> > messages to?
>>
>> /var/log/xen/xl-<domname>.log. You can also do &= quot;xl -vvv <command>" to
>> get some additional debug= output.
>>
> Yes. the logs would be great. Also, by frozen,= do you mean the domain
> remains
> in "suspended" state? or is Windows hung?
Not sure what the difference between Windows being suspend= ed and hung
is. Here is the xl list output:
Name =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 = =A0 =A0 =A0 =A0 =A0 =A0 =A0ID =A0 Mem VCPUs =A0 =A0 =A0State =A0 Time(s)
Domain-0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0= =A0 =A0 =A0 0 =A02914 =A0 =A0 4 =A0 =A0 r----- =A0 =A01259.0
win7 =A0 = =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0= 15 =A01019 =A0 =A0 2 =A0 =A0 ---ss- =A0 =A0 =A0 0.3

Here is the log:
Saving to win7chk new xl format (info 0x0/0x0/255)<= br>libxl: debug: libxl_dom.c:378:libxl__domain_suspend_common_callback
C= alling xc_domain_shutdown on HVM domain
libxl: debug: libxl_dom.c:438:li= bxl__domain_suspend_common_callback
wait for the guest to suspend
libxl: debug: libxl_dom.c:450:libxl__domai= n_suspend_common_callback
guest has suspended
xc: debug: outbuf_write= : 4194304 > 90092@16687124
xc: debug: outbuf_write: 4194304 > 4169= 716@12607500
xc: debug: outbuf_write: 4194304 > 4169716@12607500
xc: debug: outbuf= _write: 4194304 > 4169716@12607500
xc: debug: outbuf_write: 4194304 &= gt; 4169716@12607500
xc: debug: outbuf_write: 4194304 > 4169716@12607= 500
xc: debug: outbuf_write: 4194304 > 4169716@12607500
xc: debug: outbuf= _write: 4194304 > 4169716@12607500
xc: debug: outbuf_write: 4194304 &= gt; 4169716@12607500
xc: debug: outbuf_write: 4194304 > 4169716@12607= 500
xc: debug: outbuf_write: 4194304 > 4169716@12607500
xc: debug: outbuf= _write: 4194304 > 4169716@12607500
xc: debug: outbuf_write: 4194304 &= gt; 4169716@12607500
xc: debug: outbuf_write: 4194304 > 4169716@12607= 500
xc: debug: outbuf_write: 4194304 > 4169716@12607500
xc: debug: outbuf= _write: 4194304 > 4169716@12607500
xc: debug: outbuf_write: 4194304 &= gt; 4169716@12607500
xc: debug: outbuf_write: 4194304 > 4169716@12607= 500
xc: debug: outbuf_write: 4194304 > 4169716@12607500
xc: debug: outbuf= _write: 4194304 > 4169716@12607500
xc: debug: outbuf_write: 4194304 &= gt; 4169716@12607500
xc: debug: outbuf_write: 4194304 > 4169716@12607= 500
xc: debug: outbuf_write: 4194304 > 4169716@12607500
xc: debug: outbuf= _write: 4194304 > 4169716@12607500
xc: debug: outbuf_write: 4194304 &= gt; 4169716@12607500
xc: debug: outbuf_write: 4194304 > 4169716@12607= 500
xc: debug: outbuf_write: 4194304 > 4169716@12607500
xc: debug: outbuf= _write: 4194304 > 4169716@12607500
xc: debug: outbuf_write: 4194304 &= gt; 4169716@12607500
xc: debug: outbuf_write: 4194304 > 4169716@12607= 500
xc: debug: outbuf_write: 4194304 > 4169716@12607500
xc: debug: outbuf= _write: 4194304 > 4169716@12607500
xc: debug: outbuf_write: 4194304 &= gt; 4169716@12607500
xc: debug: outbuf_write: 4194304 > 4169716@12607= 500
xc: debug: outbuf_write: 4194304 > 4169716@12607500
xc: debug: outbuf= _write: 4194304 > 4169716@12607500
xc: debug: outbuf_write: 4194304 &= gt; 4169716@12607500
xc: debug: outbuf_write: 4194304 > 4169716@12607= 500
xc: debug: outbuf_write: 4194304 > 4169716@12607500
xc: debug: outbuf= _write: 4194304 > 4169716@12607500
xc: debug: outbuf_write: 4194304 &= gt; 4169716@12607500
xc: debug: outbuf_write: 4194304 > 4169716@12607= 500
xc: debug: outbuf_write: 4194304 > 4169716@12607500
xc: debug: outbuf= _write: 4194304 > 4169716@12607500
xc: debug: outbuf_write: 4194304 &= gt; 4169716@12607500
xc: debug: outbuf_write: 4194304 > 4169716@12607= 500
xc: debug: outbuf_write: 4194304 > 4169716@12607500
xc: debug: outbuf= _write: 4194304 > 4169716@12607500
xc: debug: outbuf_write: 4194304 &= gt; 4169716@12607500
xc: debug: outbuf_write: 4194304 > 4169716@12607= 500
xc: debug: outbuf_write: 4194304 > 4169716@12607500
xc: debug: outbuf= _write: 4194304 > 4169716@12607500
xc: debug: outbuf_write: 4194304 &= gt; 4169716@12607500
xc: debug: outbuf_write: 4194304 > 4169716@12607= 500
xc: debug: outbuf_write: 4194304 > 4169716@12607500
xc: debug: outbuf= _write: 4194304 > 4169716@12607500
xc: debug: outbuf_write: 4194304 &= gt; 4169716@12607500
xc: debug: outbuf_write: 4194304 > 4169716@12607= 500
xc: debug: outbuf_write: 4194304 > 4169716@12607500
xc: debug: outbuf= _write: 4194304 > 4169716@12607500
xc: debug: outbuf_write: 4194304 &= gt; 4169716@12607500
xc: debug: outbuf_write: 4194304 > 4169716@12607= 500
xc: debug: outbuf_write: 4194304 > 4169716@12607500
xc: debug: outbuf= _write: 4194304 > 4169716@12607500
xc: debug: outbuf_write: 4194304 &= gt; 4169716@12607500
xc: debug: outbuf_write: 4194304 > 4169716@12607= 500
xc: debug: outbuf_write: 4194304 > 4169716@12607500
xc: debug: outbuf= _write: 4194304 > 4169716@12607500
xc: debug: outbuf_write: 4194304 &= gt; 4169716@12607500
xc: debug: outbuf_write: 4194304 > 4169716@12607= 500
xc: debug: outbuf_write: 4194304 > 4169716@12607500
xc: debug: outbuf= _write: 4194304 > 4169716@12607500
xc: debug: outbuf_write: 4194304 &= gt; 4169716@12607500
xc: debug: outbuf_write: 4194304 > 4169716@12607= 500
xc: debug: outbuf_write: 4194304 > 4169716@12607500
xc: debug: outbuf= _write: 4194304 > 4169716@12607500
xc: debug: outbuf_write: 4194304 &= gt; 4169716@12607500
xc: debug: outbuf_write: 4194304 > 4169716@12607= 500
xc: debug: outbuf_write: 4194304 > 4169716@12607500
xc: debug: outbuf= _write: 4194304 > 4169716@12607500
xc: debug: outbuf_write: 4194304 &= gt; 4169716@12607500
xc: debug: outbuf_write: 4194304 > 4169716@12607= 500
xc: debug: outbuf_write: 4194304 > 4169716@12607500
xc: debug: outbuf= _write: 4194304 > 4169716@12607500
xc: detail: delta 9991ms, dom0 27%= , target 0%, sent 863Mb/s, dirtied
0Mb/s 0 pages
xc: detail: Total pages sent=3D 263168 (= 0.25x)
xc: detail: (of which 0 were fixups)
xc: detail: All memory is= saved
xc: detail: Save exit rc=3D0
libxl: debug: libxl_dom.c:5= 34:libxl__domain_save_device_model Saving
device model state to /var/lib/xen/qemu-save.15
libxl: debug: libxl_dom.= c:546:libxl__domain_save_device_model Qemu
state is 7204 bytes

Ok.=A0I see a "HVM shutdown". But where is the resume?
Going through the libxl code, one obvious difference I see between xl&= #39;s implementation of save
and the old xm implementation is, xl calls "xc_domain_unpause&quo= t; while the xm implementation
calls xc_domain_resume.
=A0
Just in case, have you tried the same with xm save -c ?
shriram
> shriram
>>
>= > Ian.
>>
>>
>
>


--000325556532afc04004a288655a-- --===============1023462677== Content-Type: text/plain; charset="us-ascii" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Content-Disposition: inline _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel --===============1023462677==--