S3 sleep in dom0 breaks dom0<->domU wallclock synchronization

S3 sleep in dom0 breaks dom0<->domU wallclock synchronization

[Rafal Wojtczuk]

Hello,
xen-3.4.3 x86_64, dom0 2.6.34-9.xenlinux as dom0,
2.6.32.14-1.2.105.pvops0 in PV domU. /proc/sys/xen/independent_wallclock is
0.
After resume from S3 sleep in dom0, the wall clock in domU is
desynchronized from dom0's one (the delta is the length of S3 sleep). It
does not seem that adj_timex is in progress (the delta is constant in time).

Is it a known issue ? If so, could someone point me to a solution ?
Regards,
Rafal Wojtczuk

Re: S3 sleep in dom0 breaks dom0<->domU wallclock synchronization

[Keir Fraser]

On 01/07/2010 04:04, "Rafal Wojtczuk" <rafal@invisiblethingslab.com> wrote:

> Hello,
> xen-3.4.3 x86_64, dom0 2.6.34-9.xenlinux as dom0,
> 2.6.32.14-1.2.105.pvops0 in PV domU. /proc/sys/xen/independent_wallclock is
> 0.
> After resume from S3 sleep in dom0, the wall clock in domU is
> desynchronized from dom0's one (the delta is the length of S3 sleep). It
> does not seem that adj_timex is in progress (the delta is constant in time).
> 
> Is it a known issue ? If so, could someone point me to a solution ?

I think that pv_ops domU kernels pick up Xen's wallclock at boot time, but
won't listen for updates thereafter. So if you ran a non-pv_ops domU, you'd
probably find that its wallclock would be correctly updated after S3. Cc'ing
Jeremy as he'll be able to confirm this. I think his answer will be that you
should run ntp in every guest, but I'm not sure how that will react to
unexpected warps in time.

 -- Keir

> Regards,
> Rafal Wojtczuk
> 
> _______________________________________________
> Xen-devel mailing list
> Xen-devel@lists.xensource.com
> http://lists.xensource.com/xen-devel

Re: S3 sleep in dom0 breaks dom0<->domU wallclock synchronization

[Keir Fraser]

On 01/07/2010 15:50, "Keir Fraser" <keir.fraser@eu.citrix.com> wrote:

> On 01/07/2010 04:04, "Rafal Wojtczuk" <rafal@invisiblethingslab.com> wrote:
> 
>> Hello,
>> xen-3.4.3 x86_64, dom0 2.6.34-9.xenlinux as dom0,
>> 2.6.32.14-1.2.105.pvops0 in PV domU. /proc/sys/xen/independent_wallclock is
>> 0.
>> After resume from S3 sleep in dom0, the wall clock in domU is
>> desynchronized from dom0's one (the delta is the length of S3 sleep). It
>> does not seem that adj_timex is in progress (the delta is constant in time).
>> 
>> Is it a known issue ? If so, could someone point me to a solution ?
> 
> I think that pv_ops domU kernels pick up Xen's wallclock at boot time, but
> won't listen for updates thereafter. So if you ran a non-pv_ops domU, you'd
> probably find that its wallclock would be correctly updated after S3. Cc'ing
> Jeremy as he'll be able to confirm this. I think his answer will be that you
> should run ntp in every guest, but I'm not sure how that will react to
> unexpected warps in time.

Actually cc'ing Jeremy this time...

>  -- Keir
> 
>> Regards,
>> Rafal Wojtczuk
>> 
>> _______________________________________________
>> Xen-devel mailing list
>> Xen-devel@lists.xensource.com
>> http://lists.xensource.com/xen-devel
> 

Re: S3 sleep in dom0 breaks dom0<->domU wallclock synchronization

[Joanna Rutkowska]

[-- Attachment #1.1: Type: text/plain, Size: 1535 bytes --]

On 07/01/10 16:50, Keir Fraser wrote:
> On 01/07/2010 04:04, "Rafal Wojtczuk" <rafal@invisiblethingslab.com> wrote:
> 
>> Hello,
>> xen-3.4.3 x86_64, dom0 2.6.34-9.xenlinux as dom0,
>> 2.6.32.14-1.2.105.pvops0 in PV domU. /proc/sys/xen/independent_wallclock is
>> 0.
>> After resume from S3 sleep in dom0, the wall clock in domU is
>> desynchronized from dom0's one (the delta is the length of S3 sleep). It
>> does not seem that adj_timex is in progress (the delta is constant in time).
>>
>> Is it a known issue ? If so, could someone point me to a solution ?
> 
> I think that pv_ops domU kernels pick up Xen's wallclock at boot time, but
> won't listen for updates thereafter. So if you ran a non-pv_ops domU, you'd
> probably find that its wallclock would be correctly updated after S3. Cc'ing
> Jeremy as he'll be able to confirm this. I think his answer will be that you
> should run ntp in every guest, but I'm not sure how that will react to
> unexpected warps in time.
> 
Actually we're running a pvops kernel in DomUs (in fact a fairly recent
pvops0, as we had some bad experience with regular Fedora kernels in DomU).

Running an NTP in every VM is not a good solution. Some VMs might be
forbidden any access to the network (e.g. my "vault" VM, that I use for
storing passwords, and other very sensitive stuff, doesn't have any
networking), while some other might be allowed only very limited network
traffic, e.g. only HTTPS to specific, white-listed servers (e.g.
"banking" VM).

joanna.


[-- Attachment #1.2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 226 bytes --]

[-- Attachment #2: Type: text/plain, Size: 138 bytes --]

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

Re: S3 sleep in dom0 breaks dom0<->domU wallclock synchronization

[Keir Fraser]

On 01/07/2010 16:18, "Joanna Rutkowska" <joanna@invisiblethingslab.com>
wrote:

> Actually we're running a pvops kernel in DomUs (in fact a fairly recent
> pvops0, as we had some bad experience with regular Fedora kernels in DomU).
> 
> Running an NTP in every VM is not a good solution. Some VMs might be
> forbidden any access to the network (e.g. my "vault" VM, that I use for
> storing passwords, and other very sensitive stuff, doesn't have any
> networking), while some other might be allowed only very limited network
> traffic, e.g. only HTTPS to specific, white-listed servers (e.g.
> "banking" VM).

Well it would be good to confirm first that this is a pv_ops domU issue. If
so, it can probably be solved with a command-line option or somesuch, even
if the default policy will not change.

 -- Keir

Re: S3 sleep in dom0 breaks dom0<->domU wallclock synchronization

[Jeremy Fitzhardinge]

On 07/01/2010 09:12 AM, Keir Fraser wrote:
> On 01/07/2010 16:18, "Joanna Rutkowska" <joanna@invisiblethingslab.com>
> wrote:
>
>   
>> Actually we're running a pvops kernel in DomUs (in fact a fairly recent
>> pvops0, as we had some bad experience with regular Fedora kernels in DomU).
>>
>> Running an NTP in every VM is not a good solution. Some VMs might be
>> forbidden any access to the network (e.g. my "vault" VM, that I use for
>> storing passwords, and other very sensitive stuff, doesn't have any
>> networking), while some other might be allowed only very limited network
>> traffic, e.g. only HTTPS to specific, white-listed servers (e.g.
>> "banking" VM).
>>     
> Well it would be good to confirm first that this is a pv_ops domU issue. If
> so, it can probably be solved with a command-line option or somesuch, even
> if the default policy will not change.
>   

So the problem is that dom0 does the S3 suspend/resume, and presumably
its wallclock time is updated properly via Linux's normal mechanisms. 
But the S3 suspend/resume is unnoticed by all the domUs, so they don't
know that an enormous amount of time has passed in an instant?  Does
that affect all the guest clocks, or just wallclock?

How does Xen deal with the S3 suspend/resume?  Does the system clock
just keep ticking as usual (so the whole suspended time appears to be
sub-nanosecond), but the wallclock offset gets updated?  Or does it try
to workout how long the suspended time was and adjusts the system time
accordingly?  That would allow guest timekeeping to compensate for the
suspended time, assuming they can deal with large forward leaps.

What happens with pending timers?

    J

Re: S3 sleep in dom0 breaks dom0<->domU wallclock synchronization

[Keir Fraser]

On 05/07/2010 20:18, "Jeremy Fitzhardinge" <jeremy@goop.org> wrote:

> So the problem is that dom0 does the S3 suspend/resume, and presumably
> its wallclock time is updated properly via Linux's normal mechanisms.
> But the S3 suspend/resume is unnoticed by all the domUs, so they don't
> know that an enormous amount of time has passed in an instant?  Does
> that affect all the guest clocks, or just wallclock?

Um, just wallclock I think?

> How does Xen deal with the S3 suspend/resume?  Does the system clock
> just keep ticking as usual (so the whole suspended time appears to be
> sub-nanosecond), but the wallclock offset gets updated?

This.

 -- Keir

> Or does it try
> to workout how long the suspended time was and adjusts the system time
> accordingly?  That would allow guest timekeeping to compensate for the
> suspended time, assuming they can deal with large forward leaps.

Re: S3 sleep in dom0 breaks dom0<->domU wallclock synchronization

[Joanna Rutkowska]

[-- Attachment #1.1: Type: text/plain, Size: 1976 bytes --]

On 07/05/10 21:18, Jeremy Fitzhardinge wrote:
> On 07/01/2010 09:12 AM, Keir Fraser wrote:
>> On 01/07/2010 16:18, "Joanna Rutkowska" <joanna@invisiblethingslab.com>
>> wrote:
>>
>>   
>>> Actually we're running a pvops kernel in DomUs (in fact a fairly recent
>>> pvops0, as we had some bad experience with regular Fedora kernels in DomU).
>>>
>>> Running an NTP in every VM is not a good solution. Some VMs might be
>>> forbidden any access to the network (e.g. my "vault" VM, that I use for
>>> storing passwords, and other very sensitive stuff, doesn't have any
>>> networking), while some other might be allowed only very limited network
>>> traffic, e.g. only HTTPS to specific, white-listed servers (e.g.
>>> "banking" VM).
>>>     
>> Well it would be good to confirm first that this is a pv_ops domU issue. If
>> so, it can probably be solved with a command-line option or somesuch, even
>> if the default policy will not change.
>>   
> 
> So the problem is that dom0 does the S3 suspend/resume, and presumably
> its wallclock time is updated properly via Linux's normal mechanisms.

Yes.

> But the S3 suspend/resume is unnoticed by all the domUs, so they don't
> know that an enormous amount of time has passed in an instant?

Correct. I don't think DomU are notified in any way about system suspend
-- at least nothing is in the dmesg/messages logs.

BTW: wouldn't it be good to actually notify them? Consider e.g. DomU
that has some device assigned to it (say a NIC) -- if we emulated S3
suspend/resume for this DomU, there is a hope it would properly
suspend/reinitialize the NIC, wouldn't it?

> Does that affect all the guest clocks, or just wallclock?
> 
Not sure if I understand your question -- what do you mean by "all guest
clocks"? Like timers? They don't seem to be affected [*], as the apps
run smoothly.

joanna.

[*] Except for when running on Core i5 -- see my other question in a
different thread.


[-- Attachment #1.2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 226 bytes --]

[-- Attachment #2: Type: text/plain, Size: 138 bytes --]

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

Re: S3 sleep in dom0 breaks dom0<->domU wallclock synchronization

[Jeremy Fitzhardinge]

On 07/05/2010 03:43 PM, Joanna Rutkowska wrote:
>> But the S3 suspend/resume is unnoticed by all the domUs, so they don't
>> know that an enormous amount of time has passed in an instant?
>>     
> Correct. I don't think DomU are notified in any way about system suspend
> -- at least nothing is in the dmesg/messages logs.
>
> BTW: wouldn't it be good to actually notify them? Consider e.g. DomU
> that has some device assigned to it (say a NIC) -- if we emulated S3
> suspend/resume for this DomU, there is a hope it would properly
> suspend/reinitialize the NIC, wouldn't it?
>   

I guess?  That implies some kind of PV S3 suspend and resume event to
feed into the dom U's device model.  What does 2.6.18-xen do?

As far as time goes, I think it makes most sense to try and resuse as
much of the existing kernel machinery as possible, rather than inventing
anything new.

I wonder if it makes sense to do a PV checkpoint-resume to PV domains on
host S3?

>> Does that affect all the guest clocks, or just wallclock?
>>
>>     
> Not sure if I understand your question -- what do you mean by "all guest
> clocks"? Like timers? They don't seem to be affected [*], as the apps
> run smoothly.
>   

I mean the other clocks you can real with clock_gettime(), such as
CLOCK_MONOTONIC or CLOCK_PROCESS_CPUTIME_ID, in addition to CLOCK_REALTIME.

    J

Re: S3 sleep in dom0 breaks dom0<->domU wallclock synchronization

[Joanna Rutkowska]

[-- Attachment #1.1: Type: text/plain, Size: 1862 bytes --]

On 07/06/10 00:50, Jeremy Fitzhardinge wrote:
> On 07/05/2010 03:43 PM, Joanna Rutkowska wrote:
>>> But the S3 suspend/resume is unnoticed by all the domUs, so they don't
>>> know that an enormous amount of time has passed in an instant?
>>>     
>> Correct. I don't think DomU are notified in any way about system suspend
>> -- at least nothing is in the dmesg/messages logs.
>>
>> BTW: wouldn't it be good to actually notify them? Consider e.g. DomU
>> that has some device assigned to it (say a NIC) -- if we emulated S3
>> suspend/resume for this DomU, there is a hope it would properly
>> suspend/reinitialize the NIC, wouldn't it?
>>   
> 
> I guess?  That implies some kind of PV S3 suspend and resume event to
> feed into the dom U's device model.  What does 2.6.18-xen do?
> 
> As far as time goes, I think it makes most sense to try and resuse as
> much of the existing kernel machinery as possible, rather than inventing
> anything new.
> 
> I wonder if it makes sense to do a PV checkpoint-resume to PV domains on
> host S3?
> 
>>> Does that affect all the guest clocks, or just wallclock?
>>>
>>>     
>> Not sure if I understand your question -- what do you mean by "all guest
>> clocks"? Like timers? They don't seem to be affected [*], as the apps
>> run smoothly.
>>   
> 
> I mean the other clocks you can real with clock_gettime(), such as
> CLOCK_MONOTONIC or CLOCK_PROCESS_CPUTIME_ID, in addition to CLOCK_REALTIME.
> 

I guess all these other clocks have nothing to do with RTC/wallclock,
right? They are effectively just like any other system timers (they are
system timers), and they are updated by the timer interrupt and they
don't need RTC to be happy? So, I would say they are fine, just like all
the other timers, as otherwise I would expect DomUs to explode/hang
after resume.

joanna.


[-- Attachment #1.2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 226 bytes --]

[-- Attachment #2: Type: text/plain, Size: 138 bytes --]

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

Re: S3 sleep in dom0 breaks dom0<->domU wallclock synchronization

[Jeremy Fitzhardinge]

On 07/05/2010 04:03 PM, Joanna Rutkowska wrote:
> I guess all these other clocks have nothing to do with RTC/wallclock,
> right? They are effectively just like any other system timers (they are
> system timers), and they are updated by the timer interrupt and they
> don't need RTC to be happy? So, I would say they are fine, just like all
> the other timers, as otherwise I would expect DomUs to explode/hang
> after resume.
>   

CLOCK_MONOTONIC is directly derived from the Xen system time, so it will
presumably pause while the machine is suspended.  CLOCK_REALTIME is just
the Xen system time with an offset added to make the normal Unix TOD; if
the offset isn't updated after resume, then it will also not advance
over the suspend.

    J

Re: S3 sleep in dom0 breaks dom0<->domU wallclock synchronization

[Joanna Rutkowska]

[-- Attachment #1.1: Type: text/plain, Size: 914 bytes --]

On 07/06/10 01:19, Jeremy Fitzhardinge wrote:
> On 07/05/2010 04:03 PM, Joanna Rutkowska wrote:
>> I guess all these other clocks have nothing to do with RTC/wallclock,
>> right? They are effectively just like any other system timers (they are
>> system timers), and they are updated by the timer interrupt and they
>> don't need RTC to be happy? So, I would say they are fine, just like all
>> the other timers, as otherwise I would expect DomUs to explode/hang
>> after resume.
>>   
> 
> CLOCK_MONOTONIC is directly derived from the Xen system time, so it will
> presumably pause while the machine is suspended.  CLOCK_REALTIME is just
> the Xen system time with an offset added to make the normal Unix TOD; if
> the offset isn't updated after resume, then it will also not advance
> over the suspend.
> 

What is an easy way to dump all those clocks, like e.g. via some /proc
entry?

j.


[-- Attachment #1.2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 226 bytes --]

[-- Attachment #2: Type: text/plain, Size: 138 bytes --]

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

Re: S3 sleep in dom0 breaks dom0<->domU wallclock synchronization

[Jeremy Fitzhardinge]

On 07/06/2010 02:12 AM, Joanna Rutkowska wrote:
>> CLOCK_MONOTONIC is directly derived from the Xen system time, so it will
>> presumably pause while the machine is suspended.  CLOCK_REALTIME is just
>> the Xen system time with an offset added to make the normal Unix TOD; if
>> the offset isn't updated after resume, then it will also not advance
>> over the suspend.
>>
>>     
> What is an easy way to dump all those clocks, like e.g. via some /proc
> entry?
>   

They're available via the clock_gettime function in librt.  I don't know
if there's a standard utility which allows them to be displayed/called.

    J

Re: S3 sleep in dom0 breaks dom0<->domU wallclock synchronization

[Keir Fraser]

On 05/07/2010 23:50, "Jeremy Fitzhardinge" <jeremy@goop.org> wrote:

>> BTW: wouldn't it be good to actually notify them? Consider e.g. DomU
>> that has some device assigned to it (say a NIC) -- if we emulated S3
>> suspend/resume for this DomU, there is a hope it would properly
>> suspend/reinitialize the NIC, wouldn't it?
>>   
> 
> I guess?  That implies some kind of PV S3 suspend and resume event to
> feed into the dom U's device model.  What does 2.6.18-xen do?

I don't think our S3 support is very compatible with PV device passthrough.
We support HVM virtual S3, and can S3-sleep HVM guests across real host S3,
but we don't have similar for PV guests.

 -- Keir

Re: S3 sleep in dom0 breaks dom0<->domU wallclock synchronization

[Joanna Rutkowska]

[-- Attachment #1.1: Type: text/plain, Size: 1552 bytes --]

On 07/06/10 05:52, Keir Fraser wrote:
> On 05/07/2010 23:50, "Jeremy Fitzhardinge" <jeremy@goop.org> wrote:
> 
>>> BTW: wouldn't it be good to actually notify them? Consider e.g. DomU
>>> that has some device assigned to it (say a NIC) -- if we emulated S3
>>> suspend/resume for this DomU, there is a hope it would properly
>>> suspend/reinitialize the NIC, wouldn't it?
>>>   
>>
>> I guess?  That implies some kind of PV S3 suspend and resume event to
>> feed into the dom U's device model.  What does 2.6.18-xen do?
> 
> I don't think our S3 support is very compatible with PV device passthrough.
> We support HVM virtual S3, and can S3-sleep HVM guests across real host S3,
> but we don't have similar for PV guests.
> 

How about implementing something very simple, like a notification via
xenstore (say, Dom0 would be setting some key)? Interested DomUs could
then register a watch, and get notified when the system was resumed from
S3. This would let them e.g. to call whatever hypercall is used normally
on DomU boot to sync DomU wallclock, or reinitialize/reconnect the NIC.

Obviously DomUs would not be notified when the system is just going to
sleep, as this would require some more sophisticated protocol (I guess
each DomU would have to ack within some given max timeout that it's done
with preparing from sleep?). But perhaps we can just ignore it? Even if
DomU has a NIC card assigned, wouldn't it be put to sleep by the
southbridge anyway? So, seems like we only care about the resume event?

joanna.


[-- Attachment #1.2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 226 bytes --]

[-- Attachment #2: Type: text/plain, Size: 138 bytes --]

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

Re: S3 sleep in dom0 breaks dom0<->domU wallclock synchronization

[Jan Beulich]

>>> On 06.07.10 at 11:10, Joanna Rutkowska <joanna@invisiblethingslab.com> wrote:
> On 07/06/10 05:52, Keir Fraser wrote:
>> On 05/07/2010 23:50, "Jeremy Fitzhardinge" <jeremy@goop.org> wrote:
>> 
>>>> BTW: wouldn't it be good to actually notify them? Consider e.g. DomU
>>>> that has some device assigned to it (say a NIC) -- if we emulated S3
>>>> suspend/resume for this DomU, there is a hope it would properly
>>>> suspend/reinitialize the NIC, wouldn't it?
>>>>   
>>>
>>> I guess?  That implies some kind of PV S3 suspend and resume event to
>>> feed into the dom U's device model.  What does 2.6.18-xen do?
>> 
>> I don't think our S3 support is very compatible with PV device passthrough.
>> We support HVM virtual S3, and can S3-sleep HVM guests across real host S3,
>> but we don't have similar for PV guests.
>> 
> 
> How about implementing something very simple, like a notification via
> xenstore (say, Dom0 would be setting some key)? Interested DomUs could
> then register a watch, and get notified when the system was resumed from
> S3. This would let them e.g. to call whatever hypercall is used normally
> on DomU boot to sync DomU wallclock, or reinitialize/reconnect the NIC.

Wouldn't it be much simpler to not introduce any new logic at all and
just let Dom0 tools/scripts take care of properly suspending
(checkpointing) all (minimally all pv, but I would really think treating
different kinds of guests differently here is unnecessary) guests
before doing a host suspend, as Jeremy had suggested in an earlier
reply?

Jan

Re: S3 sleep in dom0 breaks dom0<->domU wallclock synchronization

[Joanna Rutkowska]

[-- Attachment #1.1: Type: text/plain, Size: 1784 bytes --]

On 07/06/10 12:02, Jan Beulich wrote:
>>>> On 06.07.10 at 11:10, Joanna Rutkowska <joanna@invisiblethingslab.com> wrote:
>> On 07/06/10 05:52, Keir Fraser wrote:
>>> On 05/07/2010 23:50, "Jeremy Fitzhardinge" <jeremy@goop.org> wrote:
>>>
>>>>> BTW: wouldn't it be good to actually notify them? Consider e.g. DomU
>>>>> that has some device assigned to it (say a NIC) -- if we emulated S3
>>>>> suspend/resume for this DomU, there is a hope it would properly
>>>>> suspend/reinitialize the NIC, wouldn't it?
>>>>>   
>>>>
>>>> I guess?  That implies some kind of PV S3 suspend and resume event to
>>>> feed into the dom U's device model.  What does 2.6.18-xen do?
>>>
>>> I don't think our S3 support is very compatible with PV device passthrough.
>>> We support HVM virtual S3, and can S3-sleep HVM guests across real host S3,
>>> but we don't have similar for PV guests.
>>>
>>
>> How about implementing something very simple, like a notification via
>> xenstore (say, Dom0 would be setting some key)? Interested DomUs could
>> then register a watch, and get notified when the system was resumed from
>> S3. This would let them e.g. to call whatever hypercall is used normally
>> on DomU boot to sync DomU wallclock, or reinitialize/reconnect the NIC.
> 
> Wouldn't it be much simpler to not introduce any new logic at all and
> just let Dom0 tools/scripts take care of properly suspending
> (checkpointing) all (minimally all pv, but I would really think treating
> different kinds of guests differently here is unnecessary) guests
> before doing a host suspend, as Jeremy had suggested in an earlier
> reply?
> 

But wouldn't this require dumping all the VMs memory do disk? Can we use
xm pause instead, i.e. will it notify VMs properly?

j.


[-- Attachment #1.2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 226 bytes --]

[-- Attachment #2: Type: text/plain, Size: 138 bytes --]

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

Re: S3 sleep in dom0 breaks dom0<->domU wallclock synchronization

[Keir Fraser]

On 06/07/2010 11:27, "Joanna Rutkowska" <joanna@invisiblethingslab.com>
wrote:

>> Wouldn't it be much simpler to not introduce any new logic at all and
>> just let Dom0 tools/scripts take care of properly suspending
>> (checkpointing) all (minimally all pv, but I would really think treating
>> different kinds of guests differently here is unnecessary) guests
>> before doing a host suspend, as Jeremy had suggested in an earlier
>> reply?
>> 
> 
> But wouldn't this require dumping all the VMs memory do disk? Can we use
> xm pause instead, i.e. will it notify VMs properly?

It just requires the guests to be put through a guest-side suspend/resume
cycle. The usual tools-side work of saving to disk etc could be elided. Yes,
it probably makes sense to extend that existing mechanism to include S3 as
well. Just needs someone to do the work. :-)

 -- Keir

Re: S3 sleep in dom0 breaks dom0<->domU wallclock synchronization

[Joanna Rutkowska]

[-- Attachment #1.1: Type: text/plain, Size: 1022 bytes --]

On 07/06/10 14:50, Keir Fraser wrote:
> On 06/07/2010 11:27, "Joanna Rutkowska" <joanna@invisiblethingslab.com>
> wrote:
> 
>>> Wouldn't it be much simpler to not introduce any new logic at all and
>>> just let Dom0 tools/scripts take care of properly suspending
>>> (checkpointing) all (minimally all pv, but I would really think treating
>>> different kinds of guests differently here is unnecessary) guests
>>> before doing a host suspend, as Jeremy had suggested in an earlier
>>> reply?
>>>
>>
>> But wouldn't this require dumping all the VMs memory do disk? Can we use
>> xm pause instead, i.e. will it notify VMs properly?
> 
> It just requires the guests to be put through a guest-side suspend/resume
> cycle. The usual tools-side work of saving to disk etc could be elided. Yes,
> it probably makes sense to extend that existing mechanism to include S3 as
> well. Just needs someone to do the work. :-)
> 

Perhaps it will also solve the resume problem on Core i5/HT systems...

joanna.


[-- Attachment #1.2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 226 bytes --]

[-- Attachment #2: Type: text/plain, Size: 138 bytes --]

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

Re: S3 sleep in dom0 breaks dom0<->domU wallclock synchronization

[Joanna Rutkowska]

[-- Attachment #1.1: Type: text/plain, Size: 1435 bytes --]

On 07/06/10 16:09, Joanna Rutkowska wrote:
> On 07/06/10 14:50, Keir Fraser wrote:
>> On 06/07/2010 11:27, "Joanna Rutkowska" <joanna@invisiblethingslab.com>
>> wrote:
>>
>>>> Wouldn't it be much simpler to not introduce any new logic at all and
>>>> just let Dom0 tools/scripts take care of properly suspending
>>>> (checkpointing) all (minimally all pv, but I would really think treating
>>>> different kinds of guests differently here is unnecessary) guests
>>>> before doing a host suspend, as Jeremy had suggested in an earlier
>>>> reply?
>>>>
>>>
>>> But wouldn't this require dumping all the VMs memory do disk? Can we use
>>> xm pause instead, i.e. will it notify VMs properly?
>>
>> It just requires the guests to be put through a guest-side suspend/resume
>> cycle. The usual tools-side work of saving to disk etc could be elided. Yes,
>> it probably makes sense to extend that existing mechanism to include S3 as
>> well. Just needs someone to do the work. :-)
>>
> 
> Perhaps it will also solve the resume problem on Core i5/HT systems...
> 

For now, I have added another pm-utils hook that does something like
this on every system resume:

DATE=$(date)
qvm-run --all -u root "date -s \"$DATE\""

The qvm-run command is Qubes-specific, but I can imagine one could add a
similar functionality to xm.

I've been using this for a few days now, and it seems to work very well.

joanna.


[-- Attachment #1.2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 226 bytes --]

[-- Attachment #2: Type: text/plain, Size: 138 bytes --]

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

RE: S3 sleep in dom0 breaks dom0<->domU wallclock synchronization

[Dan Magenheimer]

> >> Wouldn't it be much simpler to not introduce any new logic at all
> and
> >> just let Dom0 tools/scripts take care of properly suspending
> >> (checkpointing) all (minimally all pv, but I would really think
> treating
> >> different kinds of guests differently here is unnecessary) guests
> >> before doing a host suspend, as Jeremy had suggested in an earlier
> >> reply?
> >>
> >
> > But wouldn't this require dumping all the VMs memory do disk? Can we
> use
> > xm pause instead, i.e. will it notify VMs properly?
> 
> It just requires the guests to be put through a guest-side
> suspend/resume
> cycle. The usual tools-side work of saving to disk etc could be elided.
> Yes,
> it probably makes sense to extend that existing mechanism to include S3
> as
> well. Just needs someone to do the work. :-)

In the meantime, is there any easy way to stop dom0
from entering S3?  (E.g., like the Xen max_cstate parameter
for C states?)

I've been seeing some similar guest issues recently running
RHEL6beta guests and wonder if I am seeing the same problem.

Re: S3 sleep in dom0 breaks dom0<->domU wallclock synchronization

[Jeremy Fitzhardinge]

On 07/06/2010 07:53 AM, Dan Magenheimer wrote:
> In the meantime, is there any easy way to stop dom0
> from entering S3?  (E.g., like the Xen max_cstate parameter
> for C states?)
>
> I've been seeing some similar guest issues recently running
> RHEL6beta guests and wonder if I am seeing the same problem.
>   

S3 is lid-close suspend on a laptop, or putting a desktop into sleep
mode.  It isn't something that occurs except when user initiated (or by
policy, such as suspend on low battery).

    J