ACPI shutdown unreliable with win7?

ACPI shutdown unreliable with win7?

[Ian Campbell]

In osstest self gate flight 56412 a change to make HVM guests
configurably use "xl shutdown -F" (-F == use ACPI fallback if PV drivers
absent) and apply the option to the win7 and winxpsp3 tests went into
production.

On winxpsp3 this appears to have been a success and things are working
pretty well since that flight:
http://logs.test-lab.xenproject.org/osstest/results/history.test-amd64-i386-xl-qemuu-winxpsp3.html
http://logs.test-lab.xenproject.org/osstest/results/history.test-amd64-i386-xl-qemuu-winxpsp3-vcpus1.html
http://logs.test-lab.xenproject.org/osstest/results/history.test-amd64-amd64-xl-qemut-winxpsp3.html
http://logs.test-lab.xenproject.org/osstest/results/history.test-amd64-i386-xl-qemut-winxpsp3.html
http://logs.test-lab.xenproject.org/osstest/results/history.test-amd64-i386-xl-qemut-winxpsp3-vcpus1.html

However win7 seems to be in somewhat poorer shape:

http://logs.test-lab.xenproject.org/osstest/results/history.test-amd64-amd64-xl-qemuu-win7-amd64.html
http://logs.test-lab.xenproject.org/osstest/results/history.test-amd64-i386-xl-qemuu-win7-amd64.html
http://logs.test-lab.xenproject.org/osstest/results/history.test-amd64-amd64-xl-qemut-win7-amd64.html
http://logs.test-lab.xenproject.org/osstest/results/history.test-amd64-i386-xl-qemut-win7-amd64.html

it passes less than one time in 10 (of the ones which don't get blocked
earlier on).

The problem seems to be independent of the qemu version and with qemuu
of seabios vs rombios. dom0 kernel also seems irrelevant.

Some random examples which I looked at:
http://logs.test-lab.xenproject.org/osstest/logs/56929/test-amd64-amd64-xl-qemut-win7-amd64/info.html
http://logs.test-lab.xenproject.org/osstest/logs/56929/test-amd64-i386-xl-qemuu-win7-amd64/info.html
http://logs.test-lab.xenproject.org/osstest/logs/56718/test-amd64-amd64-xl-qemut-win7-amd64/info.html

all showed in there vnc screenshot a guest which is showing no
indication of shutting down.

Looking a bit closer at the first 56929 one xl list shows the windows
domain 18 in state blocked.

The host serial log shows an apparently successful restore into dom18,
then the debug key output after failure show CPU1, corresponding to
d18v0, sat in hvm_io_pending:
        May 22 01:27:58.889074 (XEN) *** Dumping CPU1 host state: ***
        May 22 01:27:58.897025 (XEN) ----[ Xen-4.6-unstable  x86_64  debug=y  Not tainted ]----
        May 22 01:27:58.904993 (XEN) CPU:    1
        May 22 01:27:58.905036 (XEN) RIP:    e008:[<ffff82d0801bdce5>] hvm_io_pending+0/0x52
        May 22 01:27:58.912996 (XEN) RFLAGS: 0000000000000297   CONTEXT: hypervisor (d18v0)
        May 22 01:27:58.913056 (XEN) rax: 0000000000000000   rbx: ffff8300cceeb000   rcx: 00000000fed000f0
        May 22 01:27:58.920996 (XEN) rdx: 0000000000000004   rsi: 00000000fed000f4   rdi: ffff8300cceeb000
        May 22 01:27:58.928997 (XEN) rbp: ffff83022b4bf5b8   rsp: ffff83022b4bf520   r8:  0000000000000001
        May 22 01:27:58.936987 (XEN) r9:  0000000000000000   r10: 0000000000000000   r11: fffff80000b9c9e8
        May 22 01:27:58.944986 (XEN) r12: ffff830219db5000   r13: 0000000000000004   r14: ffff82e0024433a0
        May 22 01:27:58.953029 (XEN) r15: 0000000000000000   cr0: 0000000080050033   cr4: 00000000001526f0
        May 22 01:27:58.960994 (XEN) cr3: 000000010c38e000   cr2: fffff8a0007d1538
        May 22 01:27:58.968987 (XEN) ds: 0000   es: 0000   fs: 0000   gs: 0000   ss: 0000   cs: e008
        May 22 01:27:58.969027 (XEN) Xen stack trace from rsp=ffff83022b4bf520:
        May 22 01:27:58.976990 (XEN)    ffff82d0801b8300 0000000000000001 ffff83022b4bfb60 ffff83022b4bf628
        May 22 01:27:58.985003 (XEN)    0000000100000002 00000000fed000f0 0100000000000001 ffff83022b4bf578
        May 22 01:27:58.993017 (XEN)    00000000801b8d1c 00000000fed000f0 0000000000000000 0000000400000000
        May 22 01:27:59.001010 (XEN)    0120000000000000 ffff83022b4bf630 0000000000000004 0000000000000004
        May 22 01:27:59.009040 (XEN)    00000000000000f0 ffff83022b4bf9a8 0000000000000002 ffff83022b4bf5d8
        May 22 01:27:59.017016 (XEN)    ffff82d0801b85fb ffff830200000000 ffff83022b4bf9a8 ffff83022b4bf668
        May 22 01:27:59.025011 (XEN)    ffff82d0801b9086 ffff83022b4bf9a8 ffff82d0801b85fb ffff830200000000
        May 22 01:27:59.033009 (XEN)    0000000229000000 ffff83022b4bfb60 ffff8300000000f4 ffff83022b4bf978
        May 22 01:27:59.040990 (XEN)    00000000fed000f0 0000000000000001 ffffffffffd090f0 ffff83022b4bf688
        May 22 01:27:59.041030 (XEN)    000000000000008b 0000000000000000 0000000000000000 ffff82d08028d900
        May 22 01:27:59.048993 (XEN)    ffff83022b4bfb60 ffff83022b4bf678 ffff82d0801b92b8 ffff83022b4bf688
        May 22 01:27:59.057007 (XEN)    ffff82d0801958de ffff83022b4bfac8 ffff82d080197875 ffff83022b4bf6c8
        May 22 01:27:59.064999 (XEN)    ffff82d0801f7b1c ffff83012c9ea008 0000000000000001 0000000000000080
        May 22 01:27:59.073007 (XEN)    00000000000001d9 ffff830200000001 ffff82d0801f890f 8086000000008086
        May 22 01:27:59.080990 (XEN)    0000000200020001 ffffffffffd090f0 dc00000000000008 ffff83022b4b0000
        May 22 01:27:59.089011 (XEN)    0000000900000000 00000000ffffff18 ffff830100000004 0000000000000144
        May 22 01:27:59.097029 (XEN)    ffff830100000008 ffff82d0801b92b8 0000000000000000 ffff83012c9ea010
        May 22 01:27:59.105006 (XEN)    ffff83022b4bf8c4 00000000000000f0 ffff83022b4bf7f8 ffff83022b4bf800
        May 22 01:27:59.112998 (XEN)    0000000000000012 000000000012c9e8 0000000000000001 ffff83022b4bf7a8
        May 22 01:27:59.121016 (XEN)    ffff82d0801f7b1c ffff83012c9ea010 0000000000000001 0000000000000000
        May 22 01:27:59.129007 (XEN) Xen call trace:
        May 22 01:27:59.129072 (XEN)    [<ffff82d0801bdce5>] hvm_io_pending+0/0x52
        May 22 01:27:59.137022 (XEN)    [<ffff82d0801b85fb>] hvmemul_do_mmio+0x2d/0x2f
        May 22 01:27:59.137060 (XEN)    [<ffff82d0801b9086>] __hvmemul_read+0x11c/0x29c
        May 22 01:27:59.145011 (XEN)    [<ffff82d0801b92b8>] hvmemul_read+0x12/0x14
        May 22 01:27:59.152979 (XEN)    [<ffff82d0801958de>] read_ulong+0xe/0x10
        May 22 01:27:59.153012 (XEN)    [<ffff82d080197875>] x86_emulate+0x1577/0x10362
        May 22 01:27:59.161001 (XEN)    [<ffff82d0801b8a5a>] _hvm_emulate_one+0x197/0x2bb
        May 22 01:27:59.169007 (XEN)    [<ffff82d0801b8c3f>] hvm_emulate_one+0x10/0x12
        May 22 01:27:59.176996 (XEN)    [<ffff82d0801c8274>] handle_mmio+0x54/0xd4
        May 22 01:27:59.177033 (XEN)    [<ffff82d0801c8338>] handle_mmio_with_translation+0x44/0x46
        May 22 01:27:59.185009 (XEN)    [<ffff82d0801c6624>] hvm_hap_nested_page_fault+0x163/0x541
        May 22 01:27:59.193029 (XEN)    [<ffff82d0801e7df6>] vmx_vmexit_handler+0x16a8/0x1a63
        May 22 01:27:59.201048 (XEN)    [<ffff82d0801ee091>] vmx_asm_vmexit_handler+0x41/0xc0
        May 22 01:27:59.201088 (XEN) 
        May 22 01:27:59.209011 (XEN) *** Dumping CPU1 guest state (d18v0): ***
        May 22 01:27:59.209060 (XEN) ----[ Xen-4.6-unstable  x86_64  debug=y  Not tainted ]----
        May 22 01:27:59.217002 (XEN) CPU:    1
        May 22 01:27:59.217018 (XEN) RIP:    0010:[<fffff8000260eeea>]
        May 22 01:27:59.225029 (XEN) RFLAGS: 0000000000010246   CONTEXT: hvm guest (d18v0)
        May 22 01:27:59.225067 (XEN) rax: ffffffffffd09000   rbx: fffffa8001404000   rcx: 0000001280000b9f
        May 22 01:27:59.233042 (XEN) rdx: 000000000342b947   rsi: 0000000000006ec8   rdi: 0000000000000000
        May 22 01:27:59.241045 (XEN) rbp: 0000000000010000   rsp: fffff80000b9c828   r8:  fffff80000b9c900
        May 22 01:27:59.249008 (XEN) r9:  0000000003b9aca0   r10: 0000000000000000   r11: fffff80000b9c9e8
        May 22 01:27:59.256996 (XEN) r12: 0000000000000020   r13: fffffa80006e7010   r14: fffff80000b9c948
        May 22 01:27:59.265034 (XEN) r15: 0000000000000000   cr0: 0000000080050031   cr4: 00000000000406f8
        May 22 01:27:59.273026 (XEN) cr3: 0000000000187000   cr2: fffff8a0007d1538
        May 22 01:27:59.280994 (XEN) ds: 002b   es: 002b   fs: 0053   gs: 002b   ss: 0018   cs: 0010
        May 22 01:27:59.288972 (XEN) 

 and later:
        May 22 01:28:33.044999 (XEN) General information for domain 18:
        May 22 01:28:33.045034 (XEN)     refcnt=3 dying=0 pause_count=0
        May 22 01:28:33.053003 (XEN)     nr_pages=131074 xenheap_pages=6 shared_pages=0 paged_pages=0 dirty_cpus={1-2} max_pages=131328
        May 22 01:28:33.061035 (XEN)     handle=e2333298-4442-4ff6-aaaf-fd6168c5c8f7 vm_assist=00000000
        May 22 01:28:33.068986 (XEN)     paging assistance: hap refcounts translate external 
        May 22 01:28:33.076975 (XEN) Rangesets belonging to domain 18:
        May 22 01:28:33.077009 (XEN)     I/O Ports  { }
        May 22 01:28:33.085015 (XEN)     log-dirty  { }
        May 22 01:28:33.085047 (XEN)     Interrupts { }
        May 22 01:28:33.085073 (XEN)     I/O Memory { }
        May 22 01:28:33.093013 (XEN) Memory pages belonging to domain 18:
        May 22 01:28:33.093046 (XEN)     DomPage list too long to display
        May 22 01:28:33.101054 (XEN)     PoD entries=0 cachesize=0
        May 22 01:28:33.101098 (XEN)     XenPage 000000000012c8af: caf=c000000000000001, taf=7400000000000001
        May 22 01:28:33.108981 (XEN)     XenPage 000000000012c9ee: caf=c000000000000001, taf=7400000000000001
        May 22 01:28:33.116979 (XEN)     XenPage 000000000010c39d: caf=c000000000000001, taf=7400000000000001
        May 22 01:28:33.125010 (XEN)     XenPage 000000000012c9f5: caf=c000000000000001, taf=7400000000000001
        May 22 01:28:33.132998 (XEN)     XenPage 00000000000ccef2: caf=c000000000000001, taf=7400000000000001
        May 22 01:28:33.141013 (XEN)     XenPage 000000000012c8ec: caf=c000000000000001, taf=7400000000000001
        May 22 01:28:33.148973 (XEN) NODE affinity for domain 18: [0]
        May 22 01:28:33.149028 (XEN) VCPU information and callbacks for domain 18:
        May 22 01:28:33.156972 (XEN)     VCPU0: CPU2 [has=F] poll=0 upcall_pend=00 upcall_mask=00 dirty_cpus={2}
        May 22 01:28:33.165038 (XEN)     cpu_hard_affinity={0-47} cpu_soft_affinity={0-3}
        May 22 01:28:33.173029 (XEN)     pause_count=0 pause_flags=1
        May 22 01:28:33.173064 (XEN)     paging assistance: hap, 4 levels
        May 22 01:28:33.181020 (XEN)     No periodic timer
        May 22 01:28:33.181052 (XEN)     VCPU1: CPU1 [has=F] poll=0 upcall_pend=00 upcall_mask=00 dirty_cpus={1}
        May 22 01:28:33.189031 (XEN)     cpu_hard_affinity={0-47} cpu_soft_affinity={0-3}
        May 22 01:28:33.197069 (XEN)     pause_count=0 pause_flags=1
        May 22 01:28:33.197105 (XEN)     paging assistance: hap, 4 levels
        May 22 01:28:33.205013 (XEN)     No periodic timer

The xl log just shows it waiting for the domain to die.

The qemu log
http://logs.test-lab.xenproject.org/osstest/logs/56929/test-amd64-amd64-xl-qemut-win7-amd64/elbling1---var-log-xen-qemu-dm-win.guest.osstest--incoming.log seems uninteresting.

Can anything be gleaned from this?

Ian.

Re: ACPI shutdown unreliable with win7?

[Andrew Cooper]

On 22/05/15 09:54, Ian Campbell wrote:
> In osstest self gate flight 56412 a change to make HVM guests
> configurably use "xl shutdown -F" (-F == use ACPI fallback if PV drivers
> absent) and apply the option to the win7 and winxpsp3 tests went into
> production.
>
> On winxpsp3 this appears to have been a success and things are working
> pretty well since that flight:
> http://logs.test-lab.xenproject.org/osstest/results/history.test-amd64-i386-xl-qemuu-winxpsp3.html
> http://logs.test-lab.xenproject.org/osstest/results/history.test-amd64-i386-xl-qemuu-winxpsp3-vcpus1.html
> http://logs.test-lab.xenproject.org/osstest/results/history.test-amd64-amd64-xl-qemut-winxpsp3.html
> http://logs.test-lab.xenproject.org/osstest/results/history.test-amd64-i386-xl-qemut-winxpsp3.html
> http://logs.test-lab.xenproject.org/osstest/results/history.test-amd64-i386-xl-qemut-winxpsp3-vcpus1.html
>
> However win7 seems to be in somewhat poorer shape:
>
> http://logs.test-lab.xenproject.org/osstest/results/history.test-amd64-amd64-xl-qemuu-win7-amd64.html
> http://logs.test-lab.xenproject.org/osstest/results/history.test-amd64-i386-xl-qemuu-win7-amd64.html
> http://logs.test-lab.xenproject.org/osstest/results/history.test-amd64-amd64-xl-qemut-win7-amd64.html
> http://logs.test-lab.xenproject.org/osstest/results/history.test-amd64-i386-xl-qemut-win7-amd64.html
>
> it passes less than one time in 10 (of the ones which don't get blocked
> earlier on).
>
> The problem seems to be independent of the qemu version and with qemuu
> of seabios vs rombios. dom0 kernel also seems irrelevant.

This is because there is no such thing as ACPI shutdown.

There is ACPI "Someone pressed the power button" and ACPI "Someone
closed the laptop lid".

I believe Win7 attempts to sleep by default, rather than to shut down,
although IIRC this does depend on which S states are offered in the ACPI
tables.  One thing to try would be to boot windows without any S3 or S4
SSDT tables, which should persuade windows into believing that "someone
pressed the power button" should mean S5 shutdown.

~Andrew

Re: ACPI shutdown unreliable with win7?

[Ian Campbell]

On Fri, 2015-05-22 at 09:59 +0100, Andrew Cooper wrote:
> On 22/05/15 09:54, Ian Campbell wrote:
> > In osstest self gate flight 56412 a change to make HVM guests
> > configurably use "xl shutdown -F" (-F == use ACPI fallback if PV drivers
> > absent) and apply the option to the win7 and winxpsp3 tests went into
> > production.
> >
> > On winxpsp3 this appears to have been a success and things are working
> > pretty well since that flight:
> > http://logs.test-lab.xenproject.org/osstest/results/history.test-amd64-i386-xl-qemuu-winxpsp3.html
> > http://logs.test-lab.xenproject.org/osstest/results/history.test-amd64-i386-xl-qemuu-winxpsp3-vcpus1.html
> > http://logs.test-lab.xenproject.org/osstest/results/history.test-amd64-amd64-xl-qemut-winxpsp3.html
> > http://logs.test-lab.xenproject.org/osstest/results/history.test-amd64-i386-xl-qemut-winxpsp3.html
> > http://logs.test-lab.xenproject.org/osstest/results/history.test-amd64-i386-xl-qemut-winxpsp3-vcpus1.html
> >
> > However win7 seems to be in somewhat poorer shape:
> >
> > http://logs.test-lab.xenproject.org/osstest/results/history.test-amd64-amd64-xl-qemuu-win7-amd64.html
> > http://logs.test-lab.xenproject.org/osstest/results/history.test-amd64-i386-xl-qemuu-win7-amd64.html
> > http://logs.test-lab.xenproject.org/osstest/results/history.test-amd64-amd64-xl-qemut-win7-amd64.html
> > http://logs.test-lab.xenproject.org/osstest/results/history.test-amd64-i386-xl-qemut-win7-amd64.html
> >
> > it passes less than one time in 10 (of the ones which don't get blocked
> > earlier on).
> >
> > The problem seems to be independent of the qemu version and with qemuu
> > of seabios vs rombios. dom0 kernel also seems irrelevant.
> 
> This is because there is no such thing as ACPI shutdown.
> 
> There is ACPI "Someone pressed the power button" and ACPI "Someone
> closed the laptop lid".

Understood, however this did work in my manual testing and does
occasionally pass.

e.g. in
http://logs.test-lab.xenproject.org/osstest/logs/56759/test-amd64-amd64-xl-qemut-win7-amd64/info.html which was successful the qemu log shows:
        shutdown requested in cpu_handle_ioreq
        Issued domain 18 poweroff
        Log-dirty: no command yet.

In the failure cases there is no indication in the logs or screenshots
that windows is either hibernating or shutting down, or doing anything
else at all.

> I believe Win7 attempts to sleep by default, rather than to shut down,
> although IIRC this does depend on which S states are offered in the ACPI
> tables.

osstest ought to be providing the exact same thing each time, so unless
there is some non-determinism in win7's logic (I would hope not) I think
it ought to be making the same choice each time.

>   One thing to try would be to boot windows without any S3 or S4
> SSDT tables, which should persuade windows into believing that "someone
> pressed the power button" should mean S5 shutdown.

If win7 doesn't shutdown given a power button request I'd be more
inclined to remove the setting in osstest for those flights and let
guest-stop go back to being never pass than to start making such changes
to the VM config which I think would probably break the preceding
suspend and migration tests (which aren't completely reliable, but are
far more so than this shutdown one).

Ian.

Re: ACPI shutdown unreliable with win7?

[Ian Campbell]

On Fri, 2015-05-22 at 10:08 +0100, Ian Campbell wrote:
> If win7 doesn't shutdown given a power button request I'd be more
> inclined to remove the setting in osstest for those flights and let
> guest-stop go back to being never pass than to start making such changes
> to the VM config which I think would probably break the preceding
> suspend and migration tests (which aren't completely reliable, but are
> far more so than this shutdown one).

Does anyone have any ideas here or shall I propose:

-----8>--------------

>From 2d1b814e65676c3cf56ce2e569491953607f53f7 Mon Sep 17 00:00:00 2001
From: Ian Campbell <ian.campbell@citrix.com>
Date: Fri, 29 May 2015 13:51:33 +0100
Subject: [PATCH] Turn off acpi_shutdown for windows 7

As described in <1432284841.10746.136.camel@citrix.com> /
http://lists.xen.org/archives/html/xen-devel/2015-05/msg03016.html
Windows 7 does not appear to reliably actually shutdown when asked to
via the ACPI power button.

Once this patch is applied some force pushes will likely be needed in
order for this to not appear as a regression.

Signed-off-by: Ian Campbell <ian.campbell@citrix.com>
Cc: Andrew Cooper <Andrew.Cooper3@citrix.com>
Cc: Paul Durrant <paul.durrant@citrix.com>
Cc: Ian Jackson <Ian.Jackson@eu.citrix.com>,
Cc: Jan Beulich <JBeulich@suse.com>
---
Alternatively could make it an allowed/non-blocking failure?
---
 make-flight | 1 -
 1 file changed, 1 deletion(-)

diff --git a/make-flight b/make-flight
index 8a1fceb..5120891 100755
--- a/make-flight
+++ b/make-flight
@@ -206,7 +206,6 @@ do_hvm_win7_x64_tests () {
   job_create_test test-$xenarch$kern-$dom0arch-xl$qemuu_suffix-win7-amd64 \
             test-win xl $xenarch $dom0arch $qemuu_runvar \
             win_image=win7-x64.iso \
-            win_acpi_shutdown=true \
             all_hostflags=$most_hostflags,hvm
 }
 
-- 
2.1.4

Re: ACPI shutdown unreliable with win7?

[Jan Beulich]

>>> On 29.05.15 at 14:54, <ian.campbell@citrix.com> wrote:
> On Fri, 2015-05-22 at 10:08 +0100, Ian Campbell wrote:
>> If win7 doesn't shutdown given a power button request I'd be more
>> inclined to remove the setting in osstest for those flights and let
>> guest-stop go back to being never pass than to start making such changes
>> to the VM config which I think would probably break the preceding
>> suspend and migration tests (which aren't completely reliable, but are
>> far more so than this shutdown one).
> 
> Does anyone have any ideas here or shall I propose:

Unless we have a way to make an adjustment inside the guest for the
power button to gain "shutdown" meaning, I think there's no alternative
to the change below.

Jan

> -----8>--------------
> 
> From 2d1b814e65676c3cf56ce2e569491953607f53f7 Mon Sep 17 00:00:00 2001
> From: Ian Campbell <ian.campbell@citrix.com>
> Date: Fri, 29 May 2015 13:51:33 +0100
> Subject: [PATCH] Turn off acpi_shutdown for windows 7
> 
> As described in <1432284841.10746.136.camel@citrix.com> /
> http://lists.xen.org/archives/html/xen-devel/2015-05/msg03016.html 
> Windows 7 does not appear to reliably actually shutdown when asked to
> via the ACPI power button.
> 
> Once this patch is applied some force pushes will likely be needed in
> order for this to not appear as a regression.
> 
> Signed-off-by: Ian Campbell <ian.campbell@citrix.com>
> Cc: Andrew Cooper <Andrew.Cooper3@citrix.com>
> Cc: Paul Durrant <paul.durrant@citrix.com>
> Cc: Ian Jackson <Ian.Jackson@eu.citrix.com>,
> Cc: Jan Beulich <JBeulich@suse.com>
> ---
> Alternatively could make it an allowed/non-blocking failure?
> ---
>  make-flight | 1 -
>  1 file changed, 1 deletion(-)
> 
> diff --git a/make-flight b/make-flight
> index 8a1fceb..5120891 100755
> --- a/make-flight
> +++ b/make-flight
> @@ -206,7 +206,6 @@ do_hvm_win7_x64_tests () {
>    job_create_test test-$xenarch$kern-$dom0arch-xl$qemuu_suffix-win7-amd64 \
>              test-win xl $xenarch $dom0arch $qemuu_runvar \
>              win_image=win7-x64.iso \
> -            win_acpi_shutdown=true \
>              all_hostflags=$most_hostflags,hvm
>  }
>  
> -- 
> 2.1.4

Re: ACPI shutdown unreliable with win7?

[Ian Campbell]

On Fri, 2015-05-29 at 14:04 +0100, Jan Beulich wrote:
> >>> On 29.05.15 at 14:54, <ian.campbell@citrix.com> wrote:
> > On Fri, 2015-05-22 at 10:08 +0100, Ian Campbell wrote:
> >> If win7 doesn't shutdown given a power button request I'd be more
> >> inclined to remove the setting in osstest for those flights and let
> >> guest-stop go back to being never pass than to start making such changes
> >> to the VM config which I think would probably break the preceding
> >> suspend and migration tests (which aren't completely reliable, but are
> >> far more so than this shutdown one).
> > 
> > Does anyone have any ideas here or shall I propose:
> 
> Unless we have a way to make an adjustment inside the guest for the
> power button to gain "shutdown" meaning, I think there's no alternative
> to the change below.

The strange this is that it does work _sometimes_, either by complete
coincidence or because there is something non-deterministic about how
Win7 reacts to this ACPI event.

Does anyone know which setting we would want to change? In particular it
would be good if I knew what to look for to check the current status...

> 
> Jan
> 
> > -----8>--------------
> > 
> > From 2d1b814e65676c3cf56ce2e569491953607f53f7 Mon Sep 17 00:00:00 2001
> > From: Ian Campbell <ian.campbell@citrix.com>
> > Date: Fri, 29 May 2015 13:51:33 +0100
> > Subject: [PATCH] Turn off acpi_shutdown for windows 7
> > 
> > As described in <1432284841.10746.136.camel@citrix.com> /
> > http://lists.xen.org/archives/html/xen-devel/2015-05/msg03016.html 
> > Windows 7 does not appear to reliably actually shutdown when asked to
> > via the ACPI power button.
> > 
> > Once this patch is applied some force pushes will likely be needed in
> > order for this to not appear as a regression.
> > 
> > Signed-off-by: Ian Campbell <ian.campbell@citrix.com>
> > Cc: Andrew Cooper <Andrew.Cooper3@citrix.com>
> > Cc: Paul Durrant <paul.durrant@citrix.com>
> > Cc: Ian Jackson <Ian.Jackson@eu.citrix.com>,
> > Cc: Jan Beulich <JBeulich@suse.com>
> > ---
> > Alternatively could make it an allowed/non-blocking failure?
> > ---
> >  make-flight | 1 -
> >  1 file changed, 1 deletion(-)
> > 
> > diff --git a/make-flight b/make-flight
> > index 8a1fceb..5120891 100755
> > --- a/make-flight
> > +++ b/make-flight
> > @@ -206,7 +206,6 @@ do_hvm_win7_x64_tests () {
> >    job_create_test test-$xenarch$kern-$dom0arch-xl$qemuu_suffix-win7-amd64 \
> >              test-win xl $xenarch $dom0arch $qemuu_runvar \
> >              win_image=win7-x64.iso \
> > -            win_acpi_shutdown=true \
> >              all_hostflags=$most_hostflags,hvm
> >  }
> >  
> > -- 
> > 2.1.4
> 
> 
> 

Re: ACPI shutdown unreliable with win7?

[Paul Durrant]

> -----Original Message-----
> From: Ian Campbell [mailto:ian.campbell@citrix.com]
> Sent: 29 May 2015 14:14
> To: Jan Beulich
> Cc: Andrew Cooper; Paul Durrant; Ian Jackson; xen-devel
> Subject: Re: [Xen-devel] ACPI shutdown unreliable with win7?
> 
> On Fri, 2015-05-29 at 14:04 +0100, Jan Beulich wrote:
> > >>> On 29.05.15 at 14:54, <ian.campbell@citrix.com> wrote:
> > > On Fri, 2015-05-22 at 10:08 +0100, Ian Campbell wrote:
> > >> If win7 doesn't shutdown given a power button request I'd be more
> > >> inclined to remove the setting in osstest for those flights and let
> > >> guest-stop go back to being never pass than to start making such
> changes
> > >> to the VM config which I think would probably break the preceding
> > >> suspend and migration tests (which aren't completely reliable, but are
> > >> far more so than this shutdown one).
> > >
> > > Does anyone have any ideas here or shall I propose:
> >
> > Unless we have a way to make an adjustment inside the guest for the
> > power button to gain "shutdown" meaning, I think there's no alternative
> > to the change below.
> 
> The strange this is that it does work _sometimes_, either by complete
> coincidence or because there is something non-deterministic about how
> Win7 reacts to this ACPI event.
> 

How long is the test waiting for the OS to shut down though? If you get unlucky, Windows will wander off to Windows Update, download a bazillion patches and take more than an hour to shut down. If you’re lucky, it may shut down in 10 seconds or less.

  Paul

> Does anyone know which setting we would want to change? In particular it
> would be good if I knew what to look for to check the current status...
> 
> >
> > Jan
> >
> > > -----8>--------------
> > >
> > > From 2d1b814e65676c3cf56ce2e569491953607f53f7 Mon Sep 17 00:00:00
> 2001
> > > From: Ian Campbell <ian.campbell@citrix.com>
> > > Date: Fri, 29 May 2015 13:51:33 +0100
> > > Subject: [PATCH] Turn off acpi_shutdown for windows 7
> > >
> > > As described in <1432284841.10746.136.camel@citrix.com> /
> > > http://lists.xen.org/archives/html/xen-devel/2015-05/msg03016.html
> > > Windows 7 does not appear to reliably actually shutdown when asked to
> > > via the ACPI power button.
> > >
> > > Once this patch is applied some force pushes will likely be needed in
> > > order for this to not appear as a regression.
> > >
> > > Signed-off-by: Ian Campbell <ian.campbell@citrix.com>
> > > Cc: Andrew Cooper <Andrew.Cooper3@citrix.com>
> > > Cc: Paul Durrant <paul.durrant@citrix.com>
> > > Cc: Ian Jackson <Ian.Jackson@eu.citrix.com>,
> > > Cc: Jan Beulich <JBeulich@suse.com>
> > > ---
> > > Alternatively could make it an allowed/non-blocking failure?
> > > ---
> > >  make-flight | 1 -
> > >  1 file changed, 1 deletion(-)
> > >
> > > diff --git a/make-flight b/make-flight
> > > index 8a1fceb..5120891 100755
> > > --- a/make-flight
> > > +++ b/make-flight
> > > @@ -206,7 +206,6 @@ do_hvm_win7_x64_tests () {
> > >    job_create_test test-$xenarch$kern-$dom0arch-xl$qemuu_suffix-
> win7-amd64 \
> > >              test-win xl $xenarch $dom0arch $qemuu_runvar \
> > >              win_image=win7-x64.iso \
> > > -            win_acpi_shutdown=true \
> > >              all_hostflags=$most_hostflags,hvm
> > >  }
> > >
> > > --
> > > 2.1.4
> >
> >
> >
> 

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: ACPI shutdown unreliable with win7?

[Ian Campbell]

On Fri, 2015-05-29 at 15:25 +0100, Paul Durrant wrote:
> > -----Original Message-----
> > From: Ian Campbell [mailto:ian.campbell@citrix.com]
> > Sent: 29 May 2015 14:14
> > To: Jan Beulich
> > Cc: Andrew Cooper; Paul Durrant; Ian Jackson; xen-devel
> > Subject: Re: [Xen-devel] ACPI shutdown unreliable with win7?
> > 
> > On Fri, 2015-05-29 at 14:04 +0100, Jan Beulich wrote:
> > > >>> On 29.05.15 at 14:54, <ian.campbell@citrix.com> wrote:
> > > > On Fri, 2015-05-22 at 10:08 +0100, Ian Campbell wrote:
> > > >> If win7 doesn't shutdown given a power button request I'd be more
> > > >> inclined to remove the setting in osstest for those flights and let
> > > >> guest-stop go back to being never pass than to start making such
> > changes
> > > >> to the VM config which I think would probably break the preceding
> > > >> suspend and migration tests (which aren't completely reliable, but are
> > > >> far more so than this shutdown one).
> > > >
> > > > Does anyone have any ideas here or shall I propose:
> > >
> > > Unless we have a way to make an adjustment inside the guest for the
> > > power button to gain "shutdown" meaning, I think there's no alternative
> > > to the change below.
> > 
> > The strange this is that it does work _sometimes_, either by complete
> > coincidence or because there is something non-deterministic about how
> > Win7 reacts to this ACPI event.
> > 
> 
> How long is the test waiting for the OS to shut down though? If you
> get unlucky, Windows will wander off to Windows Update, download a
> bazillion patches and take more than an hour to shut down. If you’re
> lucky, it may shut down in 10 seconds or less.

The screenshot in e.g.
http://logs.test-lab.xenproject.org/osstest/logs/56929/test-amd64-amd64-xl-qemut-win7-amd64/info.html
seems to show that the guest isn't even trying to shut down, it's just
sat there at the desktop:

http://logs.test-lab.xenproject.org/osstest/logs/56929/test-amd64-amd64-xl-qemut-win7-amd64/win.guest.osstest--vnc.jpeg

I think if it had hit WU there would be activity on the screen?

FWIW We appear to wait 200s, if we were seeing failures due to windows
update then I'd be inclined to extend that, but I think right now that
would be premature, unless WU happens with no status on the screen.

Ian


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: ACPI shutdown unreliable with win7?

[Paul Durrant]

> -----Original Message-----
> From: Ian Campbell [mailto:ian.campbell@citrix.com]
> Sent: 29 May 2015 15:36
> To: Paul Durrant
> Cc: Jan Beulich; Andrew Cooper; Ian Jackson; xen-devel
> Subject: Re: [Xen-devel] ACPI shutdown unreliable with win7?
> 
> On Fri, 2015-05-29 at 15:25 +0100, Paul Durrant wrote:
> > > -----Original Message-----
> > > From: Ian Campbell [mailto:ian.campbell@citrix.com]
> > > Sent: 29 May 2015 14:14
> > > To: Jan Beulich
> > > Cc: Andrew Cooper; Paul Durrant; Ian Jackson; xen-devel
> > > Subject: Re: [Xen-devel] ACPI shutdown unreliable with win7?
> > >
> > > On Fri, 2015-05-29 at 14:04 +0100, Jan Beulich wrote:
> > > > >>> On 29.05.15 at 14:54, <ian.campbell@citrix.com> wrote:
> > > > > On Fri, 2015-05-22 at 10:08 +0100, Ian Campbell wrote:
> > > > >> If win7 doesn't shutdown given a power button request I'd be more
> > > > >> inclined to remove the setting in osstest for those flights and let
> > > > >> guest-stop go back to being never pass than to start making such
> > > changes
> > > > >> to the VM config which I think would probably break the preceding
> > > > >> suspend and migration tests (which aren't completely reliable, but
> are
> > > > >> far more so than this shutdown one).
> > > > >
> > > > > Does anyone have any ideas here or shall I propose:
> > > >
> > > > Unless we have a way to make an adjustment inside the guest for the
> > > > power button to gain "shutdown" meaning, I think there's no
> alternative
> > > > to the change below.
> > >
> > > The strange this is that it does work _sometimes_, either by complete
> > > coincidence or because there is something non-deterministic about how
> > > Win7 reacts to this ACPI event.
> > >
> >
> > How long is the test waiting for the OS to shut down though? If you
> > get unlucky, Windows will wander off to Windows Update, download a
> > bazillion patches and take more than an hour to shut down. If you’re
> > lucky, it may shut down in 10 seconds or less.
> 
> The screenshot in e.g.
> http://logs.test-lab.xenproject.org/osstest/logs/56929/test-amd64-amd64-
> xl-qemut-win7-amd64/info.html
> seems to show that the guest isn't even trying to shut down, it's just
> sat there at the desktop:
> 
> http://logs.test-lab.xenproject.org/osstest/logs/56929/test-amd64-amd64-
> xl-qemut-win7-amd64/win.guest.osstest--vnc.jpeg
> 
> I think if it had hit WU there would be activity on the screen?
> 
> FWIW We appear to wait 200s, if we were seeing failures due to windows
> update then I'd be inclined to extend that, but I think right now that
> would be premature, unless WU happens with no status on the screen.
> 

No, you'd see something. Perhaps our ACPI lid/power switch code is just buggy then?

  Paul

> Ian

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: ACPI shutdown unreliable with win7?

[Ian Campbell]

On Fri, 2015-05-29 at 16:06 +0100, Paul Durrant wrote:
> > FWIW We appear to wait 200s, if we were seeing failures due to windows
> > update then I'd be inclined to extend that, but I think right now that
> > would be premature, unless WU happens with no status on the screen.
> > 
> 
> No, you'd see something. Perhaps our ACPI lid/power switch code is just buggy then?

It seems to work reliably for the WinXP tests, FWIW...

Ian.

Re: ACPI shutdown unreliable with win7?

[Ross Philipson]

On 05/29/2015 11:11 AM, Ian Campbell wrote:
> On Fri, 2015-05-29 at 16:06 +0100, Paul Durrant wrote:
>>> FWIW We appear to wait 200s, if we were seeing failures due to windows
>>> update then I'd be inclined to extend that, but I think right now that
>>> would be premature, unless WU happens with no status on the screen.
>>>
>>
>> No, you'd see something. Perhaps our ACPI lid/power switch code is just buggy then?
>
> It seems to work reliably for the WinXP tests, FWIW...
>
> Ian.
>

One thing I find confusing is that the firmware code does not even have 
a power button device (PNP0C0C) or the fixed feature power button that 
is enabled in the FADT (flag == PWR_BUTTON bit 4). So I don't see how 
the shutdown is purely an ACPI function. Is there something else to the 
story? Is it relying on PV tools to do it?

Ross

>
>
> _______________________________________________
> Xen-devel mailing list
> Xen-devel@lists.xen.org
> http://lists.xen.org/xen-devel
>


-- 
Ross Philipson

Re: ACPI shutdown unreliable with win7?

[Paul Durrant]

> -----Original Message-----
> From: Ross Philipson [mailto:ross.philipson@gmail.com]
> Sent: 29 May 2015 16:35
> To: Ian Campbell; Paul Durrant
> Cc: Andrew Cooper; xen-devel; Jan Beulich; Ian Jackson
> Subject: Re: [Xen-devel] ACPI shutdown unreliable with win7?
> 
> On 05/29/2015 11:11 AM, Ian Campbell wrote:
> > On Fri, 2015-05-29 at 16:06 +0100, Paul Durrant wrote:
> >>> FWIW We appear to wait 200s, if we were seeing failures due to
> windows
> >>> update then I'd be inclined to extend that, but I think right now that
> >>> would be premature, unless WU happens with no status on the screen.
> >>>
> >>
> >> No, you'd see something. Perhaps our ACPI lid/power switch code is just
> buggy then?
> >
> > It seems to work reliably for the WinXP tests, FWIW...
> >
> > Ian.
> >
> 
> One thing I find confusing is that the firmware code does not even have
> a power button device (PNP0C0C) or the fixed feature power button that
> is enabled in the FADT (flag == PWR_BUTTON bit 4). So I don't see how
> the shutdown is purely an ACPI function. Is there something else to the
> story? Is it relying on PV tools to do it?
> 

Xen (and QEMU seemingly) implement the 'Fixed Power Button' (section 4.8.2.2.1.1 on my spec) and this requires the PWR_BUTTON flag to be clear (according table 4-13). It also does not require a power button device to be implemented (which is presumably why this way of doing it was chosen).

  Paul

> Ross
> 
> >
> >
> > _______________________________________________
> > Xen-devel mailing list
> > Xen-devel@lists.xen.org
> > http://lists.xen.org/xen-devel
> >
> 
> 
> --
> Ross Philipson

Re: ACPI shutdown unreliable with win7?

[Ross Philipson]

On 05/29/2015 12:00 PM, Paul Durrant wrote:
>> -----Original Message-----
>> From: Ross Philipson [mailto:ross.philipson@gmail.com]
>> Sent: 29 May 2015 16:35
>> To: Ian Campbell; Paul Durrant
>> Cc: Andrew Cooper; xen-devel; Jan Beulich; Ian Jackson
>> Subject: Re: [Xen-devel] ACPI shutdown unreliable with win7?
>>
>> On 05/29/2015 11:11 AM, Ian Campbell wrote:
>>> On Fri, 2015-05-29 at 16:06 +0100, Paul Durrant wrote:
>>>>> FWIW We appear to wait 200s, if we were seeing failures due to
>> windows
>>>>> update then I'd be inclined to extend that, but I think right now that
>>>>> would be premature, unless WU happens with no status on the screen.
>>>>>
>>>>
>>>> No, you'd see something. Perhaps our ACPI lid/power switch code is just
>> buggy then?
>>>
>>> It seems to work reliably for the WinXP tests, FWIW...
>>>
>>> Ian.
>>>
>>
>> One thing I find confusing is that the firmware code does not even have
>> a power button device (PNP0C0C) or the fixed feature power button that
>> is enabled in the FADT (flag == PWR_BUTTON bit 4). So I don't see how
>> the shutdown is purely an ACPI function. Is there something else to the
>> story? Is it relying on PV tools to do it?
>>
>
> Xen (and QEMU seemingly) implement the 'Fixed Power Button' (section 4.8.2.2.1.1 on my spec) and this requires the PWR_BUTTON flag to be clear (according table 4-13). It also does not require a power button device to be implemented (which is presumably why this way of doing it was chosen).

Ah got it. I read the spec backwards - that the bit was set not clear 
for the fixed feature power button :)


>
>    Paul
>
>> Ross
>>
>>>
>>>
>>> _______________________________________________
>>> Xen-devel mailing list
>>> Xen-devel@lists.xen.org
>>> http://lists.xen.org/xen-devel
>>>
>>
>>
>> --
>> Ross Philipson


-- 
Ross Philipson

Re: ACPI shutdown unreliable with win7?

[Ross Philipson]

On 05/29/2015 12:00 PM, Paul Durrant wrote:
>> -----Original Message-----
>> From: Ross Philipson [mailto:ross.philipson@gmail.com]
>> Sent: 29 May 2015 16:35
>> To: Ian Campbell; Paul Durrant
>> Cc: Andrew Cooper; xen-devel; Jan Beulich; Ian Jackson
>> Subject: Re: [Xen-devel] ACPI shutdown unreliable with win7?
>>
>> On 05/29/2015 11:11 AM, Ian Campbell wrote:
>>> On Fri, 2015-05-29 at 16:06 +0100, Paul Durrant wrote:
>>>>> FWIW We appear to wait 200s, if we were seeing failures due to
>> windows
>>>>> update then I'd be inclined to extend that, but I think right now that
>>>>> would be premature, unless WU happens with no status on the screen.
>>>>>
>>>>
>>>> No, you'd see something. Perhaps our ACPI lid/power switch code is just
>> buggy then?
>>>
>>> It seems to work reliably for the WinXP tests, FWIW...
>>>
>>> Ian.
>>>
>>
>> One thing I find confusing is that the firmware code does not even have
>> a power button device (PNP0C0C) or the fixed feature power button that
>> is enabled in the FADT (flag == PWR_BUTTON bit 4). So I don't see how
>> the shutdown is purely an ACPI function. Is there something else to the
>> story? Is it relying on PV tools to do it?
>>
>
> Xen (and QEMU seemingly) implement the 'Fixed Power Button' (section 4.8.2.2.1.1 on my spec) and this requires the PWR_BUTTON flag to be clear (according table 4-13). It also does not require a power button device to be implemented (which is presumably why this way of doing it was chosen).

"which is presumably why this way of doing it was chosen"

Yea it is simpler plus the power button device would then need a bunch 
of GPE plumbing to get events.

Ross
>
>    Paul
>
>> Ross
>>
>>>
>>>
>>> _______________________________________________
>>> Xen-devel mailing list
>>> Xen-devel@lists.xen.org
>>> http://lists.xen.org/xen-devel
>>>
>>
>>
>> --
>> Ross Philipson


-- 
Ross Philipson

Re: ACPI shutdown unreliable with win7?

[Paul Durrant]

> -----Original Message-----
> From: Paul Durrant
> Sent: 29 May 2015 16:07
> To: Ian Campbell
> Cc: Jan Beulich; Andrew Cooper; Ian Jackson; xen-devel
> Subject: RE: [Xen-devel] ACPI shutdown unreliable with win7?
> 
> > -----Original Message-----
> > From: Ian Campbell [mailto:ian.campbell@citrix.com]
> > Sent: 29 May 2015 15:36
> > To: Paul Durrant
> > Cc: Jan Beulich; Andrew Cooper; Ian Jackson; xen-devel
> > Subject: Re: [Xen-devel] ACPI shutdown unreliable with win7?
> >
> > On Fri, 2015-05-29 at 15:25 +0100, Paul Durrant wrote:
> > > > -----Original Message-----
> > > > From: Ian Campbell [mailto:ian.campbell@citrix.com]
> > > > Sent: 29 May 2015 14:14
> > > > To: Jan Beulich
> > > > Cc: Andrew Cooper; Paul Durrant; Ian Jackson; xen-devel
> > > > Subject: Re: [Xen-devel] ACPI shutdown unreliable with win7?
> > > >
> > > > On Fri, 2015-05-29 at 14:04 +0100, Jan Beulich wrote:
> > > > > >>> On 29.05.15 at 14:54, <ian.campbell@citrix.com> wrote:
> > > > > > On Fri, 2015-05-22 at 10:08 +0100, Ian Campbell wrote:
> > > > > >> If win7 doesn't shutdown given a power button request I'd be
> more
> > > > > >> inclined to remove the setting in osstest for those flights and let
> > > > > >> guest-stop go back to being never pass than to start making such
> > > > changes
> > > > > >> to the VM config which I think would probably break the preceding
> > > > > >> suspend and migration tests (which aren't completely reliable, but
> > are
> > > > > >> far more so than this shutdown one).
> > > > > >
> > > > > > Does anyone have any ideas here or shall I propose:
> > > > >
> > > > > Unless we have a way to make an adjustment inside the guest for the
> > > > > power button to gain "shutdown" meaning, I think there's no
> > alternative
> > > > > to the change below.
> > > >
> > > > The strange this is that it does work _sometimes_, either by complete
> > > > coincidence or because there is something non-deterministic about
> how
> > > > Win7 reacts to this ACPI event.
> > > >
> > >
> > > How long is the test waiting for the OS to shut down though? If you
> > > get unlucky, Windows will wander off to Windows Update, download a
> > > bazillion patches and take more than an hour to shut down. If you’re
> > > lucky, it may shut down in 10 seconds or less.
> >
> > The screenshot in e.g.
> > http://logs.test-lab.xenproject.org/osstest/logs/56929/test-amd64-
> amd64-
> > xl-qemut-win7-amd64/info.html
> > seems to show that the guest isn't even trying to shut down, it's just
> > sat there at the desktop:
> >
> > http://logs.test-lab.xenproject.org/osstest/logs/56929/test-amd64-
> amd64-
> > xl-qemut-win7-amd64/win.guest.osstest--vnc.jpeg
> >
> > I think if it had hit WU there would be activity on the screen?
> >
> > FWIW We appear to wait 200s, if we were seeing failures due to windows
> > update then I'd be inclined to extend that, but I think right now that
> > would be premature, unless WU happens with no status on the screen.
> >
> 
> No, you'd see something. Perhaps our ACPI lid/power switch code is just
> buggy then?
> 

I already said this to Ian, but for the record... From my reading of the ACPI spec (v 5.0, page 23 glossary entry) the SCI is an active low, shareable, level-sensitive interrupt, but our code (pmtimer.c:pmt_update_sci) seems to treat it as active high. I'll have a look at the QEMU SCI code as reference, but maybe it is just broken.

  Paul

>   Paul
> 
> > Ian

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: ACPI shutdown unreliable with win7?

[Jan Beulich]

>>> On 29.05.15 at 17:34, <Paul.Durrant@citrix.com> wrote:
>> From: Paul Durrant
>> Sent: 29 May 2015 16:07
>> No, you'd see something. Perhaps our ACPI lid/power switch code is just
>> buggy then?
> 
> I already said this to Ian, but for the record... From my reading of the 
> ACPI spec (v 5.0, page 23 glossary entry) the SCI is an active low, 
> shareable, level-sensitive interrupt, but our code (pmtimer.c:pmt_update_sci) 
> seems to treat it as active high. I'll have a look at the QEMU SCI code as 
> reference, but maybe it is just broken.

While ACPI may prefer it to be that way, I don't think this is a
requirement (both physical systems I checked just now have it
level-high), and in any case subject to an interrupt source
override in MADT (which hvmloader doesn't produce afaict).

Jan

Re: ACPI shutdown unreliable with win7?

[Paul Durrant]

> -----Original Message-----
> From: Paul Durrant
> Sent: 29 May 2015 16:35
> To: Ian Campbell
> Cc: 'Jan Beulich'; Andrew Cooper; Ian Jackson; 'xen-devel'
> Subject: RE: [Xen-devel] ACPI shutdown unreliable with win7?
> 
> > -----Original Message-----
> > From: Paul Durrant
> > Sent: 29 May 2015 16:07
> > To: Ian Campbell
> > Cc: Jan Beulich; Andrew Cooper; Ian Jackson; xen-devel
> > Subject: RE: [Xen-devel] ACPI shutdown unreliable with win7?
> >
> > > -----Original Message-----
> > > From: Ian Campbell [mailto:ian.campbell@citrix.com]
> > > Sent: 29 May 2015 15:36
> > > To: Paul Durrant
> > > Cc: Jan Beulich; Andrew Cooper; Ian Jackson; xen-devel
> > > Subject: Re: [Xen-devel] ACPI shutdown unreliable with win7?
> > >
> > > On Fri, 2015-05-29 at 15:25 +0100, Paul Durrant wrote:
> > > > > -----Original Message-----
> > > > > From: Ian Campbell [mailto:ian.campbell@citrix.com]
> > > > > Sent: 29 May 2015 14:14
> > > > > To: Jan Beulich
> > > > > Cc: Andrew Cooper; Paul Durrant; Ian Jackson; xen-devel
> > > > > Subject: Re: [Xen-devel] ACPI shutdown unreliable with win7?
> > > > >
> > > > > On Fri, 2015-05-29 at 14:04 +0100, Jan Beulich wrote:
> > > > > > >>> On 29.05.15 at 14:54, <ian.campbell@citrix.com> wrote:
> > > > > > > On Fri, 2015-05-22 at 10:08 +0100, Ian Campbell wrote:
> > > > > > >> If win7 doesn't shutdown given a power button request I'd be
> > more
> > > > > > >> inclined to remove the setting in osstest for those flights and let
> > > > > > >> guest-stop go back to being never pass than to start making such
> > > > > changes
> > > > > > >> to the VM config which I think would probably break the
> preceding
> > > > > > >> suspend and migration tests (which aren't completely reliable,
> but
> > > are
> > > > > > >> far more so than this shutdown one).
> > > > > > >
> > > > > > > Does anyone have any ideas here or shall I propose:
> > > > > >
> > > > > > Unless we have a way to make an adjustment inside the guest for
> the
> > > > > > power button to gain "shutdown" meaning, I think there's no
> > > alternative
> > > > > > to the change below.
> > > > >
> > > > > The strange this is that it does work _sometimes_, either by complete
> > > > > coincidence or because there is something non-deterministic about
> > how
> > > > > Win7 reacts to this ACPI event.
> > > > >
> > > >
> > > > How long is the test waiting for the OS to shut down though? If you
> > > > get unlucky, Windows will wander off to Windows Update, download a
> > > > bazillion patches and take more than an hour to shut down. If you’re
> > > > lucky, it may shut down in 10 seconds or less.
> > >
> > > The screenshot in e.g.
> > > http://logs.test-lab.xenproject.org/osstest/logs/56929/test-amd64-
> > amd64-
> > > xl-qemut-win7-amd64/info.html
> > > seems to show that the guest isn't even trying to shut down, it's just
> > > sat there at the desktop:
> > >
> > > http://logs.test-lab.xenproject.org/osstest/logs/56929/test-amd64-
> > amd64-
> > > xl-qemut-win7-amd64/win.guest.osstest--vnc.jpeg
> > >
> > > I think if it had hit WU there would be activity on the screen?
> > >
> > > FWIW We appear to wait 200s, if we were seeing failures due to windows
> > > update then I'd be inclined to extend that, but I think right now that
> > > would be premature, unless WU happens with no status on the screen.
> > >
> >
> > No, you'd see something. Perhaps our ACPI lid/power switch code is just
> > buggy then?
> >
> 
> I already said this to Ian, but for the record... From my reading of the ACPI
> spec (v 5.0, page 23 glossary entry) the SCI is an active low, shareable, level-
> sensitive interrupt, but our code (pmtimer.c:pmt_update_sci) seems to treat
> it as active high. I'll have a look at the QEMU SCI code as reference, but
> maybe it is just broken.
> 

Upstream QEMU does seem to have it the same way up as our pmtimer code (hw/acpi/core.c:acpi_update_sci) so at least we're in good company :-/

  Paul

>   Paul
> 
> >   Paul
> >
> > > Ian

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: ACPI shutdown unreliable with win7?

[Jan Beulich]

>>> On 29.05.15 at 16:25, <Paul.Durrant@citrix.com> wrote:
>> From: Ian Campbell [mailto:ian.campbell@citrix.com]
> How long is the test waiting for the OS to shut down though? If you get 
> unlucky, Windows will wander off to Windows Update, download a bazillion 
> patches and take more than an hour to shut down. If you’re lucky, it may shut 
> down in 10 seconds or less.

But that would require the download of all those patches to complete
by the time shutdown is requested, and I think would also not leave
Windows at the login screen (i.e. one would be able to tell from the
screen shots taken).

Jan

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: ACPI shutdown unreliable with win7?

[Andrew Cooper]

On 29/05/15 14:04, Jan Beulich wrote:
>>>> On 29.05.15 at 14:54, <ian.campbell@citrix.com> wrote:
>> On Fri, 2015-05-22 at 10:08 +0100, Ian Campbell wrote:
>>> If win7 doesn't shutdown given a power button request I'd be more
>>> inclined to remove the setting in osstest for those flights and let
>>> guest-stop go back to being never pass than to start making such changes
>>> to the VM config which I think would probably break the preceding
>>> suspend and migration tests (which aren't completely reliable, but are
>>> far more so than this shutdown one).
>> Does anyone have any ideas here or shall I propose:
> Unless we have a way to make an adjustment inside the guest for the
> power button to gain "shutdown" meaning, I think there's no alternative
> to the change below.

You can avoid advertising S3/S4 in the ACPI tables, which iirc causes
the same alteration to happen.

Hvmloader uses the platform/acpi_s{3,4} booleans to control whether the
relevant SSDTs are exposed.

~Andrew

Re: ACPI shutdown unreliable with win7?

[Ian Campbell]

On Fri, 2015-05-29 at 14:14 +0100, Andrew Cooper wrote:
> On 29/05/15 14:04, Jan Beulich wrote:
> >>>> On 29.05.15 at 14:54, <ian.campbell@citrix.com> wrote:
> >> On Fri, 2015-05-22 at 10:08 +0100, Ian Campbell wrote:
> >>> If win7 doesn't shutdown given a power button request I'd be more
> >>> inclined to remove the setting in osstest for those flights and let
> >>> guest-stop go back to being never pass than to start making such changes
> >>> to the VM config which I think would probably break the preceding
> >>> suspend and migration tests (which aren't completely reliable, but are
> >>> far more so than this shutdown one).
> >> Does anyone have any ideas here or shall I propose:
> > Unless we have a way to make an adjustment inside the guest for the
> > power button to gain "shutdown" meaning, I think there's no alternative
> > to the change below.
> 
> You can avoid advertising S3/S4 in the ACPI tables, which iirc causes
> the same alteration to happen.
> 
> Hvmloader uses the platform/acpi_s{3,4} booleans to control whether the
> relevant SSDTs are exposed.

As I said in <1432285699.10746.147.camel@citrix.com> and again in reply
to Jan just now, this test does sometimes pass.

So whether or not we are exposing the correct set of tables to the guest
there is something else non-deterministic going on here I think.

I'd also be concerned about the impact on other tests (in particular the
SRM ones) on disabling s3 and/or s4.

Ian.

Re: ACPI shutdown unreliable with win7?

[Jan Beulich]

>>> On 29.05.15 at 15:14, <andrew.cooper3@citrix.com> wrote:
> On 29/05/15 14:04, Jan Beulich wrote:
>>>>> On 29.05.15 at 14:54, <ian.campbell@citrix.com> wrote:
>>> On Fri, 2015-05-22 at 10:08 +0100, Ian Campbell wrote:
>>>> If win7 doesn't shutdown given a power button request I'd be more
>>>> inclined to remove the setting in osstest for those flights and let
>>>> guest-stop go back to being never pass than to start making such changes
>>>> to the VM config which I think would probably break the preceding
>>>> suspend and migration tests (which aren't completely reliable, but are
>>>> far more so than this shutdown one).
>>> Does anyone have any ideas here or shall I propose:
>> Unless we have a way to make an adjustment inside the guest for the
>> power button to gain "shutdown" meaning, I think there's no alternative
>> to the change below.
> 
> You can avoid advertising S3/S4 in the ACPI tables, which iirc causes
> the same alteration to happen.
> 
> Hvmloader uses the platform/acpi_s{3,4} booleans to control whether the
> relevant SSDTs are exposed.

Which libxl even has settings for. That would perhaps be a better first
try than disabling ACPI shutdown.

Jan

Re: ACPI shutdown unreliable with win7?

[Ian Campbell]

On Fri, 2015-05-29 at 14:28 +0100, Jan Beulich wrote:
> >>> On 29.05.15 at 15:14, <andrew.cooper3@citrix.com> wrote:
> > On 29/05/15 14:04, Jan Beulich wrote:
> >>>>> On 29.05.15 at 14:54, <ian.campbell@citrix.com> wrote:
> >>> On Fri, 2015-05-22 at 10:08 +0100, Ian Campbell wrote:
> >>>> If win7 doesn't shutdown given a power button request I'd be more
> >>>> inclined to remove the setting in osstest for those flights and let
> >>>> guest-stop go back to being never pass than to start making such changes
> >>>> to the VM config which I think would probably break the preceding
> >>>> suspend and migration tests (which aren't completely reliable, but are
> >>>> far more so than this shutdown one).
> >>> Does anyone have any ideas here or shall I propose:
> >> Unless we have a way to make an adjustment inside the guest for the
> >> power button to gain "shutdown" meaning, I think there's no alternative
> >> to the change below.
> > 
> > You can avoid advertising S3/S4 in the ACPI tables, which iirc causes
> > the same alteration to happen.
> > 
> > Hvmloader uses the platform/acpi_s{3,4} booleans to control whether the
> > relevant SSDTs are exposed.
> 
> Which libxl even has settings for. That would perhaps be a better first
> try than disabling ACPI shutdown.

I've mentioned it at least twice now but nobody seems to think it is of
note that even with the current settings it does the right thing about 1
time in 10? Is Win7 really expected to behave so randomly here? 

Also, when the test fails the guest is also not hibernating either.

Plus I've queried the impact of change the ACPI s3/s4 settings on the
save/restore/migrate tests in the flight more than once and nobody has
responded to that either.

Ian

Re: ACPI shutdown unreliable with win7?

[Andrew Cooper]

On 29/05/15 15:00, Ian Campbell wrote:
> On Fri, 2015-05-29 at 14:28 +0100, Jan Beulich wrote:
>>>>> On 29.05.15 at 15:14, <andrew.cooper3@citrix.com> wrote:
>>> On 29/05/15 14:04, Jan Beulich wrote:
>>>>>>> On 29.05.15 at 14:54, <ian.campbell@citrix.com> wrote:
>>>>> On Fri, 2015-05-22 at 10:08 +0100, Ian Campbell wrote:
>>>>>> If win7 doesn't shutdown given a power button request I'd be more
>>>>>> inclined to remove the setting in osstest for those flights and let
>>>>>> guest-stop go back to being never pass than to start making such changes
>>>>>> to the VM config which I think would probably break the preceding
>>>>>> suspend and migration tests (which aren't completely reliable, but are
>>>>>> far more so than this shutdown one).
>>>>> Does anyone have any ideas here or shall I propose:
>>>> Unless we have a way to make an adjustment inside the guest for the
>>>> power button to gain "shutdown" meaning, I think there's no alternative
>>>> to the change below.
>>> You can avoid advertising S3/S4 in the ACPI tables, which iirc causes
>>> the same alteration to happen.
>>>
>>> Hvmloader uses the platform/acpi_s{3,4} booleans to control whether the
>>> relevant SSDTs are exposed.
>> Which libxl even has settings for. That would perhaps be a better first
>> try than disabling ACPI shutdown.
> I've mentioned it at least twice now but nobody seems to think it is of
> note that even with the current settings it does the right thing about 1
> time in 10? Is Win7 really expected to behave so randomly here? 

Windows is far from bug free, and also a black box.  I really cannot
answer whether this is intended behaviour or not.

>
> Also, when the test fails the guest is also not hibernating either.
>
> Plus I've queried the impact of change the ACPI s3/s4 settings on the
> save/restore/migrate tests in the flight more than once and nobody has
> responded to that either.

save/restore/migrate necessarily need PV drivers inside the guest, and
installing PV drivers should set the sane defaults inside the guest.

What does OSSTest do with PV drivers?

~Andrew

Re: ACPI shutdown unreliable with win7?

[Ian Campbell]

On Fri, 2015-05-29 at 15:31 +0100, Andrew Cooper wrote:
> On 29/05/15 15:00, Ian Campbell wrote:
> > On Fri, 2015-05-29 at 14:28 +0100, Jan Beulich wrote:
> >>>>> On 29.05.15 at 15:14, <andrew.cooper3@citrix.com> wrote:
> >>> On 29/05/15 14:04, Jan Beulich wrote:
> >>>>>>> On 29.05.15 at 14:54, <ian.campbell@citrix.com> wrote:
> >>>>> On Fri, 2015-05-22 at 10:08 +0100, Ian Campbell wrote:
> >>>>>> If win7 doesn't shutdown given a power button request I'd be more
> >>>>>> inclined to remove the setting in osstest for those flights and let
> >>>>>> guest-stop go back to being never pass than to start making such changes
> >>>>>> to the VM config which I think would probably break the preceding
> >>>>>> suspend and migration tests (which aren't completely reliable, but are
> >>>>>> far more so than this shutdown one).
> >>>>> Does anyone have any ideas here or shall I propose:
> >>>> Unless we have a way to make an adjustment inside the guest for the
> >>>> power button to gain "shutdown" meaning, I think there's no alternative
> >>>> to the change below.
> >>> You can avoid advertising S3/S4 in the ACPI tables, which iirc causes
> >>> the same alteration to happen.
> >>>
> >>> Hvmloader uses the platform/acpi_s{3,4} booleans to control whether the
> >>> relevant SSDTs are exposed.
> >> Which libxl even has settings for. That would perhaps be a better first
> >> try than disabling ACPI shutdown.
> > I've mentioned it at least twice now but nobody seems to think it is of
> > note that even with the current settings it does the right thing about 1
> > time in 10? Is Win7 really expected to behave so randomly here? 
> 
> Windows is far from bug free, and also a black box.  I really cannot
> answer whether this is intended behaviour or not.
> 
> >
> > Also, when the test fails the guest is also not hibernating either.
> >
> > Plus I've queried the impact of change the ACPI s3/s4 settings on the
> > save/restore/migrate tests in the flight more than once and nobody has
> > responded to that either.
> 
> save/restore/migrate necessarily need PV drivers inside the guest, and
> installing PV drivers should set the sane defaults inside the guest.
> 
> What does OSSTest do with PV drivers?

Nothing AFAIK, unless they are baked into the ISOs we take from the
XenRT folks (which I think is not the case).

Yet our s/r/m tests of these guests do pass (mostly), I believe because
they end up leveraging HVM s3 but I might be wrong.

Ian.

Re: ACPI shutdown unreliable with win7?

[Jan Beulich]

>>> On 29.05.15 at 16:00, <ian.campbell@citrix.com> wrote:
> On Fri, 2015-05-29 at 14:28 +0100, Jan Beulich wrote:
>> >>> On 29.05.15 at 15:14, <andrew.cooper3@citrix.com> wrote:
>> > On 29/05/15 14:04, Jan Beulich wrote:
>> >>>>> On 29.05.15 at 14:54, <ian.campbell@citrix.com> wrote:
>> >>> On Fri, 2015-05-22 at 10:08 +0100, Ian Campbell wrote:
>> >>>> If win7 doesn't shutdown given a power button request I'd be more
>> >>>> inclined to remove the setting in osstest for those flights and let
>> >>>> guest-stop go back to being never pass than to start making such changes
>> >>>> to the VM config which I think would probably break the preceding
>> >>>> suspend and migration tests (which aren't completely reliable, but are
>> >>>> far more so than this shutdown one).
>> >>> Does anyone have any ideas here or shall I propose:
>> >> Unless we have a way to make an adjustment inside the guest for the
>> >> power button to gain "shutdown" meaning, I think there's no alternative
>> >> to the change below.
>> > 
>> > You can avoid advertising S3/S4 in the ACPI tables, which iirc causes
>> > the same alteration to happen.
>> > 
>> > Hvmloader uses the platform/acpi_s{3,4} booleans to control whether the
>> > relevant SSDTs are exposed.
>> 
>> Which libxl even has settings for. That would perhaps be a better first
>> try than disabling ACPI shutdown.
> 
> I've mentioned it at least twice now but nobody seems to think it is of
> note that even with the current settings it does the right thing about 1
> time in 10? Is Win7 really expected to behave so randomly here? 

Perhaps not expected, but then I also don't think we install all the
latest bits, but whatever is on the install media? I know I've seen the
default action preselected for the user to be different on different
instances of native hardware Win7 installs (but of course that could
as well have been due to firmware differences, albeit I doubt there's
many moder systems around that don't support both S3 and S4).

> Also, when the test fails the guest is also not hibernating either.

Would we have ways to tell it at least tried to enter e.g. S3?

> Plus I've queried the impact of change the ACPI s3/s4 settings on the
> save/restore/migrate tests in the flight more than once and nobody has
> responded to that either.

I have no idea, and hence didn't respond to that part.

Jan

Re: ACPI shutdown unreliable with win7?

[Ian Campbell]

On Fri, 2015-05-29 at 15:40 +0100, Jan Beulich wrote:
> > I've mentioned it at least twice now but nobody seems to think it is of
> > note that even with the current settings it does the right thing about 1
> > time in 10? Is Win7 really expected to behave so randomly here? 
> 
> Perhaps not expected, but then I also don't think we install all the
> latest bits, but whatever is on the install media? I know I've seen the
> default action preselected for the user to be different on different
> instances of native hardware Win7 installs (but of course that could
> as well have been due to firmware differences, albeit I doubt there's
> many moder systems around that don't support both S3 and S4).

OK, so the gist I'm getting is that it's quite possible that Win7
actually does just roll a d20 and only hibernate on a prime number...

> > Also, when the test fails the guest is also not hibernating either.
> 
> Would we have ways to tell it at least tried to enter e.g. S3?

I'm not sure, but it's not apparent in the screen shot nor in any of the
logs (qemu etc), I think I'd expect to see _something_ there?

> > Plus I've queried the impact of change the ACPI s3/s4 settings on the
> > save/restore/migrate tests in the flight more than once and nobody has
> > responded to that either.
> 
> I have no idea, and hence didn't respond to that part.

OK, thanks.

Ian.

Re: ACPI shutdown unreliable with win7?

[Don Slutz]

On 05/29/15 10:49, Ian Campbell wrote:
> On Fri, 2015-05-29 at 15:40 +0100, Jan Beulich wrote:
>>> I've mentioned it at least twice now but nobody seems to think it is of
>>> note that even with the current settings it does the right thing about 1
>>> time in 10? Is Win7 really expected to behave so randomly here? 
>> Perhaps not expected, but then I also don't think we install all the
>> latest bits, but whatever is on the install media? I know I've seen the
>> default action preselected for the user to be different on different
>> instances of native hardware Win7 installs (but of course that could
>> as well have been due to firmware differences, albeit I doubt there's
>> many moder systems around that don't support both S3 and S4).
> OK, so the gist I'm getting is that it's quite possible that Win7
> actually does just roll a d20 and only hibernate on a prime number...

I would expect that this is a uninitialized variable path in Win7 on the
virtual hardware config (the roll a d20).  Since it looks like you can
do things in the guest, a better thing would be to force the action to
do on the power button press.

I just found a web page


http://www.pcworld.com/article/225833/Tweak_Your_Windows_PCs_Sleep_Mode.html

That says there are BIOS settings that Win7 will also look at.  Could be
the colo needs some changes
also.

   -Don Slutz


>>> Also, when the test fails the guest is also not hibernating either.
>> Would we have ways to tell it at least tried to enter e.g. S3?
> I'm not sure, but it's not apparent in the screen shot nor in any of the
> logs (qemu etc), I think I'd expect to see _something_ there?
>
>>> Plus I've queried the impact of change the ACPI s3/s4 settings on the
>>> save/restore/migrate tests in the flight more than once and nobody has
>>> responded to that either.
>> I have no idea, and hence didn't respond to that part.
> OK, thanks.
>
> Ian.
>
>
> _______________________________________________
> Xen-devel mailing list
> Xen-devel@lists.xen.org
> http://lists.xen.org/xen-devel

Re: ACPI shutdown unreliable with win7?

[Jan Beulich]

>>> On 22.05.15 at 10:54, <ian.campbell@citrix.com> wrote:
>         May 22 01:28:33.149028 (XEN) VCPU information and callbacks for domain 18:
>         May 22 01:28:33.156972 (XEN)     VCPU0: CPU2 [has=F] poll=0 upcall_pend=00 upcall_mask=00 dirty_cpus={2}
>         May 22 01:28:33.165038 (XEN)     cpu_hard_affinity={0-47} cpu_soft_affinity={0-3}
>         May 22 01:28:33.173029 (XEN)     pause_count=0 pause_flags=1
>         May 22 01:28:33.173064 (XEN)     paging assistance: hap, 4 levels
>         May 22 01:28:33.181020 (XEN)     No periodic timer
>         May 22 01:28:33.181052 (XEN)     VCPU1: CPU1 [has=F] poll=0 upcall_pend=00 upcall_mask=00 dirty_cpus={1}
>         May 22 01:28:33.189031 (XEN)     cpu_hard_affinity={0-47} cpu_soft_affinity={0-3}
>         May 22 01:28:33.197069 (XEN)     pause_count=0 pause_flags=1
>         May 22 01:28:33.197105 (XEN)     paging assistance: hap, 4 levels
>         May 22 01:28:33.205013 (XEN)     No periodic timer

At least this last part certainly isn't the same as in e.g. flight 56898,
where pause_flags=0 for both guest vCPU-s.

Jan