Powerdown problem on XEN | ACPI S5

[Atom2 <ariel.atom2@web2web.at>]

[-- Attachment #1: Type: text/plain, Size: 3680 bytes --]

Hi guys,
I have been liasing with Ian Campbell on the xen-user list and Ian 
suggested I should take this to xen-devel.

The issue I am currently facing is as follows:
I seem to be unable to powerdown my system under the XEN hypervisor by 
issuing the command

	shutdown -h now

from my gentoo-hardened 3.9.5 dom0 machine. Instead of turning off 
power, it goes through a BIOS power-on sequence and reboots. There's no 
other domU running at the time of the attempted powerdown.

If I do the same using *exactly the same dom0 kernel* without XEN 
involved (i.e. boot from my gentoo-hardened 3.9.5 kernel only), 
powerdown reliably works as expected and 'shutdown -h now' turns off the 
system's power.

I have tested both 4.2.2 (the gentoo e-build) and 4.3 (downloaded 
directly from xenbits). There's no difference between those two versions 
in terms of reboot versus powerdown.

Upon advise from Ian I have also experimented with the various reboot= 
options on the command line - also with no success.

Adding a serial console (also thanks to Ian's input) I was able to 
examine the messages during shutdown. I have attached those from 4.2.2 
for your reference.


For the latest test using Xen 4.2.2 I made a few changes to the source 
file xen/arch/x86/acpi/power.c to see where the actual problem may be 
hidden. I'm far from being a kernel or XEN programmer, but I am able to 
read and basically understand and (to an extent) modify C code. 
Supported by finding and identifying the messages I had on the serial 
console I decided to add a few additional printk statements after the 
last message that was displayed on the console to see where the system 
probably crashes / the problems could possibly start:

After my change, the relevant code snippet now looks as follows (NOTE: 
The printk messages starting with "After" or "Before" stem from me, the 
first one and the one within the if-construct are both unchanged; the 
initial one was originally always displayed on the serial console as the 
final line):

     printk("Entering ACPI S%d state.\n", state);

     local_irq_save(flags);
     printk("After local_irq_save\n");

     spin_debug_disable();
     printk("After spin_debug_disable\n");

     if ( (error = device_power_down()) )
     {
         printk(XENLOG_ERR "Some devices failed to power down.");
         system_state = SYS_STATE_resume;
         goto done;
     }

     printk("Before ACPI_FLUSH_CPU_CACHE\n");
     ACPI_FLUSH_CPU_CACHE();
     printk("After ACPI_FLUSH_CPU_CACHE\n");

The final few messages of the *new* output *after my amateur mods* on 
the serial console now read as follows:
(XEN) Entering ACPI S5 state.
(XEN) After local_irq_save
(XEN) After spin_debug_disable

There is neither a message reading
(XEN) Some devices failed to power down.
(NOTE: this printk statement however has a XENLOG_ERR before the text - 
so I am not sure whether that should actually appear on the serial 
console at all)

nor one reading
(XEN) Before ACPI_FLUSH_CPU_CACHE

This to me seems to indicate, that the problematic code is somewhere in
between the following lines:
     if ( (error = device_power_down()) )
     {
         printk(XENLOG_ERR "Some devices failed to power down.");
         system_state = SYS_STATE_resume;
         goto done;
     }

I hope that might provide you with some insight which, with your help, 
could be used to make a step forward.

On the other hand I might be completely on the wrong track as I have no 
clue where the actual requested power-down (or as is: reboot) actually 
happens. That was not obvious for me from the code in power.c without 
further knowledge ...

Many thanks in advance for suggestions.

[-- Attachment #2: XEN console --]
[-- Type: text/plain, Size: 9636 bytes --]

 \ \/ /___ _ __   | || |  |___ \  |___ \ 
  \  // _ \ '_ \  | || |_   __) |   __) |
  /  \  __/ | | | |__   _| / __/ _ / __/ 
 /_/\_\___|_| |_|    |_|(_)_____(_)_____|
                                         
(XEN) Xen version 4.2.2 (@herrenhauspark.com) (gcc (Gentoo Hardened 4.6.3 p1.13, pie-0.5.2) 4.6.3) Fri Jul  5 12:09:53 CEST 2013
(XEN) Latest ChangeSet: unavailable
(XEN) Bootloader: GNU GRUB 0.97
(XEN) Command line: console=com1 loglvl=all com1=115200,8n1,0x2d8,10 dom0_mem=512M,max:512M dom0_max_vcpus=1 dom0_vcpus_pin iommu=1
(XEN) Video information:
(XEN)  VGA is text mode 80x25, font 8x16
(XEN)  VBE/DDC methods: V2; EDID transfer time: 1 seconds
(XEN) Disc information:
(XEN)  Found 2 MBR signatures
(XEN)  Found 2 EDD information structures
(XEN) Xen-e820 RAM map:
(XEN)  0000000000000000 - 0000000000099c00 (usable)
(XEN)  0000000000099c00 - 00000000000a0000 (reserved)
(XEN)  00000000000e0000 - 0000000000100000 (reserved)
(XEN)  0000000000100000 - 0000000020000000 (usable)
(XEN)  0000000020000000 - 0000000020200000 (reserved)
(XEN)  0000000020200000 - 0000000040000000 (usable)
(XEN)  0000000040000000 - 0000000040200000 (reserved)
(XEN)  0000000040200000 - 00000000db9f1000 (usable)
(XEN)  00000000db9f1000 - 00000000dc0db000 (reserved)
(XEN)  00000000dc0db000 - 00000000dc1fa000 (ACPI NVS)
(XEN)  00000000dc1fa000 - 00000000dc652000 (reserved)
(XEN)  00000000dc652000 - 00000000dc653000 (usable)
(XEN)  00000000dc653000 - 00000000dc696000 (ACPI NVS)
(XEN)  00000000dc696000 - 00000000dcdba000 (usable)
(XEN)  00000000dcdba000 - 00000000dcff2000 (reserved)
(XEN)  00000000dcff2000 - 00000000dd000000 (usable)
(XEN)  00000000dd800000 - 00000000dfa00000 (reserved)
(XEN)  00000000f8000000 - 00000000fc000000 (reserved)
(XEN)  00000000fec00000 - 00000000fec01000 (reserved)
(XEN)  00000000fed00000 - 00000000fed04000 (reserved)
(XEN)  00000000fed1c000 - 00000000fed20000 (reserved)
(XEN)  00000000fee00000 - 00000000fee01000 (reserved)
(XEN)  00000000ff000000 - 0000000100000000 (reserved)
(XEN)  0000000100000000 - 000000081e600000 (usable)
(XEN) ACPI: RSDP 000F0490, 0024 (r2 ALASKA)
(XEN) ACPI: XSDT DC1EA078, 0074 (r1 ALASKA    A M I  1072009 AMI     10013)
(XEN) ACPI: FACP DC1F4710, 00F4 (r4 ALASKA    A M I  1072009 AMI     10013)
(XEN) ACPI: DSDT DC1EA188, A587 (r2 ALASKA    A M I        1 INTL 20051117)
(XEN) ACPI: FACS DC1F8F80, 0040
(XEN) ACPI: APIC DC1F4808, 0092 (r3 ALASKA    A M I  1072009 AMI     10013)
(XEN) ACPI: FPDT DC1F48A0, 0044 (r1 ALASKA    A M I  1072009 AMI     10013)
(XEN) ACPI: MCFG DC1F48E8, 003C (r1 ALASKA    A M I  1072009 MSFT       97)
(XEN) ACPI: HPET DC1F4928, 0038 (r1 ALASKA    A M I  1072009 AMI.        5)
(XEN) ACPI: SSDT DC1F4960, 036D (r1 SataRe SataTabl     1000 INTL 20091112)
(XEN) ACPI: SSDT DC1F4CD0, 081E (r1  PmRef  Cpu0Ist     3000 INTL 20051117)
(XEN) ACPI: SSDT DC1F54F0, 0A92 (r1  PmRef    CpuPm     3000 INTL 20051117)
(XEN) ACPI: DMAR DC1F5F88, 00B0 (r1 INTEL      SNB         1 INTL        1)
(XEN) ACPI: ASF! DC1F6038, 00A5 (r32 INTEL       HCG        1 TFSM    F4240)
(XEN) System RAM: 32674MB (33458932kB)
(XEN) No NUMA configuration found
(XEN) Faking a node at 0000000000000000-000000081e600000
(XEN) Domain heap initialised
(XEN) found SMP MP-table at 000fd8d0
(XEN) DMI 2.7 present.
(XEN) Using APIC driver default
(XEN) ACPI: PM-Timer IO Port: 0x408
(XEN) ACPI: ACPI SLEEP INFO: pm1x_cnt[404,0], pm1x_evt[400,0]
(XEN) ACPI: 32/64X FACS address mismatch in FADT - dc1f8f80/0000000000000000, using 32
(XEN) ACPI:                  wakeup_vec[dc1f8f8c], vec_size[20]
(XEN) ACPI: Local APIC address 0xfee00000
(XEN) ACPI: LAPIC (acpi_id[0x01] lapic_id[0x00] enabled)
(XEN) Processor #0 6:10 APIC version 21
(XEN) ACPI: LAPIC (acpi_id[0x02] lapic_id[0x02] enabled)
(XEN) Processor #2 6:10 APIC version 21
(XEN) ACPI: LAPIC (acpi_id[0x03] lapic_id[0x04] enabled)
(XEN) Processor #4 6:10 APIC version 21
(XEN) ACPI: LAPIC (acpi_id[0x04] lapic_id[0x06] enabled)
(XEN) Processor #6 6:10 APIC version 21
(XEN) ACPI: LAPIC (acpi_id[0x05] lapic_id[0x01] enabled)
(XEN) Processor #1 6:10 APIC version 21
(XEN) ACPI: LAPIC (acpi_id[0x06] lapic_id[0x03] enabled)
(XEN) Processor #3 6:10 APIC version 21
(XEN) ACPI: LAPIC (acpi_id[0x07] lapic_id[0x05] enabled)
(XEN) Processor #5 6:10 APIC version 21
(XEN) ACPI: LAPIC (acpi_id[0x08] lapic_id[0x07] enabled)
(XEN) Processor #7 6:10 APIC version 21
(XEN) ACPI: LAPIC_NMI (acpi_id[0xff] high edge lint[0x1])
(XEN) ACPI: IOAPIC (id[0x02] address[0xfec00000] gsi_base[0])
(XEN) IOAPIC[0]: apic_id 2, version 32, address 0xfec00000, GSI 0-23
(XEN) ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl)
(XEN) ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level)
(XEN) ACPI: IRQ0 used by override.
(XEN) ACPI: IRQ2 used by override.
(XEN) ACPI: IRQ9 used by override.
(XEN) Enabling APIC mode:  Flat.  Using 1 I/O APICs
(XEN) ACPI: HPET id: 0x8086a701 base: 0xfed00000
(XEN) ERST table was not found
(XEN) Using ACPI (MADT) for SMP configuration information
(XEN) SMP: Allowing 8 CPUs (0 hotplug CPUs)
(XEN) IRQ limits: 24 GSI, 1528 MSI/MSI-X
(XEN) Switched to APIC driver x2apic_cluster.
(XEN) Using scheduler: SMP Credit Scheduler (credit)
(XEN) Detected 2394.660 MHz processor.
(XEN) Initing memory sharing.
(XEN) xstate_init: using cntxt_size: 0x340 and states: 0x7
(XEN) mce_intel.c:1238: MCA Capability: BCAST 1 SER 0 CMCI 1 firstbank 0 extended MCE MSR 0
(XEN) Intel machine check reporting enabled
(XEN) PCI: MCFG configuration 0: base f8000000 segment 0000 buses 00 - 3f
(XEN) PCI: MCFG area at f8000000 reserved in E820
(XEN) PCI: Using MCFG for segment 0000 bus 00-3f
(XEN) Intel VT-d iommu 0 supported page sizes: 4kB.
(XEN) Intel VT-d iommu 1 supported page sizes: 4kB.
(XEN) Intel VT-d Snoop Control not enabled.
(XEN) Intel VT-d Dom0 DMA Passthrough not enabled.
(XEN) Intel VT-d Queued Invalidation enabled.
(XEN) Intel VT-d Interrupt Remapping enabled.
(XEN) Intel VT-d Shared EPT tables not enabled.
(XEN) I/O virtualisation enabled
(XEN)  - Dom0 mode: Relaxed
(XEN) Interrupt remapping enabled
(XEN) Enabled directed EOI with ioapic_ack_old on!
(XEN) ENABLING IO-APIC IRQs
(XEN)  -> Using old ACK method
(XEN) ..TIMER: vector=0xF0 apic1=0 pin1=2 apic2=-1 pin2=-1
(XEN) TSC deadline timer enabled
(XEN) Platform timer is 14.318MHz HPET
(XEN) Allocated console ring of 64 KiB.
(XEN) VMX: Supported advanced features:
(XEN)  - APIC MMIO access virtualisation
(XEN)  - APIC TPR shadow
(XEN)  - Extended Page Tables (EPT)
(XEN)  - Virtual-Processor Identifiers (VPID)
(XEN)  - Virtual NMI
(XEN)  - MSR direct-access bitmap
(XEN)  - Unrestricted Guest
(XEN) HVM: ASIDs enabled.
(XEN) HVM: VMX enabled
(XEN) HVM: Hardware Assisted Paging (HAP) detected
(XEN) HVM: HAP page sizes: 4kB, 2MB
(XEN) Brought up 8 CPUs
(XEN) ACPI sleep modes: S3
(XEN) mcheck_poll: Machine check polling timer started.
(XEN) *** LOADING DOMAIN 0 ***
(XEN)  Xen  kernel: 64-bit, lsb, compat32
(XEN)  Dom0 kernel: 64-bit, PAE, lsb, paddr 0x1000000 -> 0x1c00000
(XEN) PHYSICAL MEMORY ARRANGEMENT:
(XEN)  Dom0 alloc.:   0000000804000000->0000000806000000 (122880 pages to be allocated)
(XEN) VIRTUAL MEMORY ARRANGEMENT:
(XEN)  Loaded kernel: ffffffff81000000->ffffffff81c00000
(XEN)  Init. ramdisk: ffffffff81c00000->ffffffff81c00000
(XEN)  Phys-Mach map: ffffffff81c00000->ffffffff81d00000
(XEN)  Start info:    ffffffff81d00000->ffffffff81d004b4
(XEN)  Page tables:   ffffffff81d01000->ffffffff81d14000
(XEN)  Boot stack:    ffffffff81d14000->ffffffff81d15000
(XEN)  TOTAL:         ffffffff80000000->ffffffff82000000
(XEN)  ENTRY ADDRESS: ffffffff815df1e0
(XEN) Dom0 has maximum 1 VCPUs
(XEN) Scrubbing Free RAM: .............................................................................................................................................................................................................................................................................................................................done.
(XEN) Initial low memory virq threshold set at 0x4000 pages.
(XEN) Std. Loglevel: All
(XEN) Guest Loglevel: Nothing (Rate-limited: Errors and warnings)
(XEN) *** Serial input -> DOM0 (type 'CTRL-a' three times to switch input to Xen)
(XEN) Freed 264kB init memory.
(XEN) PCI add device 0000:00:00.0
(XEN) PCI add device 0000:00:01.0
(XEN) PCI add device 0000:00:01.1
(XEN) PCI add device 0000:00:02.0
(XEN) PCI add device 0000:00:06.0
(XEN) PCI add device 0000:00:16.0
(XEN) PCI add device 0000:00:16.3
(XEN) PCI add device 0000:00:19.0
(XEN) PCI add device 0000:00:1a.0
(XEN) PCI add device 0000:00:1b.0
(XEN) PCI add device 0000:00:1c.0
(XEN) PCI add device 0000:00:1c.4
(XEN) PCI add device 0000:00:1c.6
(XEN) PCI add device 0000:00:1d.0
(XEN) PCI add device 0000:00:1e.0
(XEN) PCI add device 0000:00:1f.0
(XEN) PCI add device 0000:00:1f.2
(XEN) PCI add device 0000:00:1f.3
(XEN) PCI add device 0000:02:00.0
(XEN) PCI add device 0000:03:00.0
(XEN) PCI add device 0000:04:00.0
(XEN) PCI add device 0000:05:00.0
(XEN) PCI add device 0000:06:00.0
(XEN) PCI add device 0000:07:00.0
(XEN) PCI add device 0000:08:00.0
(XEN) PCI add device 0000:09:00.0
(XEN) PCI add device 0000:09:02.0
(XEN) PCI add device 0000:0a:08.0
(XEN) PCI add device 0000:0a:09.0
(XEN) PCI add device 0000:0a:0a.0
(XEN) PCI add device 0000:0a:0b.0
(XEN) traps.c:2600:d0 Domain attempted WRMSR 00000000000001fc from 0x000000000004005f to 0x000000000004005d.
(XEN) Monitor-Mwait will be used to enter C1 state
(XEN) Monitor-Mwait will be used to enter C2 state
(XEN) Monitor-Mwait will be used to enter C3 state
(XEN) Preparing system for ACPI S5 state.
(XEN) Disabling non-boot CPUs ...
(XEN) Entering ACPI S5 state.

[-- Attachment #3: Type: text/plain, Size: 126 bytes --]

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel