Powerdown problem on XEN

xen-devel.lists.xenproject.org archive mirror
 help / color / mirror / Atom feed

* Powerdown problem on XEN | ACPI S5
@ 2013-08-14  8:48 Atom2
  2013-08-14 10:30 ` Jan Beulich
  0 siblings, 1 reply; 28+ messages in thread
From: Atom2 @ 2013-08-14  8:48 UTC (permalink / raw)
  To: xen-devel; +Cc: Ian Campbell, jbeulich

[-- Attachment #1: Type: text/plain, Size: 3680 bytes --]

Hi guys,
I have been liasing with Ian Campbell on the xen-user list and Ian 
suggested I should take this to xen-devel.

The issue I am currently facing is as follows:
I seem to be unable to powerdown my system under the XEN hypervisor by 
issuing the command

	shutdown -h now

from my gentoo-hardened 3.9.5 dom0 machine. Instead of turning off 
power, it goes through a BIOS power-on sequence and reboots. There's no 
other domU running at the time of the attempted powerdown.

If I do the same using *exactly the same dom0 kernel* without XEN 
involved (i.e. boot from my gentoo-hardened 3.9.5 kernel only), 
powerdown reliably works as expected and 'shutdown -h now' turns off the 
system's power.

I have tested both 4.2.2 (the gentoo e-build) and 4.3 (downloaded 
directly from xenbits). There's no difference between those two versions 
in terms of reboot versus powerdown.

Upon advise from Ian I have also experimented with the various reboot= 
options on the command line - also with no success.

Adding a serial console (also thanks to Ian's input) I was able to 
examine the messages during shutdown. I have attached those from 4.2.2 
for your reference.

For the latest test using Xen 4.2.2 I made a few changes to the source 
file xen/arch/x86/acpi/power.c to see where the actual problem may be 
hidden. I'm far from being a kernel or XEN programmer, but I am able to 
read and basically understand and (to an extent) modify C code. 
Supported by finding and identifying the messages I had on the serial 
console I decided to add a few additional printk statements after the 
last message that was displayed on the console to see where the system 
probably crashes / the problems could possibly start:

After my change, the relevant code snippet now looks as follows (NOTE: 
The printk messages starting with "After" or "Before" stem from me, the 
first one and the one within the if-construct are both unchanged; the 
initial one was originally always displayed on the serial console as the 
final line):

     printk("Entering ACPI S%d state.\n", state);

     local_irq_save(flags);
     printk("After local_irq_save\n");

     spin_debug_disable();
     printk("After spin_debug_disable\n");

     if ( (error = device_power_down()) )
     {
         printk(XENLOG_ERR "Some devices failed to power down.");
         system_state = SYS_STATE_resume;
         goto done;
     }

     printk("Before ACPI_FLUSH_CPU_CACHE\n");
     ACPI_FLUSH_CPU_CACHE();
     printk("After ACPI_FLUSH_CPU_CACHE\n");

The final few messages of the *new* output *after my amateur mods* on 
the serial console now read as follows:
(XEN) Entering ACPI S5 state.
(XEN) After local_irq_save
(XEN) After spin_debug_disable

There is neither a message reading
(XEN) Some devices failed to power down.
(NOTE: this printk statement however has a XENLOG_ERR before the text - 
so I am not sure whether that should actually appear on the serial 
console at all)

nor one reading
(XEN) Before ACPI_FLUSH_CPU_CACHE

This to me seems to indicate, that the problematic code is somewhere in
between the following lines:
     if ( (error = device_power_down()) )
     {
         printk(XENLOG_ERR "Some devices failed to power down.");
         system_state = SYS_STATE_resume;
         goto done;
     }

I hope that might provide you with some insight which, with your help, 
could be used to make a step forward.

On the other hand I might be completely on the wrong track as I have no 
clue where the actual requested power-down (or as is: reboot) actually 
happens. That was not obvious for me from the code in power.c without 
further knowledge ...

Many thanks in advance for suggestions.

[-- Attachment #2: XEN console --]
[-- Type: text/plain, Size: 9636 bytes --]

 \ \/ /___ _ __   | || |  |___ \  |___ \ 
  \  // _ \ '_ \  | || |_   __) |   __) |
  /  \  __/ | | | |__   _| / __/ _ / __/ 
 /_/\_\___|_| |_|    |_|(_)_____(_)_____|

(XEN) Xen version 4.2.2 (@herrenhauspark.com) (gcc (Gentoo Hardened 4.6.3 p1.13, pie-0.5.2) 4.6.3) Fri Jul  5 12:09:53 CEST 2013
(XEN) Latest ChangeSet: unavailable
(XEN) Bootloader: GNU GRUB 0.97
(XEN) Command line: console=com1 loglvl=all com1=115200,8n1,0x2d8,10 dom0_mem=512M,max:512M dom0_max_vcpus=1 dom0_vcpus_pin iommu=1
(XEN) Video information:
(XEN)  VGA is text mode 80x25, font 8x16
(XEN)  VBE/DDC methods: V2; EDID transfer time: 1 seconds
(XEN) Disc information:
(XEN)  Found 2 MBR signatures
(XEN)  Found 2 EDD information structures
(XEN) Xen-e820 RAM map:
(XEN)  0000000000000000 - 0000000000099c00 (usable)
(XEN)  0000000000099c00 - 00000000000a0000 (reserved)
(XEN)  00000000000e0000 - 0000000000100000 (reserved)
(XEN)  0000000000100000 - 0000000020000000 (usable)
(XEN)  0000000020000000 - 0000000020200000 (reserved)
(XEN)  0000000020200000 - 0000000040000000 (usable)
(XEN)  0000000040000000 - 0000000040200000 (reserved)
(XEN)  0000000040200000 - 00000000db9f1000 (usable)
(XEN)  00000000db9f1000 - 00000000dc0db000 (reserved)
(XEN)  00000000dc0db000 - 00000000dc1fa000 (ACPI NVS)
(XEN)  00000000dc1fa000 - 00000000dc652000 (reserved)
(XEN)  00000000dc652000 - 00000000dc653000 (usable)
(XEN)  00000000dc653000 - 00000000dc696000 (ACPI NVS)
(XEN)  00000000dc696000 - 00000000dcdba000 (usable)
(XEN)  00000000dcdba000 - 00000000dcff2000 (reserved)
(XEN)  00000000dcff2000 - 00000000dd000000 (usable)
(XEN)  00000000dd800000 - 00000000dfa00000 (reserved)
(XEN)  00000000f8000000 - 00000000fc000000 (reserved)
(XEN)  00000000fec00000 - 00000000fec01000 (reserved)
(XEN)  00000000fed00000 - 00000000fed04000 (reserved)
(XEN)  00000000fed1c000 - 00000000fed20000 (reserved)
(XEN)  00000000fee00000 - 00000000fee01000 (reserved)
(XEN)  00000000ff000000 - 0000000100000000 (reserved)
(XEN)  0000000100000000 - 000000081e600000 (usable)
(XEN) ACPI: RSDP 000F0490, 0024 (r2 ALASKA)
(XEN) ACPI: XSDT DC1EA078, 0074 (r1 ALASKA    A M I  1072009 AMI     10013)
(XEN) ACPI: FACP DC1F4710, 00F4 (r4 ALASKA    A M I  1072009 AMI     10013)
(XEN) ACPI: DSDT DC1EA188, A587 (r2 ALASKA    A M I        1 INTL 20051117)
(XEN) ACPI: FACS DC1F8F80, 0040
(XEN) ACPI: APIC DC1F4808, 0092 (r3 ALASKA    A M I  1072009 AMI     10013)
(XEN) ACPI: FPDT DC1F48A0, 0044 (r1 ALASKA    A M I  1072009 AMI     10013)
(XEN) ACPI: MCFG DC1F48E8, 003C (r1 ALASKA    A M I  1072009 MSFT       97)
(XEN) ACPI: HPET DC1F4928, 0038 (r1 ALASKA    A M I  1072009 AMI.        5)
(XEN) ACPI: SSDT DC1F4960, 036D (r1 SataRe SataTabl     1000 INTL 20091112)
(XEN) ACPI: SSDT DC1F4CD0, 081E (r1  PmRef  Cpu0Ist     3000 INTL 20051117)
(XEN) ACPI: SSDT DC1F54F0, 0A92 (r1  PmRef    CpuPm     3000 INTL 20051117)
(XEN) ACPI: DMAR DC1F5F88, 00B0 (r1 INTEL      SNB         1 INTL        1)
(XEN) ACPI: ASF! DC1F6038, 00A5 (r32 INTEL       HCG        1 TFSM    F4240)
(XEN) System RAM: 32674MB (33458932kB)
(XEN) No NUMA configuration found
(XEN) Faking a node at 0000000000000000-000000081e600000
(XEN) Domain heap initialised
(XEN) found SMP MP-table at 000fd8d0
(XEN) DMI 2.7 present.
(XEN) Using APIC driver default
(XEN) ACPI: PM-Timer IO Port: 0x408
(XEN) ACPI: ACPI SLEEP INFO: pm1x_cnt[404,0], pm1x_evt[400,0]
(XEN) ACPI: 32/64X FACS address mismatch in FADT - dc1f8f80/0000000000000000, using 32
(XEN) ACPI:                  wakeup_vec[dc1f8f8c], vec_size[20]
(XEN) ACPI: Local APIC address 0xfee00000
(XEN) ACPI: LAPIC (acpi_id[0x01] lapic_id[0x00] enabled)
(XEN) Processor #0 6:10 APIC version 21
(XEN) ACPI: LAPIC (acpi_id[0x02] lapic_id[0x02] enabled)
(XEN) Processor #2 6:10 APIC version 21
(XEN) ACPI: LAPIC (acpi_id[0x03] lapic_id[0x04] enabled)
(XEN) Processor #4 6:10 APIC version 21
(XEN) ACPI: LAPIC (acpi_id[0x04] lapic_id[0x06] enabled)
(XEN) Processor #6 6:10 APIC version 21
(XEN) ACPI: LAPIC (acpi_id[0x05] lapic_id[0x01] enabled)
(XEN) Processor #1 6:10 APIC version 21
(XEN) ACPI: LAPIC (acpi_id[0x06] lapic_id[0x03] enabled)
(XEN) Processor #3 6:10 APIC version 21
(XEN) ACPI: LAPIC (acpi_id[0x07] lapic_id[0x05] enabled)
(XEN) Processor #5 6:10 APIC version 21
(XEN) ACPI: LAPIC (acpi_id[0x08] lapic_id[0x07] enabled)
(XEN) Processor #7 6:10 APIC version 21
(XEN) ACPI: LAPIC_NMI (acpi_id[0xff] high edge lint[0x1])
(XEN) ACPI: IOAPIC (id[0x02] address[0xfec00000] gsi_base[0])
(XEN) IOAPIC[0]: apic_id 2, version 32, address 0xfec00000, GSI 0-23
(XEN) ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl)
(XEN) ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level)
(XEN) ACPI: IRQ0 used by override.
(XEN) ACPI: IRQ2 used by override.
(XEN) ACPI: IRQ9 used by override.
(XEN) Enabling APIC mode:  Flat.  Using 1 I/O APICs
(XEN) ACPI: HPET id: 0x8086a701 base: 0xfed00000
(XEN) ERST table was not found
(XEN) Using ACPI (MADT) for SMP configuration information
(XEN) SMP: Allowing 8 CPUs (0 hotplug CPUs)
(XEN) IRQ limits: 24 GSI, 1528 MSI/MSI-X
(XEN) Switched to APIC driver x2apic_cluster.
(XEN) Using scheduler: SMP Credit Scheduler (credit)
(XEN) Detected 2394.660 MHz processor.
(XEN) Initing memory sharing.
(XEN) xstate_init: using cntxt_size: 0x340 and states: 0x7
(XEN) mce_intel.c:1238: MCA Capability: BCAST 1 SER 0 CMCI 1 firstbank 0 extended MCE MSR 0
(XEN) Intel machine check reporting enabled
(XEN) PCI: MCFG configuration 0: base f8000000 segment 0000 buses 00 - 3f
(XEN) PCI: MCFG area at f8000000 reserved in E820
(XEN) PCI: Using MCFG for segment 0000 bus 00-3f
(XEN) Intel VT-d iommu 0 supported page sizes: 4kB.
(XEN) Intel VT-d iommu 1 supported page sizes: 4kB.
(XEN) Intel VT-d Snoop Control not enabled.
(XEN) Intel VT-d Dom0 DMA Passthrough not enabled.
(XEN) Intel VT-d Queued Invalidation enabled.
(XEN) Intel VT-d Interrupt Remapping enabled.
(XEN) Intel VT-d Shared EPT tables not enabled.
(XEN) I/O virtualisation enabled
(XEN)  - Dom0 mode: Relaxed
(XEN) Interrupt remapping enabled
(XEN) Enabled directed EOI with ioapic_ack_old on!
(XEN) ENABLING IO-APIC IRQs
(XEN)  -> Using old ACK method
(XEN) ..TIMER: vector=0xF0 apic1=0 pin1=2 apic2=-1 pin2=-1
(XEN) TSC deadline timer enabled
(XEN) Platform timer is 14.318MHz HPET
(XEN) Allocated console ring of 64 KiB.
(XEN) VMX: Supported advanced features:
(XEN)  - APIC MMIO access virtualisation
(XEN)  - APIC TPR shadow
(XEN)  - Extended Page Tables (EPT)
(XEN)  - Virtual-Processor Identifiers (VPID)
(XEN)  - Virtual NMI
(XEN)  - MSR direct-access bitmap
(XEN)  - Unrestricted Guest
(XEN) HVM: ASIDs enabled.
(XEN) HVM: VMX enabled
(XEN) HVM: Hardware Assisted Paging (HAP) detected
(XEN) HVM: HAP page sizes: 4kB, 2MB
(XEN) Brought up 8 CPUs
(XEN) ACPI sleep modes: S3
(XEN) mcheck_poll: Machine check polling timer started.
(XEN) *** LOADING DOMAIN 0 ***
(XEN)  Xen  kernel: 64-bit, lsb, compat32
(XEN)  Dom0 kernel: 64-bit, PAE, lsb, paddr 0x1000000 -> 0x1c00000
(XEN) PHYSICAL MEMORY ARRANGEMENT:
(XEN)  Dom0 alloc.:   0000000804000000->0000000806000000 (122880 pages to be allocated)
(XEN) VIRTUAL MEMORY ARRANGEMENT:
(XEN)  Loaded kernel: ffffffff81000000->ffffffff81c00000
(XEN)  Init. ramdisk: ffffffff81c00000->ffffffff81c00000
(XEN)  Phys-Mach map: ffffffff81c00000->ffffffff81d00000
(XEN)  Start info:    ffffffff81d00000->ffffffff81d004b4
(XEN)  Page tables:   ffffffff81d01000->ffffffff81d14000
(XEN)  Boot stack:    ffffffff81d14000->ffffffff81d15000
(XEN)  TOTAL:         ffffffff80000000->ffffffff82000000
(XEN)  ENTRY ADDRESS: ffffffff815df1e0
(XEN) Dom0 has maximum 1 VCPUs
(XEN) Scrubbing Free RAM: .............................................................................................................................................................................................................................................................................................................................done.
(XEN) Initial low memory virq threshold set at 0x4000 pages.
(XEN) Std. Loglevel: All
(XEN) Guest Loglevel: Nothing (Rate-limited: Errors and warnings)
(XEN) *** Serial input -> DOM0 (type 'CTRL-a' three times to switch input to Xen)
(XEN) Freed 264kB init memory.
(XEN) PCI add device 0000:00:00.0
(XEN) PCI add device 0000:00:01.0
(XEN) PCI add device 0000:00:01.1
(XEN) PCI add device 0000:00:02.0
(XEN) PCI add device 0000:00:06.0
(XEN) PCI add device 0000:00:16.0
(XEN) PCI add device 0000:00:16.3
(XEN) PCI add device 0000:00:19.0
(XEN) PCI add device 0000:00:1a.0
(XEN) PCI add device 0000:00:1b.0
(XEN) PCI add device 0000:00:1c.0
(XEN) PCI add device 0000:00:1c.4
(XEN) PCI add device 0000:00:1c.6
(XEN) PCI add device 0000:00:1d.0
(XEN) PCI add device 0000:00:1e.0
(XEN) PCI add device 0000:00:1f.0
(XEN) PCI add device 0000:00:1f.2
(XEN) PCI add device 0000:00:1f.3
(XEN) PCI add device 0000:02:00.0
(XEN) PCI add device 0000:03:00.0
(XEN) PCI add device 0000:04:00.0
(XEN) PCI add device 0000:05:00.0
(XEN) PCI add device 0000:06:00.0
(XEN) PCI add device 0000:07:00.0
(XEN) PCI add device 0000:08:00.0
(XEN) PCI add device 0000:09:00.0
(XEN) PCI add device 0000:09:02.0
(XEN) PCI add device 0000:0a:08.0
(XEN) PCI add device 0000:0a:09.0
(XEN) PCI add device 0000:0a:0a.0
(XEN) PCI add device 0000:0a:0b.0
(XEN) traps.c:2600:d0 Domain attempted WRMSR 00000000000001fc from 0x000000000004005f to 0x000000000004005d.
(XEN) Monitor-Mwait will be used to enter C1 state
(XEN) Monitor-Mwait will be used to enter C2 state
(XEN) Monitor-Mwait will be used to enter C3 state
(XEN) Preparing system for ACPI S5 state.
(XEN) Disabling non-boot CPUs ...
(XEN) Entering ACPI S5 state.

[-- Attachment #3: Type: text/plain, Size: 126 bytes --]

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

^ permalink raw reply	[flat|nested] 28+ messages in thread

* Re: Powerdown problem on XEN | ACPI S5
  2013-08-14  8:48 Powerdown problem on XEN | ACPI S5 Atom2
@ 2013-08-14 10:30 ` Jan Beulich
  2013-08-14 13:52   ` Atom2
  0 siblings, 1 reply; 28+ messages in thread
From: Jan Beulich @ 2013-08-14 10:30 UTC (permalink / raw)
  To: Atom2; +Cc: xen-devel, Ian Campbell

>>> On 14.08.13 at 10:48, Atom2 <ariel.atom2@web2web.at> wrote:
>      spin_debug_disable();
>      printk("After spin_debug_disable\n");
> 
>      if ( (error = device_power_down()) )
>      {
>          printk(XENLOG_ERR "Some devices failed to power down.");
>          system_state = SYS_STATE_resume;
>          goto done;
>      }
> 
>      printk("Before ACPI_FLUSH_CPU_CACHE\n");
>      ACPI_FLUSH_CPU_CACHE();
>      printk("After ACPI_FLUSH_CPU_CACHE\n");
> 
> The final few messages of the *new* output *after my amateur mods* on 
> the serial console now read as follows:
> (XEN) Entering ACPI S5 state.
> (XEN) After local_irq_save
> (XEN) After spin_debug_disable
> 
> There is neither a message reading
> (XEN) Some devices failed to power down.
> 
> nor one reading
> (XEN) Before ACPI_FLUSH_CPU_CACHE

Right, possibly because the first thing device_power_down()
does is console_suspend(). Comment that out via your debug
patch, and you might see more.

It would be particularly interesting to know whether perhaps
some of the ACPI registers live in memory space on that
system - I already have a patch queued up (but not submitted
yet) that fixes problems in that case.

Jan

^ permalink raw reply	[flat|nested] 28+ messages in thread

* Re: Powerdown problem on XEN | ACPI S5
  2013-08-14 10:30 ` Jan Beulich
@ 2013-08-14 13:52   ` Atom2
  2013-08-14 14:00     ` Andrew Cooper
  0 siblings, 1 reply; 28+ messages in thread
From: Atom2 @ 2013-08-14 13:52 UTC (permalink / raw)
  To: Jan Beulich; +Cc: xen-devel, Ian Campbell

Hi Jan,
thanks for reply. You are obviously right that the first thing 
device_power_down does, is console_suspend(). I don't know why that 
escaped my eyes when I originally searched the file ...

Anyways, I have now disabled console_suspend() and also added a few more 
lines to the code with printk statements indicating up to which point 
the system had gone (without errors). With hindsight I guess the new 
printk() statements might not have been required as now, with the 
console still active, a panic message pops up at the end, resulting in 
rebooting the system:

(XEN) Disabling non-boot CPUs ...
(XEN) Entering ACPI S5 state.
(XEN) After local_irq_save
(XEN) After spin_debug_disable
(XEN) After time_suspend
(XEN) After li8259_suspend
(XEN) After ioapic_suspend
(XEN) DMAR_IQA_REG = 80d87c002
(XEN) DMAR_IQH_REG = 120
(XEN) DMAR_IQT_REG = 140
(XEN)
(XEN) ****************************************
(XEN) Panic on CPU 0:
(XEN) queue invalidate wait descriptor was not executed
(XEN) ****************************************
(XEN)
(XEN) Reboot in five seconds...
(XEN) Resetting with ACPI MEMORY or I/O RESET_REG.

NOTE: All the messages starting with "(XEN) After" are from my changes 
to the code; the rest is as is - except me commenting out 
console_suspend() in power.c. I hope that helps in resolving the issue 
and the panic is not just the result of a knock-on effect from 
commenting out console_suspend() earlier.

Am 14.08.13 12:30, schrieb Jan Beulich:
[...]
> It would be particularly interesting to know whether perhaps
> some of the ACPI registers live in memory space on that
> system - I already have a patch queued up (but not submitted
> yet) that fixes problems in that case.
I don't know how I can find out whether ACPI registers live in memory, 
but if you can tell me what I need to do to provide you with that 
information, I am more than happy to help on that issue.

Many thanks

^ permalink raw reply	[flat|nested] 28+ messages in thread

* Re: Powerdown problem on XEN | ACPI S5
  2013-08-14 13:52   ` Atom2
@ 2013-08-14 14:00     ` Andrew Cooper
  2013-08-14 17:00       ` Atom2
  0 siblings, 1 reply; 28+ messages in thread
From: Andrew Cooper @ 2013-08-14 14:00 UTC (permalink / raw)
  To: Atom2; +Cc: xen-devel, Ian Campbell, Jan Beulich

On 14/08/13 14:52, Atom2 wrote:
> Hi Jan,
> thanks for reply. You are obviously right that the first thing
> device_power_down does, is console_suspend(). I don't know why that
> escaped my eyes when I originally searched the file ...
>
> Anyways, I have now disabled console_suspend() and also added a few
> more lines to the code with printk statements indicating up to which
> point the system had gone (without errors). With hindsight I guess the
> new printk() statements might not have been required as now, with the
> console still active, a panic message pops up at the end, resulting in
> rebooting the system:
>
> (XEN) Disabling non-boot CPUs ...
> (XEN) Entering ACPI S5 state.
> (XEN) After local_irq_save
> (XEN) After spin_debug_disable
> (XEN) After time_suspend
> (XEN) After li8259_suspend
> (XEN) After ioapic_suspend
> (XEN) DMAR_IQA_REG = 80d87c002
> (XEN) DMAR_IQH_REG = 120
> (XEN) DMAR_IQT_REG = 140
> (XEN)
> (XEN) ****************************************
> (XEN) Panic on CPU 0:
> (XEN) queue invalidate wait descriptor was not executed
> (XEN) ****************************************
> (XEN)
> (XEN) Reboot in five seconds...
> (XEN) Resetting with ACPI MEMORY or I/O RESET_REG.
>
> NOTE: All the messages starting with "(XEN) After" are from my changes
> to the code; the rest is as is - except me commenting out
> console_suspend() in power.c. I hope that helps in resolving the issue
> and the panic is not just the result of a knock-on effect from
> commenting out console_suspend() earlier.

Huh - I thought I had fixed this issue already.

Can you confirm exactly which version of Xen you are using (including
changeset), and perhaps compile in this patch:

diff --git a/xen/drivers/passthrough/vtd/qinval.c
b/xen/drivers/passthrough/vtd/qinval.c
index 6a410d8..d023b26 100644
--- a/xen/drivers/passthrough/vtd/qinval.c
+++ b/xen/drivers/passthrough/vtd/qinval.c
@@ -220,6 +220,7 @@ static int queue_invalidate_wait(struct iommu *iommu,
             if ( NOW() > (start_time + DMAR_OPERATION_TIMEOUT) )
             {
                 print_qi_regs(iommu);
+                WARN();
                 panic("queue invalidate wait descriptor was not
executed\n");
             }
             cpu_relax();

~Andrew

^ permalink raw reply related	[flat|nested] 28+ messages in thread

* Re: Powerdown problem on XEN | ACPI S5
  2013-08-14 14:00     ` Andrew Cooper
@ 2013-08-14 17:00       ` Atom2
  2013-08-14 17:30         ` Andrew Cooper
  0 siblings, 1 reply; 28+ messages in thread
From: Atom2 @ 2013-08-14 17:00 UTC (permalink / raw)
  To: Andrew Cooper; +Cc: xen-devel, Ian Campbell, Jan Beulich

[-- Attachment #1: Type: text/plain, Size: 3558 bytes --]

Hi Andrew,
please see my inline comments further below.

And many thanks to all of you for your support so far!

Am 14.08.13 16:00, schrieb Andrew Cooper:
> On 14/08/13 14:52, Atom2 wrote:
>> Hi Jan,
>> thanks for reply. You are obviously right that the first thing
>> device_power_down does, is console_suspend(). I don't know why that
>> escaped my eyes when I originally searched the file ...
>>
>> Anyways, I have now disabled console_suspend() and also added a few
>> more lines to the code with printk statements indicating up to which
>> point the system had gone (without errors). With hindsight I guess the
>> new printk() statements might not have been required as now, with the
>> console still active, a panic message pops up at the end, resulting in
>> rebooting the system:
>>
>> (XEN) Disabling non-boot CPUs ...
>> (XEN) Entering ACPI S5 state.
>> (XEN) After local_irq_save
>> (XEN) After spin_debug_disable
>> (XEN) After time_suspend
>> (XEN) After li8259_suspend
>> (XEN) After ioapic_suspend
>> (XEN) DMAR_IQA_REG = 80d87c002
>> (XEN) DMAR_IQH_REG = 120
>> (XEN) DMAR_IQT_REG = 140
>> (XEN)
>> (XEN) ****************************************
>> (XEN) Panic on CPU 0:
>> (XEN) queue invalidate wait descriptor was not executed
>> (XEN) ****************************************
>> (XEN)
>> (XEN) Reboot in five seconds...
>> (XEN) Resetting with ACPI MEMORY or I/O RESET_REG.
>>
>> NOTE: All the messages starting with "(XEN) After" are from my changes
>> to the code; the rest is as is - except me commenting out
>> console_suspend() in power.c. I hope that helps in resolving the issue
>> and the panic is not just the result of a knock-on effect from
>> commenting out console_suspend() earlier.
>
> Huh - I thought I had fixed this issue already.
>
> Can you confirm exactly which version of Xen you are using (including
> changeset), and perhaps compile in this patch:
The version I am using is 4.2.2 from the gentoo ebuild. Interesting 
enough, the log in the consoles states
(XEN) Latest ChangeSet: unavailable
I don't know what to make out of this - or is there any other way to 
figure out, what you are after?

The alternative would be to apply the WARN() to the 4.3.0 version I have 
downloaded yesterday from xenbits at 
http://www.xenproject.org/downloads/xen-archives/supported-xen-43-series/xen-430/287-xen-430-2/file.html 
(FYI: the reboot also happened there). If that helps, I'll rerun it on 
the 4.3.0 version. So far I have used the gentoo version as this allows 
me to stay within the portage system.
>
> diff --git a/xen/drivers/passthrough/vtd/qinval.c
> b/xen/drivers/passthrough/vtd/qinval.c
> index 6a410d8..d023b26 100644
> --- a/xen/drivers/passthrough/vtd/qinval.c
> +++ b/xen/drivers/passthrough/vtd/qinval.c
> @@ -220,6 +220,7 @@ static int queue_invalidate_wait(struct iommu *iommu,
>               if ( NOW() > (start_time + DMAR_OPERATION_TIMEOUT) )
>               {
>                   print_qi_regs(iommu);
> +                WARN();
>                   panic("queue invalidate wait descriptor was not
> executed\n");
>               }
>               cpu_relax();
I have manually applied the patch - which was just an added
	WARN();
inbetween if I read the patch correctly; the rest was already there in 
4.2.2 (and also 4.3.0 - I checked its source as well). I have attached 
the serial log from my 4.2.2 run to prevent line-wrap. I hope that helps.

>
> ~Andrew
>
> _______________________________________________
> Xen-devel mailing list
> Xen-devel@lists.xen.org
> http://lists.xen.org/xen-devel
>

[-- Attachment #2: XEN console log --]
[-- Type: text/plain, Size: 3453 bytes --]

(XEN) Entering ACPI S5 state.
(XEN) After local_irq_save
(XEN) After spin_debug_disable
(XEN) After time_suspend
(XEN) After li8259_suspend
(XEN) After ioapic_suspend
(XEN) DMAR_IQA_REG = 80d87c002
(XEN) DMAR_IQH_REG = 120
(XEN) DMAR_IQT_REG = 140
(XEN) Xen WARN at qinval.c:223
(XEN) ----[ Xen-4.2.2  x86_64  debug=n  Not tainted ]----
(XEN) CPU:    0
(XEN) RIP:    e008:[<ffff82c48014543c>] invalidate_sync+0x22c/0x280
(XEN) RFLAGS: 0000000000010082   CONTEXT: hypervisor
(XEN) rax: ffff82c4802f14a0   rbx: ffff83080d8855b0   rcx: 0000000000000000
(XEN) rdx: ffff82c4802a7f18   rsi: 000000000000000a   rdi: ffff82c48025a640
(XEN) rbp: 0000000b2ae4ae9f   rsp: ffff82c4802a7d20   r8:  0000000000000004
(XEN) r9:  0000000000000002   r10: 0000000000000003   r11: 0000000000000400
(XEN) r12: ffff83080d8855e4   r13: ffff83080d88562c   r14: 0000000000000002
(XEN) r15: 0000000000000002   cr0: 000000008005003b   cr4: 00000000000426f0
(XEN) cr3: 00000000db6b5000   cr2: ffff88001a89ea48
(XEN) ds: 0000   es: 0000   fs: 0000   gs: 0000   ss: e010   cs: e008
(XEN) Xen stack trace from rsp=ffff82c4802a7d20:
(XEN)    0000000000000002 000000080d87c002 0000000000000082 000000000d87c002
(XEN)    0000000000000086 ffff83080d8855b0 0000000000000000 1000000000000000
(XEN)    0000000000000000 0000000000000000 0000000000000000 ffff82c480145546
(XEN)    0000000000000000 0000000000000000 0000000800000004 ffff83080d8855b0
(XEN)    ffff83080d885530 ffff83080d885640 ffff82c48024bbe0 0000000000000000
(XEN)    ffff82c4802f14e0 ffff82c4801413ee 0000000000000000 0000000000000086
(XEN)    ffff82c4802f14a0 0000000000000005 0000000000000000 0000000000000000
(XEN)    ffff82c4802db740 ffff82c480141e05 0000000000000082 0000000000000086
(XEN)    ffff82c4802a7f18 ffff82c4802f14a0 0000000000000005 0000000000000000
(XEN)    0000000000000000 ffff82c4802db740 ffff82c4802f14e0 ffff82c48019a245
(XEN)    ffff82c4802a7f18 00000000ffffffff 0000000000000286 ffff8300dc697000
(XEN)    ffff8300dc697000 ffff830669301fc0 ffff8300dc697000 ffff82c4802f1488
(XEN)    ffff82c4802db740 ffff82c4802db740 ffff82c4802f14e0 ffff82c4801054ae
(XEN)    ffff8300dc697180 0000000000000000 ffff82c4802f1590 ffff82c480125670
(XEN)    ffff82c4802f1570 ffff82c480125878 ffff82c4802a7f18 ffff8300dc697000
(XEN)    00000000ffffffff ffff82c480156933 ffff8300dcda6000 0000000000000000
(XEN)    0000000000000003 0000000000000005 0000000000002003 ffffffff815adf58
(XEN)    0000000000000000 0000000000000246 0000000000000370 0000000000000005
(XEN)    0000000000000000 0000000000000000 ffffffff810010ea 0000000000002003
(XEN)    0000000000003c03 ffff880019dd1d18 0000010000000000 ffffffff810010ea
(XEN) Xen call trace:
(XEN)    [<ffff82c48014543c>] invalidate_sync+0x22c/0x280
(XEN)    [<ffff82c480145546>] flush_iotlb_qi+0xb6/0x100
(XEN)    [<ffff82c4801413ee>] iommu_flush_all+0x8e/0xf0
(XEN)    [<ffff82c480141e05>] vtd_suspend+0x55/0x110
(XEN)    [<ffff82c48019a245>] enter_state_helper+0x235/0x430
(XEN)    [<ffff82c4801054ae>] continue_hypercall_tasklet_handler+0xee/0x100
(XEN)    [<ffff82c480125670>] do_tasklet_work+0x60/0xc0
(XEN)    [<ffff82c480125878>] do_tasklet+0x78/0xb0
(XEN)    [<ffff82c480156933>] idle_loop+0x23/0x50
(XEN)
(XEN)
(XEN) ****************************************
(XEN) Panic on CPU 0:
(XEN) queue invalidate wait descriptor was not executed
(XEN) ****************************************
(XEN)
(XEN) Reboot in five seconds...
(XEN) Resetting with ACPI MEMORY or I/O RESET_REG.

[-- Attachment #3: Type: text/plain, Size: 126 bytes --]

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

^ permalink raw reply	[flat|nested] 28+ messages in thread

* Re: Powerdown problem on XEN | ACPI S5
  2013-08-14 17:00       ` Atom2
@ 2013-08-14 17:30         ` Andrew Cooper
  2013-08-14 18:40           ` Atom2
  0 siblings, 1 reply; 28+ messages in thread
From: Andrew Cooper @ 2013-08-14 17:30 UTC (permalink / raw)
  To: Atom2; +Cc: xen-devel, Ian Campbell, Jan Beulich

On 14/08/13 18:00, Atom2 wrote:
> Hi Andrew,
> please see my inline comments further below.
>
> And many thanks to all of you for your support so far!
>
> Am 14.08.13 16:00, schrieb Andrew Cooper:
>> On 14/08/13 14:52, Atom2 wrote:
>>> Hi Jan,
>>> thanks for reply. You are obviously right that the first thing
>>> device_power_down does, is console_suspend(). I don't know why that
>>> escaped my eyes when I originally searched the file ...
>>>
>>> Anyways, I have now disabled console_suspend() and also added a few
>>> more lines to the code with printk statements indicating up to which
>>> point the system had gone (without errors). With hindsight I guess the
>>> new printk() statements might not have been required as now, with the
>>> console still active, a panic message pops up at the end, resulting in
>>> rebooting the system:
>>>
>>> (XEN) Disabling non-boot CPUs ...
>>> (XEN) Entering ACPI S5 state.
>>> (XEN) After local_irq_save
>>> (XEN) After spin_debug_disable
>>> (XEN) After time_suspend
>>> (XEN) After li8259_suspend
>>> (XEN) After ioapic_suspend
>>> (XEN) DMAR_IQA_REG = 80d87c002
>>> (XEN) DMAR_IQH_REG = 120
>>> (XEN) DMAR_IQT_REG = 140
>>> (XEN)
>>> (XEN) ****************************************
>>> (XEN) Panic on CPU 0:
>>> (XEN) queue invalidate wait descriptor was not executed
>>> (XEN) ****************************************
>>> (XEN)
>>> (XEN) Reboot in five seconds...
>>> (XEN) Resetting with ACPI MEMORY or I/O RESET_REG.
>>>
>>> NOTE: All the messages starting with "(XEN) After" are from my changes
>>> to the code; the rest is as is - except me commenting out
>>> console_suspend() in power.c. I hope that helps in resolving the issue
>>> and the panic is not just the result of a knock-on effect from
>>> commenting out console_suspend() earlier.
>>
>> Huh - I thought I had fixed this issue already.
>>
>> Can you confirm exactly which version of Xen you are using (including
>> changeset), and perhaps compile in this patch:
> The version I am using is 4.2.2 from the gentoo ebuild. Interesting
> enough, the log in the consoles states
> (XEN) Latest ChangeSet: unavailable
> I don't know what to make out of this - or is there any other way to
> figure out, what you are after?

Ah - that good old gem.  The old logic to detect changesets was very bad
if the build wasn't happening in a mercurial tree.  It is better in 4.3.

>
> The alternative would be to apply the WARN() to the 4.3.0 version I
> have downloaded yesterday from xenbits at
> http://www.xenproject.org/downloads/xen-archives/supported-xen-43-series/xen-430/287-xen-430-2/file.html
> (FYI: the reboot also happened there). If that helps, I'll rerun it on
> the 4.3.0 version. So far I have used the gentoo version as this
> allows me to stay within the portage system.

That's fine - the bug is likely more damage from XSA-36

>>
>> diff --git a/xen/drivers/passthrough/vtd/qinval.c
>> b/xen/drivers/passthrough/vtd/qinval.c
>> index 6a410d8..d023b26 100644
>> --- a/xen/drivers/passthrough/vtd/qinval.c
>> +++ b/xen/drivers/passthrough/vtd/qinval.c
>> @@ -220,6 +220,7 @@ static int queue_invalidate_wait(struct iommu
>> *iommu,
>>               if ( NOW() > (start_time + DMAR_OPERATION_TIMEOUT) )
>>               {
>>                   print_qi_regs(iommu);
>> +                WARN();
>>                   panic("queue invalidate wait descriptor was not
>> executed\n");
>>               }
>>               cpu_relax();
> I have manually applied the patch - which was just an added
>     WARN();
> inbetween if I read the patch correctly; the rest was already there in
> 4.2.2 (and also 4.3.0 - I checked its source as well). I have attached
> the serial log from my 4.2.2 run to prevent line-wrap. I hope that helps.

Do you have the boot time dmesg output?

The problem here is that a queued_invalidate wait descriptor has been
issued, and has not been completed within 1 second.  In all previous
cases I have debugged, this is actually because we already turned off
the IOMMU, then tried to turn it off again.

Could you perhaps try with this patch as well?

diff --git a/xen/drivers/passthrough/vtd/iommu.c
b/xen/drivers/passthrough/vtd/iommu.c
index 071a91b..45fff48 100644
--- a/xen/drivers/passthrough/vtd/iommu.c
+++ b/xen/drivers/passthrough/vtd/iommu.c
@@ -791,6 +791,9 @@ static void iommu_disable_translation(struct iommu
*iommu)
     u32 sts;
     unsigned long flags;
 
+    printk("**Debug: Disabling translation for iommu %"PRId32"\n",
iommu->index);
+    WARN();
+
     /* apply platform specific errata workarounds */
     vtd_ops_preamble_quirk(iommu);
 

~Andrew

^ permalink raw reply related	[flat|nested] 28+ messages in thread

* Re: Powerdown problem on XEN | ACPI S5
  2013-08-14 17:30         ` Andrew Cooper
@ 2013-08-14 18:40           ` Atom2
  2013-08-14 19:10             ` Atom2
  0 siblings, 1 reply; 28+ messages in thread
From: Atom2 @ 2013-08-14 18:40 UTC (permalink / raw)
  To: Andrew Cooper; +Cc: xen-devel, Ian Campbell, Jan Beulich

[-- Attachment #1: Type: text/plain, Size: 1394 bytes --]

Hi Andrew

Am 14.08.13 19:30, schrieb Andrew Cooper:
[...]
> Do you have the boot time dmesg output?
Sure, please see the attached log file. Please note that this dmesg data 
comes from a system that I have booted from the standard XEN 4.2.2 
kernel (i.e. w/o the WARN() statement and the various printk statements 
I have added; furthermore this was w/o the serial console being active)

>
> The problem here is that a queued_invalidate wait descriptor has been
> issued, and has not been completed within 1 second.  In all previous
> cases I have debugged, this is actually because we already turned off
> the IOMMU, then tried to turn it off again.
>
> Could you perhaps try with this patch as well?
>
> diff --git a/xen/drivers/passthrough/vtd/iommu.c
> b/xen/drivers/passthrough/vtd/iommu.c
> index 071a91b..45fff48 100644
> --- a/xen/drivers/passthrough/vtd/iommu.c
> +++ b/xen/drivers/passthrough/vtd/iommu.c
> @@ -791,6 +791,9 @@ static void iommu_disable_translation(struct iommu
> *iommu)
>       u32 sts;
>       unsigned long flags;
>
> +    printk("**Debug: Disabling translation for iommu %"PRId32"\n",
> iommu->index);
> +    WARN();
> +
>       /* apply platform specific errata workarounds */
>       vtd_ops_preamble_quirk(iommu);
I'll get back to you in due course with the log output after I have 
applied the patch and restarted & then shutdown XEN again.

>
>
> ~Andrew
>

[-- Attachment #2: XEN dmesg.txt --]
[-- Type: text/plain, Size: 6502 bytes --]

 __  __            _  _    ____    ____
 \ \/ /___ _ __   | || |  |___ \  |___ \
  \  // _ \ '_ \  | || |_   __) |   __) |
  /  \  __/ | | | |__   _| / __/ _ / __/
 /_/\_\___|_| |_|    |_|(_)_____(_)_____|

(XEN) Xen version 4.2.2 (@herrenhauspark.com) (gcc (Gentoo Hardened 4.6.3 p1.13, pie-0.5.2) 4.6.3) Fri Jul  5 12:09:53 CEST 2013
(XEN) Latest ChangeSet: unavailable
(XEN) Bootloader: GNU GRUB 0.97
(XEN) Command line: vga=gfx-1024x768x32 dom0_mem=512M,max:512M dom0_max_vcpus=1 dom0_vcpus_pin iommu=1 i915.modeset=1
(XEN) Video information:
(XEN)  VGA is graphics mode 1024x768, 32 bpp
(XEN)  VBE/DDC methods: V2; EDID transfer time: 1 seconds
(XEN) Disc information:
(XEN)  Found 2 MBR signatures
(XEN)  Found 2 EDD information structures
(XEN) Xen-e820 RAM map:
(XEN)  0000000000000000 - 0000000000099c00 (usable)
(XEN)  0000000000099c00 - 00000000000a0000 (reserved)
(XEN)  00000000000e0000 - 0000000000100000 (reserved)
(XEN)  0000000000100000 - 0000000020000000 (usable)
(XEN)  0000000020000000 - 0000000020200000 (reserved)
(XEN)  0000000020200000 - 0000000040000000 (usable)
(XEN)  0000000040000000 - 0000000040200000 (reserved)
(XEN)  0000000040200000 - 00000000db9f1000 (usable)
(XEN)  00000000db9f1000 - 00000000dc0db000 (reserved)
(XEN)  00000000dc0db000 - 00000000dc1fa000 (ACPI NVS)
(XEN)  00000000dc1fa000 - 00000000dc652000 (reserved)
(XEN)  00000000dc652000 - 00000000dc653000 (usable)
(XEN)  00000000dc653000 - 00000000dc696000 (ACPI NVS)
(XEN)  00000000dc696000 - 00000000dcdba000 (usable)
(XEN)  00000000dcdba000 - 00000000dcff2000 (reserved)
(XEN)  00000000dcff2000 - 00000000dd000000 (usable)
(XEN)  00000000dd800000 - 00000000dfa00000 (reserved)
(XEN)  00000000f8000000 - 00000000fc000000 (reserved)
(XEN)  00000000fec00000 - 00000000fec01000 (reserved)
(XEN)  00000000fed00000 - 00000000fed04000 (reserved)
(XEN)  00000000fed1c000 - 00000000fed20000 (reserved)
(XEN)  00000000fee00000 - 00000000fee01000 (reserved)
(XEN)  00000000ff000000 - 0000000100000000 (reserved)
(XEN)  0000000100000000 - 000000081e600000 (usable)
(XEN) ACPI: RSDP 000F0490, 0024 (r2 ALASKA)
(XEN) ACPI: XSDT DC1EA078, 0074 (r1 ALASKA    A M I  1072009 AMI     10013)
(XEN) ACPI: FACP DC1F4710, 00F4 (r4 ALASKA    A M I  1072009 AMI     10013)
(XEN) ACPI: DSDT DC1EA188, A587 (r2 ALASKA    A M I        1 INTL 20051117)
(XEN) ACPI: FACS DC1F8F80, 0040
(XEN) ACPI: APIC DC1F4808, 0092 (r3 ALASKA    A M I  1072009 AMI     10013)
(XEN) ACPI: FPDT DC1F48A0, 0044 (r1 ALASKA    A M I  1072009 AMI     10013)
(XEN) ACPI: MCFG DC1F48E8, 003C (r1 ALASKA    A M I  1072009 MSFT       97)
(XEN) ACPI: HPET DC1F4928, 0038 (r1 ALASKA    A M I  1072009 AMI.        5)
(XEN) ACPI: SSDT DC1F4960, 036D (r1 SataRe SataTabl     1000 INTL 20091112)
(XEN) ACPI: SSDT DC1F4CD0, 081E (r1  PmRef  Cpu0Ist     3000 INTL 20051117)
(XEN) ACPI: SSDT DC1F54F0, 0A92 (r1  PmRef    CpuPm     3000 INTL 20051117)
(XEN) ACPI: DMAR DC1F5F88, 00B0 (r1 INTEL      SNB         1 INTL        1)
(XEN) ACPI: ASF! DC1F6038, 00A5 (r32 INTEL       HCG        1 TFSM    F4240)
(XEN) System RAM: 32674MB (33458932kB)
(XEN) Domain heap initialised
(XEN) ACPI: 32/64X FACS address mismatch in FADT - dc1f8f80/0000000000000000, using 32
(XEN) Processor #0 6:10 APIC version 21
(XEN) Processor #2 6:10 APIC version 21
(XEN) Processor #4 6:10 APIC version 21
(XEN) Processor #6 6:10 APIC version 21
(XEN) Processor #1 6:10 APIC version 21
(XEN) Processor #3 6:10 APIC version 21
(XEN) Processor #5 6:10 APIC version 21
(XEN) Processor #7 6:10 APIC version 21
(XEN) IOAPIC[0]: apic_id 2, version 32, address 0xfec00000, GSI 0-23
(XEN) Enabling APIC mode:  Flat.  Using 1 I/O APICs
(XEN) Switched to APIC driver x2apic_cluster.
(XEN) Using scheduler: SMP Credit Scheduler (credit)
(XEN) Detected 2394.607 MHz processor.
(XEN) Initing memory sharing.
(XEN) xstate_init: using cntxt_size: 0x340 and states: 0x7
(XEN) Intel VT-d iommu 0 supported page sizes: 4kB.
(XEN) Intel VT-d iommu 1 supported page sizes: 4kB.
(XEN) Intel VT-d Snoop Control not enabled.
(XEN) Intel VT-d Dom0 DMA Passthrough not enabled.
(XEN) Intel VT-d Queued Invalidation enabled.
(XEN) Intel VT-d Interrupt Remapping enabled.
(XEN) Intel VT-d Shared EPT tables not enabled.
(XEN) I/O virtualisation enabled
(XEN)  - Dom0 mode: Relaxed
(XEN) Interrupt remapping enabled
(XEN) Enabled directed EOI with ioapic_ack_old on!
(XEN) ENABLING IO-APIC IRQs
(XEN)  -> Using old ACK method
(XEN) Platform timer is 14.318MHz HPET
(XEN) Allocated console ring of 16 KiB.
(XEN) VMX: Supported advanced features:
(XEN)  - APIC MMIO access virtualisation
(XEN)  - APIC TPR shadow
(XEN)  - Extended Page Tables (EPT)
(XEN)  - Virtual-Processor Identifiers (VPID)
(XEN)  - Virtual NMI
(XEN)  - MSR direct-access bitmap
(XEN)  - Unrestricted Guest
(XEN) HVM: ASIDs enabled.
(XEN) HVM: VMX enabled
(XEN) HVM: Hardware Assisted Paging (HAP) detected
(XEN) HVM: HAP page sizes: 4kB, 2MB
(XEN) Brought up 8 CPUs
(XEN) *** LOADING DOMAIN 0 ***
(XEN)  Xen  kernel: 64-bit, lsb, compat32
(XEN)  Dom0 kernel: 64-bit, PAE, lsb, paddr 0x1000000 -> 0x1c00000
(XEN) PHYSICAL MEMORY ARRANGEMENT:
(XEN)  Dom0 alloc.:   0000000804000000->0000000806000000 (122880 pages to be allocated)
(XEN) VIRTUAL MEMORY ARRANGEMENT:
(XEN)  Loaded kernel: ffffffff81000000->ffffffff81c00000
(XEN)  Init. ramdisk: ffffffff81c00000->ffffffff81c00000
(XEN)  Phys-Mach map: ffffffff81c00000->ffffffff81d00000
(XEN)  Start info:    ffffffff81d00000->ffffffff81d004b4
(XEN)  Page tables:   ffffffff81d01000->ffffffff81d14000
(XEN)  Boot stack:    ffffffff81d14000->ffffffff81d15000
(XEN)  TOTAL:         ffffffff80000000->ffffffff82000000
(XEN)  ENTRY ADDRESS: ffffffff815df1e0
(XEN) Dom0 has maximum 1 VCPUs
(XEN) Scrubbing Free RAM: .............................................................................................................................................................................................................................................................................................................................done.
(XEN) Initial low memory virq threshold set at 0x4000 pages.
(XEN) Std. Loglevel: Errors and warnings
(XEN) Guest Loglevel: Nothing (Rate-limited: Errors and warnings)
(XEN) Xen is relinquishing VGA console.
(XEN) *** Serial input -> DOM0 (type 'CTRL-a' three times to switch input to Xen)
(XEN) Freed 264kB init memory.
(XEN) traps.c:2600:d0 Domain attempted WRMSR 00000000000001fc from 0x000000000004005f to 0x000000000004005d.

[-- Attachment #3: Type: text/plain, Size: 126 bytes --]

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

^ permalink raw reply	[flat|nested] 28+ messages in thread

* Re: Powerdown problem on XEN | ACPI S5
  2013-08-14 18:40           ` Atom2
@ 2013-08-14 19:10             ` Atom2
  2013-08-14 19:18               ` Andrew Cooper
  0 siblings, 1 reply; 28+ messages in thread
From: Atom2 @ 2013-08-14 19:10 UTC (permalink / raw)
  To: Andrew Cooper; +Cc: xen-devel, Ian Campbell, Jan Beulich

[-- Attachment #1: Type: text/plain, Size: 2132 bytes --]

Hi again Andrew,
interestingly enough, the new WARN() did not trigger.

The output on the serial console only grossly lists the same information 
from the WARN() in qinval.c:223 we have already seen before.

In any case, please find the (this time: complete) new log attached to 
this mail.

BTW I have double-checked, whether I have copied the correct file to 
/boot by grepping for your "**Debug: ..." message and it was found in 
the (uncompressed) xen-test.gz file which is started from grub.

Many thanks again.

Am 14.08.13 20:40, schrieb Atom2:
> Hi Andrew
>
> Am 14.08.13 19:30, schrieb Andrew Cooper:
> [...]
>> Do you have the boot time dmesg output?
> Sure, please see the attached log file. Please note that this dmesg data
> comes from a system that I have booted from the standard XEN 4.2.2
> kernel (i.e. w/o the WARN() statement and the various printk statements
> I have added; furthermore this was w/o the serial console being active)
>
>>
>> The problem here is that a queued_invalidate wait descriptor has been
>> issued, and has not been completed within 1 second.  In all previous
>> cases I have debugged, this is actually because we already turned off
>> the IOMMU, then tried to turn it off again.
>>
>> Could you perhaps try with this patch as well?
>>
>> diff --git a/xen/drivers/passthrough/vtd/iommu.c
>> b/xen/drivers/passthrough/vtd/iommu.c
>> index 071a91b..45fff48 100644
>> --- a/xen/drivers/passthrough/vtd/iommu.c
>> +++ b/xen/drivers/passthrough/vtd/iommu.c
>> @@ -791,6 +791,9 @@ static void iommu_disable_translation(struct iommu
>> *iommu)
>>       u32 sts;
>>       unsigned long flags;
>>
>> +    printk("**Debug: Disabling translation for iommu %"PRId32"\n",
>> iommu->index);
>> +    WARN();
>> +
>>       /* apply platform specific errata workarounds */
>>       vtd_ops_preamble_quirk(iommu);
> I'll get back to you in due course with the log output after I have
> applied the patch and restarted & then shutdown XEN again.
>
>>
>>
>> ~Andrew
>>
>
>
> _______________________________________________
> Xen-devel mailing list
> Xen-devel@lists.xen.org
> http://lists.xen.org/xen-devel
>

[-- Attachment #2: XEN console new --]
[-- Type: text/plain, Size: 13069 bytes --]

 \ \/ /___ _ __   | || |  |___ \  |___ \ 
  \  // _ \ '_ \  | || |_   __) |   __) |
  /  \  __/ | | | |__   _| / __/ _ / __/ 
 /_/\_\___|_| |_|    |_|(_)_____(_)_____|
                                         
(XEN) Xen version 4.2.2 (root@herrenhauspark.com) (gcc (Gentoo Hardened 4.6.3 p1.13, pie-0.5.2) 4.6.3) Sat Aug 17 20:43:43 CEST 2013
(XEN) Latest ChangeSet: unavailable
(XEN) Bootloader: GNU GRUB 0.97
(XEN) Command line: console=com1 loglvl=all com1=115200,8n1,0x2d8,10 dom0_mem=512M,max:512M dom0_max_vcpus=1 dom0_vcpus_pin iommu=1
(XEN) Video information:
(XEN)  VGA is text mode 80x25, font 8x16
(XEN)  VBE/DDC methods: V2; EDID transfer time: 1 seconds
(XEN) Disc information:
(XEN)  Found 2 MBR signatures
(XEN)  Found 2 EDD information structures
(XEN) Xen-e820 RAM map:
(XEN)  0000000000000000 - 0000000000099c00 (usable)
(XEN)  0000000000099c00 - 00000000000a0000 (reserved)
(XEN)  00000000000e0000 - 0000000000100000 (reserved)
(XEN)  0000000000100000 - 0000000020000000 (usable)
(XEN)  0000000020000000 - 0000000020200000 (reserved)
(XEN)  0000000020200000 - 0000000040000000 (usable)
(XEN)  0000000040000000 - 0000000040200000 (reserved)
(XEN)  0000000040200000 - 00000000db9f1000 (usable)
(XEN)  00000000db9f1000 - 00000000dc0db000 (reserved)
(XEN)  00000000dc0db000 - 00000000dc1fa000 (ACPI NVS)
(XEN)  00000000dc1fa000 - 00000000dc652000 (reserved)
(XEN)  00000000dc652000 - 00000000dc653000 (usable)
(XEN)  00000000dc653000 - 00000000dc696000 (ACPI NVS)
(XEN)  00000000dc696000 - 00000000dcdba000 (usable)
(XEN)  00000000dcdba000 - 00000000dcff2000 (reserved)
(XEN)  00000000dcff2000 - 00000000dd000000 (usable)
(XEN)  00000000dd800000 - 00000000dfa00000 (reserved)
(XEN)  00000000f8000000 - 00000000fc000000 (reserved)
(XEN)  00000000fec00000 - 00000000fec01000 (reserved)
(XEN)  00000000fed00000 - 00000000fed04000 (reserved)
(XEN)  00000000fed1c000 - 00000000fed20000 (reserved)
(XEN)  00000000fee00000 - 00000000fee01000 (reserved)
(XEN)  00000000ff000000 - 0000000100000000 (reserved)
(XEN)  0000000100000000 - 000000081e600000 (usable)
(XEN) ACPI: RSDP 000F0490, 0024 (r2 ALASKA)
(XEN) ACPI: XSDT DC1EA078, 0074 (r1 ALASKA    A M I  1072009 AMI     10013)
(XEN) ACPI: FACP DC1F4710, 00F4 (r4 ALASKA    A M I  1072009 AMI     10013)
(XEN) ACPI: DSDT DC1EA188, A587 (r2 ALASKA    A M I        1 INTL 20051117)
(XEN) ACPI: FACS DC1F8F80, 0040
(XEN) ACPI: APIC DC1F4808, 0092 (r3 ALASKA    A M I  1072009 AMI     10013)
(XEN) ACPI: FPDT DC1F48A0, 0044 (r1 ALASKA    A M I  1072009 AMI     10013)
(XEN) ACPI: MCFG DC1F48E8, 003C (r1 ALASKA    A M I  1072009 MSFT       97)
(XEN) ACPI: HPET DC1F4928, 0038 (r1 ALASKA    A M I  1072009 AMI.        5)
(XEN) ACPI: SSDT DC1F4960, 036D (r1 SataRe SataTabl     1000 INTL 20091112)
(XEN) ACPI: SSDT DC1F4CD0, 081E (r1  PmRef  Cpu0Ist     3000 INTL 20051117)
(XEN) ACPI: SSDT DC1F54F0, 0A92 (r1  PmRef    CpuPm     3000 INTL 20051117)
(XEN) ACPI: DMAR DC1F5F88, 00B0 (r1 INTEL      SNB         1 INTL        1)
(XEN) ACPI: ASF! DC1F6038, 00A5 (r32 INTEL       HCG        1 TFSM    F4240)
(XEN) System RAM: 32674MB (33458932kB)
(XEN) No NUMA configuration found
(XEN) Faking a node at 0000000000000000-000000081e600000
(XEN) Domain heap initialised
(XEN) found SMP MP-table at 000fd8d0
(XEN) DMI 2.7 present.
(XEN) Using APIC driver default
(XEN) ACPI: PM-Timer IO Port: 0x408
(XEN) ACPI: ACPI SLEEP INFO: pm1x_cnt[404,0], pm1x_evt[400,0]
(XEN) ACPI: 32/64X FACS address mismatch in FADT - dc1f8f80/0000000000000000, using 32
(XEN) ACPI:                  wakeup_vec[dc1f8f8c], vec_size[20]
(XEN) ACPI: Local APIC address 0xfee00000
(XEN) ACPI: LAPIC (acpi_id[0x01] lapic_id[0x00] enabled)
(XEN) Processor #0 6:10 APIC version 21
(XEN) ACPI: LAPIC (acpi_id[0x02] lapic_id[0x02] enabled)
(XEN) Processor #2 6:10 APIC version 21
(XEN) ACPI: LAPIC (acpi_id[0x03] lapic_id[0x04] enabled)
(XEN) Processor #4 6:10 APIC version 21
(XEN) ACPI: LAPIC (acpi_id[0x04] lapic_id[0x06] enabled)
(XEN) Processor #6 6:10 APIC version 21
(XEN) ACPI: LAPIC (acpi_id[0x05] lapic_id[0x01] enabled)
(XEN) Processor #1 6:10 APIC version 21
(XEN) ACPI: LAPIC (acpi_id[0x06] lapic_id[0x03] enabled)
(XEN) Processor #3 6:10 APIC version 21
(XEN) ACPI: LAPIC (acpi_id[0x07] lapic_id[0x05] enabled)
(XEN) Processor #5 6:10 APIC version 21
(XEN) ACPI: LAPIC (acpi_id[0x08] lapic_id[0x07] enabled)
(XEN) Processor #7 6:10 APIC version 21
(XEN) ACPI: LAPIC_NMI (acpi_id[0xff] high edge lint[0x1])
(XEN) ACPI: IOAPIC (id[0x02] address[0xfec00000] gsi_base[0])
(XEN) IOAPIC[0]: apic_id 2, version 32, address 0xfec00000, GSI 0-23
(XEN) ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl)
(XEN) ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level)
(XEN) ACPI: IRQ0 used by override.
(XEN) ACPI: IRQ2 used by override.
(XEN) ACPI: IRQ9 used by override.
(XEN) Enabling APIC mode:  Flat.  Using 1 I/O APICs
(XEN) ACPI: HPET id: 0x8086a701 base: 0xfed00000
(XEN) ERST table was not found
(XEN) Using ACPI (MADT) for SMP configuration information
(XEN) SMP: Allowing 8 CPUs (0 hotplug CPUs)
(XEN) IRQ limits: 24 GSI, 1528 MSI/MSI-X
(XEN) Switched to APIC driver x2apic_cluster.
(XEN) Using scheduler: SMP Credit Scheduler (credit)
(XEN) Detected 2394.598 MHz processor.
(XEN) Initing memory sharing.
(XEN) xstate_init: using cntxt_size: 0x340 and states: 0x7
(XEN) mce_intel.c:1238: MCA Capability: BCAST 1 SER 0 CMCI 1 firstbank 0 extended MCE MSR 0
(XEN) Intel machine check reporting enabled
(XEN) PCI: MCFG configuration 0: base f8000000 segment 0000 buses 00 - 3f
(XEN) PCI: MCFG area at f8000000 reserved in E820
(XEN) PCI: Using MCFG for segment 0000 bus 00-3f
(XEN) Intel VT-d iommu 0 supported page sizes: 4kB.
(XEN) Intel VT-d iommu 1 supported page sizes: 4kB.
(XEN) Intel VT-d Snoop Control not enabled.
(XEN) Intel VT-d Dom0 DMA Passthrough not enabled.
(XEN) Intel VT-d Queued Invalidation enabled.
(XEN) Intel VT-d Interrupt Remapping enabled.
(XEN) Intel VT-d Shared EPT tables not enabled.
(XEN) I/O virtualisation enabled
(XEN)  - Dom0 mode: Relaxed
(XEN) Interrupt remapping enabled
(XEN) Enabled directed EOI with ioapic_ack_old on!
(XEN) ENABLING IO-APIC IRQs
(XEN)  -> Using old ACK method
(XEN) ..TIMER: vector=0xF0 apic1=0 pin1=2 apic2=-1 pin2=-1
(XEN) TSC deadline timer enabled
(XEN) Platform timer is 14.318MHz HPET
(XEN) Allocated console ring of 64 KiB.
(XEN) VMX: Supported advanced features:
(XEN)  - APIC MMIO access virtualisation
(XEN)  - APIC TPR shadow
(XEN)  - Extended Page Tables (EPT)
(XEN)  - Virtual-Processor Identifiers (VPID)
(XEN)  - Virtual NMI
(XEN)  - MSR direct-access bitmap
(XEN)  - Unrestricted Guest
(XEN) HVM: ASIDs enabled.
(XEN) HVM: VMX enabled
(XEN) HVM: Hardware Assisted Paging (HAP) detected
(XEN) HVM: HAP page sizes: 4kB, 2MB
(XEN) Brought up 8 CPUs
(XEN) ACPI sleep modes: S3
(XEN) mcheck_poll: Machine check polling timer started.
(XEN) *** LOADING DOMAIN 0 ***
(XEN)  Xen  kernel: 64-bit, lsb, compat32
(XEN)  Dom0 kernel: 64-bit, PAE, lsb, paddr 0x1000000 -> 0x1c00000
(XEN) PHYSICAL MEMORY ARRANGEMENT:
(XEN)  Dom0 alloc.:   0000000804000000->0000000806000000 (122880 pages to be allocated)
(XEN) VIRTUAL MEMORY ARRANGEMENT:
(XEN)  Loaded kernel: ffffffff81000000->ffffffff81c00000
(XEN)  Init. ramdisk: ffffffff81c00000->ffffffff81c00000
(XEN)  Phys-Mach map: ffffffff81c00000->ffffffff81d00000
(XEN)  Start info:    ffffffff81d00000->ffffffff81d004b4
(XEN)  Page tables:   ffffffff81d01000->ffffffff81d14000
(XEN)  Boot stack:    ffffffff81d14000->ffffffff81d15000
(XEN)  TOTAL:         ffffffff80000000->ffffffff82000000
(XEN)  ENTRY ADDRESS: ffffffff815df1e0
(XEN) Dom0 has maximum 1 VCPUs
(XEN) Scrubbing Free RAM: .............................................................................................................................................................................................................................................................................................................................done.
(XEN) Initial low memory virq threshold set at 0x4000 pages.
(XEN) Std. Loglevel: All
(XEN) Guest Loglevel: Nothing (Rate-limited: Errors and warnings)
(XEN) *** Serial input -> DOM0 (type 'CTRL-a' three times to switch input to Xen)
(XEN) Freed 268kB init memory.
(XEN) PCI add device 0000:00:00.0
(XEN) PCI add device 0000:00:01.0
(XEN) PCI add device 0000:00:01.1
(XEN) PCI add device 0000:00:02.0
(XEN) PCI add device 0000:00:06.0
(XEN) PCI add device 0000:00:16.0
(XEN) PCI add device 0000:00:16.3
(XEN) PCI add device 0000:00:19.0
(XEN) PCI add device 0000:00:1a.0
(XEN) PCI add device 0000:00:1b.0
(XEN) PCI add device 0000:00:1c.0
(XEN) PCI add device 0000:00:1c.4
(XEN) PCI add device 0000:00:1c.6
(XEN) PCI add device 0000:00:1d.0
(XEN) PCI add device 0000:00:1e.0
(XEN) PCI add device 0000:00:1f.0
(XEN) PCI add device 0000:00:1f.2
(XEN) PCI add device 0000:00:1f.3
(XEN) PCI add device 0000:02:00.0
(XEN) PCI add device 0000:03:00.0
(XEN) PCI add device 0000:04:00.0
(XEN) PCI add device 0000:05:00.0
(XEN) PCI add device 0000:06:00.0
(XEN) PCI add device 0000:07:00.0
(XEN) PCI add device 0000:08:00.0
(XEN) PCI add device 0000:09:00.0
(XEN) PCI add device 0000:09:02.0
(XEN) PCI add device 0000:0a:08.0
(XEN) PCI add device 0000:0a:09.0
(XEN) PCI add device 0000:0a:0a.0
(XEN) PCI add device 0000:0a:0b.0
(XEN) traps.c:2584:d0 Domain attempted WRMSR 00000000000001fc from 0x000000000004005f to 0x000000000004005d.
(XEN) Monitor-Mwait will be used to enter C1 state
(XEN) Monitor-Mwait will be used to enter C2 state
(XEN) Monitor-Mwait will be used to enter C3 state
(XEN) Preparing system for ACPI S5 state.
(XEN) Disabling non-boot CPUs ...
(XEN) Entering ACPI S5 state.
(XEN) After local_irq_save
(XEN) After spin_debug_disable
(XEN) After time_suspend
(XEN) After li8259_suspend
(XEN) After ioapic_suspend
(XEN) DMAR_IQA_REG = 80d87c002
(XEN) DMAR_IQH_REG = 120
(XEN) DMAR_IQT_REG = 140
(XEN) Xen WARN at qinval.c:223
(XEN) ----[ Xen-4.2.2  x86_64  debug=n  Not tainted ]----
(XEN) CPU:    0
(XEN) RIP:    e008:[<ffff82c48014545c>] invalidate_sync+0x22c/0x280
(XEN) RFLAGS: 0000000000010082   CONTEXT: hypervisor
(XEN) rax: ffff82c4802f14a0   rbx: ffff83080d8855b0   rcx: 0000000000000000
(XEN) rdx: ffff82c4802a7f18   rsi: 000000000000000a   rdi: ffff82c48025a640
(XEN) rbp: 0000000ae1ec1671   rsp: ffff82c4802a7d20   r8:  0000000000000004
(XEN) r9:  0000000000000002   r10: 0000000000000003   r11: 0000000000000400
(XEN) r12: ffff83080d8855e4   r13: ffff83080d88562c   r14: 0000000000000002
(XEN) r15: 0000000000000002   cr0: 000000008005003b   cr4: 00000000000426f0
(XEN) cr3: 00000000db6b5000   cr2: ffff88001a89ea48
(XEN) ds: 0000   es: 0000   fs: 0000   gs: 0000   ss: e010   cs: e008
(XEN) Xen stack trace from rsp=ffff82c4802a7d20:
(XEN)    0000000000000002 000000080d87c002 0000000000000082 000000000d87c002
(XEN)    0000000000000086 ffff83080d8855b0 0000000000000000 1000000000000000
(XEN)    0000000000000000 0000000000000000 0000000000000000 ffff82c480145566
(XEN)    0000000000000000 0000000000000000 0000000800000004 ffff83080d8855b0
(XEN)    ffff83080d885530 ffff83080d885640 ffff82c48024bc60 0000000000000000
(XEN)    ffff82c4802f14e0 ffff82c48014140e 0000000000000000 0000000000000086
(XEN)    ffff82c4802f14a0 0000000000000005 0000000000000000 0000000000000000
(XEN)    ffff82c4802db740 ffff82c480141e25 0000000000000082 0000000000000086
(XEN)    ffff82c4802a7f18 ffff82c4802f14a0 0000000000000005 0000000000000000
(XEN)    0000000000000000 ffff82c4802db740 ffff82c4802f14e0 ffff82c48019a265
(XEN)    ffff82c4802a7f18 00000000ffffffff 0000000000000286 ffff8300dc697000
(XEN)    ffff8300dc697000 ffff830669301fc0 ffff8300dc697000 ffff82c4802f1488
(XEN)    ffff82c4802db740 ffff82c4802db740 ffff82c4802f14e0 ffff82c4801054ae
(XEN)    ffff8300dc697180 0000000000000000 ffff82c4802f1590 ffff82c480125670
(XEN)    ffff82c4802f1570 ffff82c480125878 ffff82c4802a7f18 ffff8300dc697000
(XEN)    00000000ffffffff ffff82c480156953 ffff8300dcda6000 0000000000000000
(XEN)    0000000000000003 0000000000000005 0000000000002003 ffffffff815adf58
(XEN)    0000000000000000 0000000000000246 0000000000000370 0000000000000005
(XEN)    0000000000000000 0000000000000000 ffffffff810010ea 0000000000002003
(XEN)    0000000000003c03 ffff88001a1f7d18 0000010000000000 ffffffff810010ea
(XEN) Xen call trace:
(XEN)    [<ffff82c48014545c>] invalidate_sync+0x22c/0x280
(XEN)    [<ffff82c480145566>] flush_iotlb_qi+0xb6/0x100
(XEN)    [<ffff82c48014140e>] iommu_flush_all+0x8e/0xf0
(XEN)    [<ffff82c480141e25>] vtd_suspend+0x55/0x110
(XEN)    [<ffff82c48019a265>] enter_state_helper+0x235/0x430
(XEN)    [<ffff82c4801054ae>] continue_hypercall_tasklet_handler+0xee/0x100
(XEN)    [<ffff82c480125670>] do_tasklet_work+0x60/0xc0
(XEN)    [<ffff82c480125878>] do_tasklet+0x78/0xb0
(XEN)    [<ffff82c480156953>] idle_loop+0x23/0x50
(XEN)    
(XEN) 
(XEN) ****************************************
(XEN) Panic on CPU 0:
(XEN) queue invalidate wait descriptor was not executed
(XEN) ****************************************
(XEN) 
(XEN) Reboot in five seconds...
(XEN) Resetting with ACPI MEMORY or I/O RESET_REG.

[-- Attachment #3: Type: text/plain, Size: 126 bytes --]

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

^ permalink raw reply	[flat|nested] 28+ messages in thread

* Re: Powerdown problem on XEN | ACPI S5
  2013-08-14 19:10             ` Atom2
@ 2013-08-14 19:18               ` Andrew Cooper
  2013-08-14 19:39                 ` Atom2
  0 siblings, 1 reply; 28+ messages in thread
From: Andrew Cooper @ 2013-08-14 19:18 UTC (permalink / raw)
  To: Atom2; +Cc: xen-devel, Ian Campbell, Jan Beulich

On 14/08/13 20:10, Atom2 wrote:
> Hi again Andrew,
> interestingly enough, the new WARN() did not trigger.
>
> The output on the serial console only grossly lists the same
> information from the WARN() in qinval.c:223 we have already seen before.
>
> In any case, please find the (this time: complete) new log attached to
> this mail.
>
> BTW I have double-checked, whether I have copied the correct file to
> /boot by grepping for your "**Debug: ..." message and it was found in
> the (uncompressed) xen-test.gz file which is started from grub.
>
> Many thanks again.

Hmm - curious.  That would indicate that the timeout occurs during the
very first attempt to shut down the iommu

Can you grab the full log with "iommu=1,debug,verbose" ?

Can you also please attach "lspci -vv" and "lspci -tv" from dom0

~Andrew

^ permalink raw reply	[flat|nested] 28+ messages in thread

* Re: Powerdown problem on XEN | ACPI S5
  2013-08-14 19:18               ` Andrew Cooper
@ 2013-08-14 19:39                 ` Atom2
  2013-08-14 20:18                   ` Andrew Cooper
  0 siblings, 1 reply; 28+ messages in thread
From: Atom2 @ 2013-08-14 19:39 UTC (permalink / raw)
  To: Andrew Cooper; +Cc: xen-devel, Ian Campbell, Jan Beulich

[-- Attachment #1: Type: text/plain, Size: 548 bytes --]

Hi Andrew,
all requested information is attached, please see notes below.

My ongoing & continued thanks anyway ...

Am 14.08.13 21:18, schrieb Andrew Cooper:
[...]
> Hmm - curious.  That would indicate that the timeout occurs during the
> very first attempt to shut down the iommu
>
> Can you grab the full log with "iommu=1,debug,verbose" ?
Sure, please see attachment "XEN console (debug, verbose)"
>
> Can you also please attach "lspci -vv" and "lspci -tv" from dom0
No problem - please see attachements "lspci-vv" and "lspci-tv"
>
> ~Andrew
>

[-- Attachment #2: XEN console (debug, verbose) --]
[-- Type: text/plain, Size: 15427 bytes --]

 \ \/ /___ _ __   | || |  |___ \  |___ \ 
  \  // _ \ '_ \  | || |_   __) |   __) |
  /  \  __/ | | | |__   _| / __/ _ / __/ 
 /_/\_\___|_| |_|    |_|(_)_____(_)_____|
                                         
(XEN) Xen version 4.2.2 (root@herrenhauspark.com) (gcc (Gentoo Hardened 4.6.3 p1.13, pie-0.5.2) 4.6.3) Sat Aug 17 20:43:43 CEST 2013
(XEN) Latest ChangeSet: unavailable
(XEN) Bootloader: GNU GRUB 0.97
(XEN) Command line: console=com1 loglvl=all com1=115200,8n1,0x2d8,10 dom0_mem=512M,max:512M dom0_max_vcpus=1 dom0_vcpus_pin iommu=1,debug,verbose
(XEN) Video information:
(XEN)  VGA is text mode 80x25, font 8x16
(XEN)  VBE/DDC methods: V2; EDID transfer time: 1 seconds
(XEN) Disc information:
(XEN)  Found 2 MBR signatures
(XEN)  Found 2 EDD information structures
(XEN) Xen-e820 RAM map:
(XEN)  0000000000000000 - 0000000000099c00 (usable)
(XEN)  0000000000099c00 - 00000000000a0000 (reserved)
(XEN)  00000000000e0000 - 0000000000100000 (reserved)
(XEN)  0000000000100000 - 0000000020000000 (usable)
(XEN)  0000000020000000 - 0000000020200000 (reserved)
(XEN)  0000000020200000 - 0000000040000000 (usable)
(XEN)  0000000040000000 - 0000000040200000 (reserved)
(XEN)  0000000040200000 - 00000000db9f1000 (usable)
(XEN)  00000000db9f1000 - 00000000dc0db000 (reserved)
(XEN)  00000000dc0db000 - 00000000dc1fa000 (ACPI NVS)
(XEN)  00000000dc1fa000 - 00000000dc652000 (reserved)
(XEN)  00000000dc652000 - 00000000dc653000 (usable)
(XEN)  00000000dc653000 - 00000000dc696000 (ACPI NVS)
(XEN)  00000000dc696000 - 00000000dcdba000 (usable)
(XEN)  00000000dcdba000 - 00000000dcff2000 (reserved)
(XEN)  00000000dcff2000 - 00000000dd000000 (usable)
(XEN)  00000000dd800000 - 00000000dfa00000 (reserved)
(XEN)  00000000f8000000 - 00000000fc000000 (reserved)
(XEN)  00000000fec00000 - 00000000fec01000 (reserved)
(XEN)  00000000fed00000 - 00000000fed04000 (reserved)
(XEN)  00000000fed1c000 - 00000000fed20000 (reserved)
(XEN)  00000000fee00000 - 00000000fee01000 (reserved)
(XEN)  00000000ff000000 - 0000000100000000 (reserved)
(XEN)  0000000100000000 - 000000081e600000 (usable)
(XEN) ACPI: RSDP 000F0490, 0024 (r2 ALASKA)
(XEN) ACPI: XSDT DC1EA078, 0074 (r1 ALASKA    A M I  1072009 AMI     10013)
(XEN) ACPI: FACP DC1F4710, 00F4 (r4 ALASKA    A M I  1072009 AMI     10013)
(XEN) ACPI: DSDT DC1EA188, A587 (r2 ALASKA    A M I        1 INTL 20051117)
(XEN) ACPI: FACS DC1F8F80, 0040
(XEN) ACPI: APIC DC1F4808, 0092 (r3 ALASKA    A M I  1072009 AMI     10013)
(XEN) ACPI: FPDT DC1F48A0, 0044 (r1 ALASKA    A M I  1072009 AMI     10013)
(XEN) ACPI: MCFG DC1F48E8, 003C (r1 ALASKA    A M I  1072009 MSFT       97)
(XEN) ACPI: HPET DC1F4928, 0038 (r1 ALASKA    A M I  1072009 AMI.        5)
(XEN) ACPI: SSDT DC1F4960, 036D (r1 SataRe SataTabl     1000 INTL 20091112)
(XEN) ACPI: SSDT DC1F4CD0, 081E (r1  PmRef  Cpu0Ist     3000 INTL 20051117)
(XEN) ACPI: SSDT DC1F54F0, 0A92 (r1  PmRef    CpuPm     3000 INTL 20051117)
(XEN) ACPI: DMAR DC1F5F88, 00B0 (r1 INTEL      SNB         1 INTL        1)
(XEN) ACPI: ASF! DC1F6038, 00A5 (r32 INTEL       HCG        1 TFSM    F4240)
(XEN) System RAM: 32674MB (33458932kB)
(XEN) No NUMA configuration found
(XEN) Faking a node at 0000000000000000-000000081e600000
(XEN) Domain heap initialised
(XEN) found SMP MP-table at 000fd8d0
(XEN) DMI 2.7 present.
(XEN) Using APIC driver default
(XEN) ACPI: PM-Timer IO Port: 0x408
(XEN) ACPI: ACPI SLEEP INFO: pm1x_cnt[404,0], pm1x_evt[400,0]
(XEN) ACPI: 32/64X FACS address mismatch in FADT - dc1f8f80/0000000000000000, using 32
(XEN) ACPI:                  wakeup_vec[dc1f8f8c], vec_size[20]
(XEN) ACPI: Local APIC address 0xfee00000
(XEN) ACPI: LAPIC (acpi_id[0x01] lapic_id[0x00] enabled)
(XEN) Processor #0 6:10 APIC version 21
(XEN) ACPI: LAPIC (acpi_id[0x02] lapic_id[0x02] enabled)
(XEN) Processor #2 6:10 APIC version 21
(XEN) ACPI: LAPIC (acpi_id[0x03] lapic_id[0x04] enabled)
(XEN) Processor #4 6:10 APIC version 21
(XEN) ACPI: LAPIC (acpi_id[0x04] lapic_id[0x06] enabled)
(XEN) Processor #6 6:10 APIC version 21
(XEN) ACPI: LAPIC (acpi_id[0x05] lapic_id[0x01] enabled)
(XEN) Processor #1 6:10 APIC version 21
(XEN) ACPI: LAPIC (acpi_id[0x06] lapic_id[0x03] enabled)
(XEN) Processor #3 6:10 APIC version 21
(XEN) ACPI: LAPIC (acpi_id[0x07] lapic_id[0x05] enabled)
(XEN) Processor #5 6:10 APIC version 21
(XEN) ACPI: LAPIC (acpi_id[0x08] lapic_id[0x07] enabled)
(XEN) Processor #7 6:10 APIC version 21
(XEN) ACPI: LAPIC_NMI (acpi_id[0xff] high edge lint[0x1])
(XEN) ACPI: IOAPIC (id[0x02] address[0xfec00000] gsi_base[0])
(XEN) IOAPIC[0]: apic_id 2, version 32, address 0xfec00000, GSI 0-23
(XEN) ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl)
(XEN) ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level)
(XEN) ACPI: IRQ0 used by override.
(XEN) ACPI: IRQ2 used by override.
(XEN) ACPI: IRQ9 used by override.
(XEN) Enabling APIC mode:  Flat.  Using 1 I/O APICs
(XEN) ACPI: HPET id: 0x8086a701 base: 0xfed00000
(XEN) [VT-D]dmar.c:720: Host address width 36
(XEN) [VT-D]dmar.c:734: found ACPI_DMAR_DRHD:
(XEN) [VT-D]dmar.c:412:   dmaru->address = fed90000
(XEN) [VT-D]iommu.c:1200: drhd->address = fed90000 iommu->reg = ffff82c3ffd57000
(XEN) [VT-D]iommu.c:1202: cap = c0000020e60262 ecap = f0101a
(XEN) [VT-D]dmar.c:338:  endpoint: 0000:00:02.0
(XEN) [VT-D]dmar.c:734: found ACPI_DMAR_DRHD:
(XEN) [VT-D]dmar.c:412:   dmaru->address = fed91000
(XEN) [VT-D]iommu.c:1200: drhd->address = fed91000 iommu->reg = ffff82c3ffd56000
(XEN) [VT-D]iommu.c:1202: cap = c9008020660262 ecap = f010da
(XEN) [VT-D]dmar.c:354:  IOAPIC: 0000:f0:1f.0
(XEN) [VT-D]dmar.c:332:  MSI HPET: 0000:f0:0f.0
(XEN) [VT-D]dmar.c:426:   flags: INCLUDE_ALL
(XEN) [VT-D]dmar.c:739: found ACPI_DMAR_RMRR:
(XEN) [VT-D]dmar.c:338:  endpoint: 0000:00:1d.0
(XEN) [VT-D]dmar.c:338:  endpoint: 0000:00:1a.0
(XEN) [VT-D]dmar.c:608:   RMRR region: base_addr dc072000 end_address dc07efff
(XEN) [VT-D]dmar.c:739: found ACPI_DMAR_RMRR:
(XEN) [VT-D]dmar.c:338:  endpoint: 0000:00:02.0
(XEN) [VT-D]dmar.c:608:   RMRR region: base_addr dd800000 end_address df9fffff
(XEN) ERST table was not found
(XEN) Using ACPI (MADT) for SMP configuration information
(XEN) SMP: Allowing 8 CPUs (0 hotplug CPUs)
(XEN) IRQ limits: 24 GSI, 1528 MSI/MSI-X
(XEN) Switched to APIC driver x2apic_cluster.
(XEN) Using scheduler: SMP Credit Scheduler (credit)
(XEN) Detected 2394.598 MHz processor.
(XEN) Initing memory sharing.
(XEN) xstate_init: using cntxt_size: 0x340 and states: 0x7
(XEN) mce_intel.c:1238: MCA Capability: BCAST 1 SER 0 CMCI 1 firstbank 0 extended MCE MSR 0
(XEN) Intel machine check reporting enabled
(XEN) PCI: MCFG configuration 0: base f8000000 segment 0000 buses 00 - 3f
(XEN) PCI: MCFG area at f8000000 reserved in E820
(XEN) PCI: Using MCFG for segment 0000 bus 00-3f
(XEN) Intel VT-d iommu 0 supported page sizes: 4kB.
(XEN) Intel VT-d iommu 1 supported page sizes: 4kB.
(XEN) Intel VT-d Snoop Control not enabled.
(XEN) Intel VT-d Dom0 DMA Passthrough not enabled.
(XEN) Intel VT-d Queued Invalidation enabled.
(XEN) Intel VT-d Interrupt Remapping enabled.
(XEN) Intel VT-d Shared EPT tables not enabled.
(XEN) I/O virtualisation enabled
(XEN)  - Dom0 mode: Relaxed
(XEN) Interrupt remapping enabled
(XEN) Enabled directed EOI with ioapic_ack_old on!
(XEN) ENABLING IO-APIC IRQs
(XEN)  -> Using old ACK method
(XEN) ..TIMER: vector=0xF0 apic1=0 pin1=2 apic2=-1 pin2=-1
(XEN) TSC deadline timer enabled
(XEN) Platform timer is 14.318MHz HPET
(XEN) Allocated console ring of 64 KiB.
(XEN) VMX: Supported advanced features:
(XEN)  - APIC MMIO access virtualisation
(XEN)  - APIC TPR shadow
(XEN)  - Extended Page Tables (EPT)
(XEN)  - Virtual-Processor Identifiers (VPID)
(XEN)  - Virtual NMI
(XEN)  - MSR direct-access bitmap
(XEN)  - Unrestricted Guest
(XEN) HVM: ASIDs enabled.
(XEN) HVM: VMX enabled
(XEN) HVM: Hardware Assisted Paging (HAP) detected
(XEN) HVM: HAP page sizes: 4kB, 2MB
(XEN) Brought up 8 CPUs
(XEN) ACPI sleep modes: S3
(XEN) mcheck_poll: Machine check polling timer started.
(XEN) *** LOADING DOMAIN 0 ***
(XEN)  Xen  kernel: 64-bit, lsb, compat32
(XEN)  Dom0 kernel: 64-bit, PAE, lsb, paddr 0x1000000 -> 0x1c00000
(XEN) PHYSICAL MEMORY ARRANGEMENT:
(XEN)  Dom0 alloc.:   0000000804000000->0000000806000000 (122880 pages to be allocated)
(XEN) VIRTUAL MEMORY ARRANGEMENT:
(XEN)  Loaded kernel: ffffffff81000000->ffffffff81c00000
(XEN)  Init. ramdisk: ffffffff81c00000->ffffffff81c00000
(XEN)  Phys-Mach map: ffffffff81c00000->ffffffff81d00000
(XEN)  Start info:    ffffffff81d00000->ffffffff81d004b4
(XEN)  Page tables:   ffffffff81d01000->ffffffff81d14000
(XEN)  Boot stack:    ffffffff81d14000->ffffffff81d15000
(XEN)  TOTAL:         ffffffff80000000->ffffffff82000000
(XEN)  ENTRY ADDRESS: ffffffff815df1e0
(XEN) Dom0 has maximum 1 VCPUs
(XEN) [VT-D]iommu.c:1497: d0:PCI: map 0000:00:00.0
(XEN) [VT-D]iommu.c:1497: d0:PCI: map 0000:00:02.0
(XEN) [VT-D]iommu.c:1497: d0:PCI: map 0000:00:16.0
(XEN) [VT-D]iommu.c:1497: d0:PCI: map 0000:00:16.3
(XEN) [VT-D]iommu.c:1497: d0:PCI: map 0000:00:19.0
(XEN) [VT-D]iommu.c:1497: d0:PCI: map 0000:00:1a.0
(XEN) [VT-D]iommu.c:1486: d0:PCIe: map 0000:00:1b.0
(XEN) [VT-D]iommu.c:1497: d0:PCI: map 0000:00:1d.0
(XEN) [VT-D]iommu.c:1497: d0:PCI: map 0000:00:1f.0
(XEN) [VT-D]iommu.c:1497: d0:PCI: map 0000:00:1f.2
(XEN) [VT-D]iommu.c:1497: d0:PCI: map 0000:00:1f.3
(XEN) [VT-D]iommu.c:1497: d0:PCI: map 0000:03:00.0
(XEN) [VT-D]iommu.c:1486: d0:PCIe: map 0000:04:00.0
(XEN) [VT-D]iommu.c:1497: d0:PCI: map 0000:06:00.0
(XEN) [VT-D]iommu.c:1486: d0:PCIe: map 0000:07:00.0
(XEN) [VT-D]iommu.c:1486: d0:PCIe: map 0000:08:00.0
(XEN) [VT-D]iommu.c:1497: d0:PCI: map 0000:09:02.0
(XEN) [VT-D]iommu.c:1497: d0:PCI: map 0000:0a:08.0
(XEN) [VT-D]iommu.c:1497: d0:PCI: map 0000:0a:09.0
(XEN) [VT-D]iommu.c:1497: d0:PCI: map 0000:0a:0a.0
(XEN) [VT-D]iommu.c:1497: d0:PCI: map 0000:0a:0b.0
(XEN) [VT-D]iommu.c:772: iommu_enable_translation: iommu->reg = ffff82c3ffd57000
(XEN) [VT-D]iommu.c:772: iommu_enable_translation: iommu->reg = ffff82c3ffd56000
(XEN) Scrubbing Free RAM: .............................................................................................................................................................................................................................................................................................................................done.
(XEN) Initial low memory virq threshold set at 0x4000 pages.
(XEN) Std. Loglevel: All
(XEN) Guest Loglevel: Nothing (Rate-limited: Errors and warnings)
(XEN) *** Serial input -> DOM0 (type 'CTRL-a' three times to switch input to Xen)
(XEN) Freed 268kB init memory.
(XEN) PCI add device 0000:00:00.0
(XEN) PCI add device 0000:00:01.0
(XEN) PCI add device 0000:00:01.1
(XEN) PCI add device 0000:00:02.0
(XEN) PCI add device 0000:00:06.0
(XEN) PCI add device 0000:00:16.0
(XEN) PCI add device 0000:00:16.3
(XEN) PCI add device 0000:00:19.0
(XEN) PCI add device 0000:00:1a.0
(XEN) PCI add device 0000:00:1b.0
(XEN) PCI add device 0000:00:1c.0
(XEN) PCI add device 0000:00:1c.4
(XEN) PCI add device 0000:00:1c.6
(XEN) PCI add device 0000:00:1d.0
(XEN) PCI add device 0000:00:1e.0
(XEN) PCI add device 0000:00:1f.0
(XEN) PCI add device 0000:00:1f.2
(XEN) PCI add device 0000:00:1f.3
(XEN) PCI add device 0000:02:00.0
(XEN) PCI add device 0000:03:00.0
(XEN) PCI add device 0000:04:00.0
(XEN) PCI add device 0000:05:00.0
(XEN) PCI add device 0000:06:00.0
(XEN) PCI add device 0000:07:00.0
(XEN) PCI add device 0000:08:00.0
(XEN) PCI add device 0000:09:00.0
(XEN) PCI add device 0000:09:02.0
(XEN) PCI add device 0000:0a:08.0
(XEN) PCI add device 0000:0a:09.0
(XEN) PCI add device 0000:0a:0a.0
(XEN) PCI add device 0000:0a:0b.0
(XEN) traps.c:2584:d0 Domain attempted WRMSR 00000000000001fc from 0x000000000004005f to 0x000000000004005d.
(XEN) Monitor-Mwait will be used to enter C1 state
(XEN) Monitor-Mwait will be used to enter C2 state
(XEN) Monitor-Mwait will be used to enter C3 state
(XEN) Preparing system for ACPI S5 state.
(XEN) Disabling non-boot CPUs ...
(XEN) Entering ACPI S5 state.
(XEN) After local_irq_save
(XEN) After spin_debug_disable
(XEN) After time_suspend
(XEN) After li8259_suspend
(XEN) After ioapic_suspend
(XEN) DMAR_IQA_REG = 80d87c002
(XEN) DMAR_IQH_REG = 120
(XEN) DMAR_IQT_REG = 140
(XEN) Xen WARN at qinval.c:223
(XEN) ----[ Xen-4.2.2  x86_64  debug=n  Not tainted ]----
(XEN) CPU:    0
(XEN) RIP:    e008:[<ffff82c48014545c>] invalidate_sync+0x22c/0x280
(XEN) RFLAGS: 0000000000010082   CONTEXT: hypervisor
(XEN) rax: ffff82c4802f14a0   rbx: ffff83080d8855b0   rcx: 0000000000000000
(XEN) rdx: ffff82c4802a7f18   rsi: 000000000000000a   rdi: ffff82c48025a640
(XEN) rbp: 000000162908ac47   rsp: ffff82c4802a7d20   r8:  0000000000000004
(XEN) r9:  0000000000000002   r10: 0000000000000003   r11: 0000000000000400
(XEN) r12: ffff83080d8855e4   r13: ffff83080d88562c   r14: 0000000000000002
(XEN) r15: 0000000000000002   cr0: 000000008005003b   cr4: 00000000000426f0
(XEN) cr3: 00000000db6b5000   cr2: ffff88001a89ea48
(XEN) ds: 0000   es: 0000   fs: 0000   gs: 0000   ss: e010   cs: e008
(XEN) Xen stack trace from rsp=ffff82c4802a7d20:
(XEN)    0000000000000002 000000080d87c002 0000000000000082 000000000d87c002
(XEN)    0000000000000086 ffff83080d8855b0 0000000000000000 1000000000000000
(XEN)    0000000000000000 0000000000000000 0000000000000000 ffff82c480145566
(XEN)    0000000000000000 0000000000000000 0000000800000004 ffff83080d8855b0
(XEN)    ffff83080d885530 ffff83080d885640 ffff82c48024bc60 0000000000000000
(XEN)    ffff82c4802f14e0 ffff82c48014140e 0000000000000000 0000000000000086
(XEN)    ffff82c4802f14a0 0000000000000005 0000000000000000 0000000000000000
(XEN)    ffff82c4802db740 ffff82c480141e25 0000000000000082 0000000000000086
(XEN)    ffff82c4802a7f18 ffff82c4802f14a0 0000000000000005 0000000000000000
(XEN)    0000000000000000 ffff82c4802db740 ffff82c4802f14e0 ffff82c48019a265
(XEN)    ffff82c4802a7f18 00000000ffffffff 0000000000000286 ffff8300dc697000
(XEN)    ffff8300dc697000 ffff830669301fc0 ffff8300dc697000 ffff82c4802f1488
(XEN)    ffff82c4802db740 ffff82c4802db740 ffff82c4802f14e0 ffff82c4801054ae
(XEN)    ffff8300dc697180 0000000000000000 ffff82c4802f1590 ffff82c480125670
(XEN)    ffff82c4802f1570 ffff82c480125878 ffff82c4802a7f18 ffff8300dc697000
(XEN)    00000000ffffffff ffff82c480156953 ffff8300dcda6000 0000000000000000
(XEN)    0000000000000003 0000000000000005 0000000000002003 ffffffff815adf58
(XEN)    0000000000000000 0000000000000246 0000000000000372 0000000000000005
(XEN)    0000000000000000 0000000000000000 ffffffff810010ea 0000000000002003
(XEN)    0000000000003c03 ffff880017727d18 0000010000000000 ffffffff810010ea
(XEN) Xen call trace:
(XEN)    [<ffff82c48014545c>] invalidate_sync+0x22c/0x280
(XEN)    [<ffff82c480145566>] flush_iotlb_qi+0xb6/0x100
(XEN)    [<ffff82c48014140e>] iommu_flush_all+0x8e/0xf0
(XEN)    [<ffff82c480141e25>] vtd_suspend+0x55/0x110
(XEN)    [<ffff82c48019a265>] enter_state_helper+0x235/0x430
(XEN)    [<ffff82c4801054ae>] continue_hypercall_tasklet_handler+0xee/0x100
(XEN)    [<ffff82c480125670>] do_tasklet_work+0x60/0xc0
(XEN)    [<ffff82c480125878>] do_tasklet+0x78/0xb0
(XEN)    [<ffff82c480156953>] idle_loop+0x23/0x50
(XEN)    
(XEN) 
(XEN) ****************************************
(XEN) Panic on CPU 0:
(XEN) queue invalidate wait descriptor was not executed
(XEN) ****************************************
(XEN) 
(XEN) Reboot in five seconds...
(XEN) Resetting with ACPI MEMORY or I/O RESET_REG.

[-- Attachment #3: lspci-vv --]
[-- Type: text/plain, Size: 48096 bytes --]

00:00.0 Host bridge: Intel Corporation Xeon E3-1200 Processor Family DRAM Controller (rev 09)
	Subsystem: Intel Corporation Xeon E3-1200 Processor Family DRAM Controller
	Control: I/O- Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx-
	Status: Cap+ 66MHz- UDF- FastB2B+ ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort+ >SERR- <PERR- INTx-
	Latency: 0
	Capabilities: [e0] Vendor Specific Information: Len=0c <?>

00:01.0 PCI bridge: Intel Corporation Xeon E3-1200/2nd Generation Core Processor Family PCI Express Root Port (rev 09) (prog-if 00 [Normal decode])
	Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx+
	Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
	Latency: 0, Cache Line Size: 64 bytes
	Bus: primary=00, secondary=01, subordinate=01, sec-latency=0
	Secondary status: 66MHz- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- <SERR- <PERR-
	BridgeCtl: Parity- SERR- NoISA- VGA- MAbort- >Reset- FastB2B-
		PriDiscTmr- SecDiscTmr- DiscTmrStat- DiscTmrSERREn-
	Capabilities: [88] Subsystem: Intel Corporation Xeon E3-1200/2nd Generation Core Processor Family PCI Express Root Port
	Capabilities: [80] Power Management version 3
		Flags: PMEClk- DSI- D1- D2- AuxCurrent=0mA PME(D0+,D1-,D2-,D3hot+,D3cold+)
		Status: D0 NoSoftRst+ PME-Enable- DSel=0 DScale=0 PME-
	Capabilities: [90] MSI: Enable+ Count=1/1 Maskable- 64bit-
		Address: fee00298  Data: 0000
	Capabilities: [a0] Express (v2) Root Port (Slot+), MSI 00
		DevCap:	MaxPayload 128 bytes, PhantFunc 0, Latency L0s <64ns, L1 <1us
			ExtTag- RBE+ FLReset-
		DevCtl:	Report errors: Correctable- Non-Fatal- Fatal- Unsupported-
			RlxdOrd- ExtTag- PhantFunc- AuxPwr- NoSnoop-
			MaxPayload 128 bytes, MaxReadReq 128 bytes
		DevSta:	CorrErr- UncorrErr- FatalErr- UnsuppReq- AuxPwr- TransPend-
		LnkCap:	Port #2, Speed 5GT/s, Width x8, ASPM L0s L1, Latency L0 <1us, L1 <4us
			ClockPM- Surprise- LLActRep- BwNot+
		LnkCtl:	ASPM Disabled; RCB 64 bytes Disabled- Retrain- CommClk-
			ExtSynch- ClockPM- AutWidDis- BWInt- AutBWInt-
		LnkSta:	Speed 2.5GT/s, Width x0, TrErr- Train- SlotClk+ DLActive- BWMgmt- ABWMgmt-
		SltCap:	AttnBtn- PwrCtrl- MRL- AttnInd- PwrInd- HotPlug- Surprise-
			Slot #1, PowerLimit 75.000W; Interlock- NoCompl+
		SltCtl:	Enable: AttnBtn- PwrFlt- MRL- PresDet- CmdCplt- HPIrq- LinkChg-
			Control: AttnInd Unknown, PwrInd Unknown, Power- Interlock-
		SltSta:	Status: AttnBtn- PowerFlt- MRL- CmdCplt- PresDet- Interlock-
			Changed: MRL- PresDet- LinkState-
		RootCtl: ErrCorrectable- ErrNon-Fatal- ErrFatal- PMEIntEna- CRSVisible-
		RootCap: CRSVisible-
		RootSta: PME ReqID 0000, PMEStatus- PMEPending-
		DevCap2: Completion Timeout: Not Supported, TimeoutDis-, LTR+, OBFF Not Supported ARIFwd-
		DevCtl2: Completion Timeout: 50us to 50ms, TimeoutDis-, LTR-, OBFF Disabled ARIFwd-
		LnkCtl2: Target Link Speed: 5GT/s, EnterCompliance- SpeedDis-
			 Transmit Margin: Normal Operating Range, EnterModifiedCompliance- ComplianceSOS-
			 Compliance De-emphasis: -6dB
		LnkSta2: Current De-emphasis Level: -6dB, EqualizationComplete-, EqualizationPhase1-
			 EqualizationPhase2-, EqualizationPhase3-, LinkEqualizationRequest-
	Capabilities: [100 v1] Virtual Channel
		Caps:	LPEVC=0 RefClk=100ns PATEntryBits=1
		Arb:	Fixed- WRR32- WRR64- WRR128-
		Ctrl:	ArbSelect=Fixed
		Status:	InProgress-
		VC0:	Caps:	PATOffset=00 MaxTimeSlots=1 RejSnoopTrans-
			Arb:	Fixed+ WRR32- WRR64- WRR128- TWRR128- WRR256-
			Ctrl:	Enable+ ID=0 ArbSelect=Fixed TC/VC=ff
			Status:	NegoPending+ InProgress-
	Capabilities: [140 v1] Root Complex Link
		Desc:	PortNumber=02 ComponentID=01 EltType=Config
		Link0:	Desc:	TargetPort=00 TargetComponent=01 AssocRCRB- LinkType=MemMapped LinkValid+
			Addr:	00000000fed19000
	Kernel driver in use: pcieport

00:01.1 PCI bridge: Intel Corporation Xeon E3-1200/2nd Generation Core Processor Family PCI Express Root Port (rev 09) (prog-if 00 [Normal decode])
	Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx+
	Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
	Latency: 0, Cache Line Size: 64 bytes
	Bus: primary=00, secondary=02, subordinate=03, sec-latency=0
	Memory behind bridge: f7e00000-f7efffff
	Secondary status: 66MHz- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort+ <SERR- <PERR-
	BridgeCtl: Parity- SERR- NoISA- VGA- MAbort- >Reset- FastB2B-
		PriDiscTmr- SecDiscTmr- DiscTmrStat- DiscTmrSERREn-
	Capabilities: [88] Subsystem: Intel Corporation Xeon E3-1200/2nd Generation Core Processor Family PCI Express Root Port
	Capabilities: [80] Power Management version 3
		Flags: PMEClk- DSI- D1- D2- AuxCurrent=0mA PME(D0+,D1-,D2-,D3hot+,D3cold+)
		Status: D0 NoSoftRst+ PME-Enable- DSel=0 DScale=0 PME-
	Capabilities: [90] MSI: Enable+ Count=1/1 Maskable- 64bit-
		Address: fee002b8  Data: 0000
	Capabilities: [a0] Express (v2) Root Port (Slot+), MSI 00
		DevCap:	MaxPayload 128 bytes, PhantFunc 0, Latency L0s <64ns, L1 <1us
			ExtTag- RBE+ FLReset-
		DevCtl:	Report errors: Correctable- Non-Fatal- Fatal- Unsupported-
			RlxdOrd- ExtTag- PhantFunc- AuxPwr- NoSnoop-
			MaxPayload 128 bytes, MaxReadReq 128 bytes
		DevSta:	CorrErr- UncorrErr- FatalErr- UnsuppReq- AuxPwr- TransPend-
		LnkCap:	Port #3, Speed 5GT/s, Width x8, ASPM L0s L1, Latency L0 <1us, L1 <4us
			ClockPM- Surprise- LLActRep- BwNot+
		LnkCtl:	ASPM Disabled; RCB 64 bytes Disabled- Retrain- CommClk-
			ExtSynch- ClockPM- AutWidDis- BWInt- AutBWInt-
		LnkSta:	Speed 2.5GT/s, Width x1, TrErr- Train- SlotClk+ DLActive- BWMgmt+ ABWMgmt-
		SltCap:	AttnBtn- PwrCtrl- MRL- AttnInd- PwrInd- HotPlug- Surprise-
			Slot #2, PowerLimit 75.000W; Interlock- NoCompl+
		SltCtl:	Enable: AttnBtn- PwrFlt- MRL- PresDet- CmdCplt- HPIrq- LinkChg-
			Control: AttnInd Unknown, PwrInd Unknown, Power- Interlock-
		SltSta:	Status: AttnBtn- PowerFlt- MRL- CmdCplt- PresDet+ Interlock-
			Changed: MRL- PresDet- LinkState-
		RootCtl: ErrCorrectable- ErrNon-Fatal- ErrFatal- PMEIntEna- CRSVisible-
		RootCap: CRSVisible-
		RootSta: PME ReqID 0000, PMEStatus- PMEPending-
		DevCap2: Completion Timeout: Not Supported, TimeoutDis-, LTR+, OBFF Not Supported ARIFwd-
		DevCtl2: Completion Timeout: 50us to 50ms, TimeoutDis-, LTR-, OBFF Disabled ARIFwd-
		LnkCtl2: Target Link Speed: 5GT/s, EnterCompliance- SpeedDis-
			 Transmit Margin: Normal Operating Range, EnterModifiedCompliance- ComplianceSOS-
			 Compliance De-emphasis: -6dB
		LnkSta2: Current De-emphasis Level: -6dB, EqualizationComplete-, EqualizationPhase1-
			 EqualizationPhase2-, EqualizationPhase3-, LinkEqualizationRequest-
	Capabilities: [100 v1] Virtual Channel
		Caps:	LPEVC=0 RefClk=100ns PATEntryBits=1
		Arb:	Fixed- WRR32- WRR64- WRR128-
		Ctrl:	ArbSelect=Fixed
		Status:	InProgress-
		VC0:	Caps:	PATOffset=00 MaxTimeSlots=1 RejSnoopTrans-
			Arb:	Fixed+ WRR32- WRR64- WRR128- TWRR128- WRR256-
			Ctrl:	Enable+ ID=0 ArbSelect=Fixed TC/VC=ff
			Status:	NegoPending- InProgress-
	Capabilities: [140 v1] Root Complex Link
		Desc:	PortNumber=03 ComponentID=01 EltType=Config
		Link0:	Desc:	TargetPort=00 TargetComponent=01 AssocRCRB- LinkType=MemMapped LinkValid+
			Addr:	00000000fed19000
	Kernel driver in use: pcieport

00:02.0 VGA compatible controller: Intel Corporation Xeon E3-1200 Processor Family Integrated Graphics Controller (rev 09) (prog-if 00 [VGA controller])
	Subsystem: Intel Corporation Device 2111
	Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx+
	Status: Cap+ 66MHz- UDF- FastB2B+ ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
	Latency: 0
	Interrupt: pin A routed to IRQ 58
	Region 0: Memory at f7400000 (64-bit, non-prefetchable) [size=4M]
	Region 2: Memory at e0000000 (64-bit, prefetchable) [size=256M]
	Region 4: I/O ports at f000 [size=64]
	Expansion ROM at <unassigned> [disabled]
	Capabilities: [90] MSI: Enable+ Count=1/1 Maskable- 64bit-
		Address: fee00018  Data: 0000
	Capabilities: [d0] Power Management version 2
		Flags: PMEClk- DSI+ D1- D2- AuxCurrent=0mA PME(D0-,D1-,D2-,D3hot-,D3cold-)
		Status: D0 NoSoftRst- PME-Enable- DSel=0 DScale=0 PME-
	Capabilities: [a4] PCI Advanced Features
		AFCap: TP+ FLR+
		AFCtrl: FLR-
		AFStatus: TP-
	Kernel driver in use: i915
	Kernel modules: i915

00:06.0 PCI bridge: Intel Corporation Xeon E3-1200/2nd Generation Core Processor Family PCI Express Root Port (rev 09) (prog-if 00 [Normal decode])
	Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx+
	Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
	Latency: 0, Cache Line Size: 64 bytes
	Bus: primary=00, secondary=04, subordinate=04, sec-latency=0
	Memory behind bridge: f7d00000-f7dfffff
	Secondary status: 66MHz- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort+ <SERR- <PERR-
	BridgeCtl: Parity- SERR- NoISA- VGA- MAbort- >Reset- FastB2B-
		PriDiscTmr- SecDiscTmr- DiscTmrStat- DiscTmrSERREn-
	Capabilities: [88] Subsystem: Intel Corporation Xeon E3-1200/2nd Generation Core Processor Family PCI Express Root Port
	Capabilities: [80] Power Management version 3
		Flags: PMEClk- DSI- D1- D2- AuxCurrent=0mA PME(D0+,D1-,D2-,D3hot+,D3cold+)
		Status: D0 NoSoftRst+ PME-Enable- DSel=0 DScale=0 PME-
	Capabilities: [90] MSI: Enable+ Count=1/1 Maskable- 64bit-
		Address: fee002d8  Data: 0000
	Capabilities: [a0] Express (v2) Root Port (Slot+), MSI 00
		DevCap:	MaxPayload 128 bytes, PhantFunc 0, Latency L0s <64ns, L1 <1us
			ExtTag- RBE+ FLReset-
		DevCtl:	Report errors: Correctable- Non-Fatal- Fatal- Unsupported-
			RlxdOrd- ExtTag- PhantFunc- AuxPwr- NoSnoop-
			MaxPayload 128 bytes, MaxReadReq 128 bytes
		DevSta:	CorrErr- UncorrErr- FatalErr- UnsuppReq- AuxPwr- TransPend-
		LnkCap:	Port #5, Speed 5GT/s, Width x4, ASPM L0s L1, Latency L0 <256ns, L1 <4us
			ClockPM- Surprise- LLActRep- BwNot+
		LnkCtl:	ASPM Disabled; RCB 64 bytes Disabled- Retrain- CommClk+
			ExtSynch- ClockPM- AutWidDis- BWInt- AutBWInt-
		LnkSta:	Speed 2.5GT/s, Width x1, TrErr- Train- SlotClk+ DLActive- BWMgmt+ ABWMgmt-
		SltCap:	AttnBtn- PwrCtrl- MRL- AttnInd- PwrInd- HotPlug- Surprise-
			Slot #4, PowerLimit 75.000W; Interlock- NoCompl+
		SltCtl:	Enable: AttnBtn- PwrFlt- MRL- PresDet- CmdCplt- HPIrq- LinkChg-
			Control: AttnInd Unknown, PwrInd Unknown, Power- Interlock-
		SltSta:	Status: AttnBtn- PowerFlt- MRL- CmdCplt- PresDet+ Interlock-
			Changed: MRL- PresDet- LinkState-
		RootCtl: ErrCorrectable- ErrNon-Fatal- ErrFatal- PMEIntEna- CRSVisible-
		RootCap: CRSVisible-
		RootSta: PME ReqID 0000, PMEStatus- PMEPending-
		DevCap2: Completion Timeout: Not Supported, TimeoutDis-, LTR+, OBFF Not Supported ARIFwd-
		DevCtl2: Completion Timeout: 50us to 50ms, TimeoutDis-, LTR-, OBFF Disabled ARIFwd-
		LnkCtl2: Target Link Speed: 5GT/s, EnterCompliance- SpeedDis-
			 Transmit Margin: Normal Operating Range, EnterModifiedCompliance- ComplianceSOS-
			 Compliance De-emphasis: -6dB
		LnkSta2: Current De-emphasis Level: -6dB, EqualizationComplete-, EqualizationPhase1-
			 EqualizationPhase2-, EqualizationPhase3-, LinkEqualizationRequest-
	Capabilities: [100 v1] Virtual Channel
		Caps:	LPEVC=0 RefClk=100ns PATEntryBits=1
		Arb:	Fixed- WRR32- WRR64- WRR128-
		Ctrl:	ArbSelect=Fixed
		Status:	InProgress-
		VC0:	Caps:	PATOffset=00 MaxTimeSlots=1 RejSnoopTrans-
			Arb:	Fixed+ WRR32- WRR64- WRR128- TWRR128- WRR256-
			Ctrl:	Enable+ ID=0 ArbSelect=Fixed TC/VC=ff
			Status:	NegoPending- InProgress-
	Capabilities: [140 v1] Root Complex Link
		Desc:	PortNumber=05 ComponentID=01 EltType=Config
		Link0:	Desc:	TargetPort=00 TargetComponent=01 AssocRCRB- LinkType=MemMapped LinkValid+
			Addr:	00000000fed19000
	Kernel driver in use: pcieport

00:16.0 Communication controller: Intel Corporation 6 Series/C200 Series Chipset Family MEI Controller #1 (rev 04)
	Subsystem: Intel Corporation 6 Series/C200 Series Chipset Family MEI Controller #1
	Control: I/O- Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx-
	Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
	Latency: 0
	Interrupt: pin A routed to IRQ 14
	Region 0: Memory at f7f2c000 (64-bit, non-prefetchable) [size=16]
	Capabilities: [50] Power Management version 3
		Flags: PMEClk- DSI- D1- D2- AuxCurrent=0mA PME(D0+,D1-,D2-,D3hot+,D3cold+)
		Status: D0 NoSoftRst+ PME-Enable- DSel=0 DScale=0 PME-
	Capabilities: [8c] MSI: Enable- Count=1/1 Maskable- 64bit+
		Address: 0000000000000000  Data: 0000

00:16.3 Serial controller: Intel Corporation 6 Series/C200 Series Chipset Family KT Controller (rev 04) (prog-if 02 [16550])
	Subsystem: Intel Corporation 6 Series/C200 Series Chipset Family KT Controller
	Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx-
	Status: Cap+ 66MHz+ UDF- FastB2B+ ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
	Latency: 0
	Interrupt: pin B routed to IRQ 5
	Region 0: I/O ports at f0e0 [size=8]
	Region 1: Memory at f7f2a000 (32-bit, non-prefetchable) [size=4K]
	Capabilities: [c8] Power Management version 3
		Flags: PMEClk- DSI+ D1- D2- AuxCurrent=0mA PME(D0-,D1-,D2-,D3hot-,D3cold-)
		Status: D0 NoSoftRst+ PME-Enable- DSel=0 DScale=0 PME-
	Capabilities: [d0] MSI: Enable- Count=1/1 Maskable- 64bit+
		Address: 0000000000000000  Data: 0000

00:19.0 Ethernet controller: Intel Corporation 82579LM Gigabit Network Connection (rev 05)
	Subsystem: Intel Corporation Device 0000
	Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx+
	Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
	Latency: 0
	Interrupt: pin A routed to IRQ 54
	Region 0: Memory at f7f00000 (32-bit, non-prefetchable) [size=128K]
	Region 1: Memory at f7f29000 (32-bit, non-prefetchable) [size=4K]
	Region 2: I/O ports at f080 [size=32]
	Capabilities: [c8] Power Management version 2
		Flags: PMEClk- DSI+ D1- D2- AuxCurrent=0mA PME(D0+,D1-,D2-,D3hot+,D3cold+)
		Status: D0 NoSoftRst- PME-Enable- DSel=0 DScale=1 PME-
	Capabilities: [d0] MSI: Enable+ Count=1/1 Maskable- 64bit+
		Address: 00000000fee00358  Data: 0000
	Capabilities: [e0] PCI Advanced Features
		AFCap: TP+ FLR+
		AFCtrl: FLR-
		AFStatus: TP-
	Kernel driver in use: e1000e

00:1a.0 USB controller: Intel Corporation 6 Series/C200 Series Chipset Family USB Enhanced Host Controller #2 (rev 05) (prog-if 20 [EHCI])
	Subsystem: Intel Corporation 6 Series/C200 Series Chipset Family USB Enhanced Host Controller #2
	Control: I/O- Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx-
	Status: Cap+ 66MHz- UDF- FastB2B+ ParErr- DEVSEL=medium >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
	Latency: 0
	Interrupt: pin A routed to IRQ 16
	Region 0: Memory at f7f28000 (32-bit, non-prefetchable) [size=1K]
	Capabilities: [50] Power Management version 2
		Flags: PMEClk- DSI- D1- D2- AuxCurrent=375mA PME(D0+,D1-,D2-,D3hot+,D3cold+)
		Status: D0 NoSoftRst- PME-Enable- DSel=0 DScale=0 PME-
	Capabilities: [58] Debug port: BAR=1 offset=00a0
	Capabilities: [98] PCI Advanced Features
		AFCap: TP+ FLR+
		AFCtrl: FLR-
		AFStatus: TP-
	Kernel driver in use: ehci-pci
	Kernel modules: ehci_pci

00:1b.0 Audio device: Intel Corporation 6 Series/C200 Series Chipset Family High Definition Audio Controller (rev 05)
	Subsystem: Intel Corporation 6 Series/C200 Series Chipset Family High Definition Audio Controller
	Control: I/O- Mem- BusMaster- SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx-
	Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
	Interrupt: pin A routed to IRQ 22
	Region 0: Memory at f7f20000 (64-bit, non-prefetchable) [disabled] [size=16K]
	Capabilities: [50] Power Management version 2
		Flags: PMEClk- DSI- D1- D2- AuxCurrent=55mA PME(D0+,D1-,D2-,D3hot+,D3cold+)
		Status: D0 NoSoftRst- PME-Enable- DSel=0 DScale=0 PME-
	Capabilities: [60] MSI: Enable- Count=1/1 Maskable- 64bit+
		Address: 0000000000000000  Data: 0000
	Capabilities: [70] Express (v1) Root Complex Integrated Endpoint, MSI 00
		DevCap:	MaxPayload 128 bytes, PhantFunc 0, Latency L0s <64ns, L1 <1us
			ExtTag- RBE- FLReset+
		DevCtl:	Report errors: Correctable- Non-Fatal- Fatal- Unsupported-
			RlxdOrd- ExtTag- PhantFunc- AuxPwr- NoSnoop+
			MaxPayload 128 bytes, MaxReadReq 128 bytes
		DevSta:	CorrErr- UncorrErr- FatalErr- UnsuppReq- AuxPwr+ TransPend-
		LnkCap:	Port #0, Speed unknown, Width x0, ASPM unknown, Latency L0 <64ns, L1 <1us
			ClockPM- Surprise- LLActRep- BwNot-
		LnkCtl:	ASPM Disabled; Disabled- Retrain- CommClk-
			ExtSynch- ClockPM- AutWidDis- BWInt- AutBWInt-
		LnkSta:	Speed unknown, Width x0, TrErr- Train- SlotClk- DLActive- BWMgmt- ABWMgmt-
	Capabilities: [100 v1] Virtual Channel
		Caps:	LPEVC=0 RefClk=100ns PATEntryBits=1
		Arb:	Fixed- WRR32- WRR64- WRR128-
		Ctrl:	ArbSelect=Fixed
		Status:	InProgress-
		VC0:	Caps:	PATOffset=00 MaxTimeSlots=1 RejSnoopTrans-
			Arb:	Fixed- WRR32- WRR64- WRR128- TWRR128- WRR256-
			Ctrl:	Enable+ ID=0 ArbSelect=Fixed TC/VC=01
			Status:	NegoPending- InProgress-
		VC1:	Caps:	PATOffset=00 MaxTimeSlots=1 RejSnoopTrans-
			Arb:	Fixed- WRR32- WRR64- WRR128- TWRR128- WRR256-
			Ctrl:	Enable+ ID=2 ArbSelect=Fixed TC/VC=44
			Status:	NegoPending- InProgress-
	Capabilities: [130 v1] Root Complex Link
		Desc:	PortNumber=0f ComponentID=00 EltType=Config
		Link0:	Desc:	TargetPort=00 TargetComponent=00 AssocRCRB- LinkType=MemMapped LinkValid+
			Addr:	00000000fed1c000
	Kernel driver in use: pciback

00:1c.0 PCI bridge: Intel Corporation 6 Series/C200 Series Chipset Family PCI Express Root Port 1 (rev b5) (prog-if 00 [Normal decode])
	Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx-
	Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
	Latency: 0, Cache Line Size: 64 bytes
	Bus: primary=00, secondary=05, subordinate=06, sec-latency=0
	Memory behind bridge: f7c00000-f7cfffff
	Secondary status: 66MHz- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort+ <SERR- <PERR-
	BridgeCtl: Parity- SERR- NoISA- VGA- MAbort- >Reset- FastB2B-
		PriDiscTmr- SecDiscTmr- DiscTmrStat- DiscTmrSERREn-
	Capabilities: [40] Express (v2) Root Port (Slot+), MSI 00
		DevCap:	MaxPayload 128 bytes, PhantFunc 0, Latency L0s <64ns, L1 <1us
			ExtTag- RBE+ FLReset-
		DevCtl:	Report errors: Correctable- Non-Fatal- Fatal- Unsupported-
			RlxdOrd- ExtTag- PhantFunc- AuxPwr- NoSnoop-
			MaxPayload 128 bytes, MaxReadReq 128 bytes
		DevSta:	CorrErr- UncorrErr- FatalErr- UnsuppReq- AuxPwr+ TransPend-
		LnkCap:	Port #1, Speed 5GT/s, Width x4, ASPM L0s L1, Latency L0 <1us, L1 <4us
			ClockPM- Surprise- LLActRep+ BwNot-
		LnkCtl:	ASPM Disabled; RCB 64 bytes Disabled- Retrain- CommClk-
			ExtSynch- ClockPM- AutWidDis- BWInt- AutBWInt-
		LnkSta:	Speed 2.5GT/s, Width x1, TrErr- Train- SlotClk+ DLActive+ BWMgmt- ABWMgmt-
		SltCap:	AttnBtn- PwrCtrl- MRL- AttnInd- PwrInd- HotPlug- Surprise-
			Slot #0, PowerLimit 25.000W; Interlock- NoCompl+
		SltCtl:	Enable: AttnBtn- PwrFlt- MRL- PresDet- CmdCplt- HPIrq- LinkChg-
			Control: AttnInd Unknown, PwrInd Unknown, Power- Interlock-
		SltSta:	Status: AttnBtn- PowerFlt- MRL- CmdCplt- PresDet+ Interlock-
			Changed: MRL- PresDet- LinkState-
		RootCtl: ErrCorrectable- ErrNon-Fatal- ErrFatal- PMEIntEna- CRSVisible-
		RootCap: CRSVisible-
		RootSta: PME ReqID 0000, PMEStatus- PMEPending-
		DevCap2: Completion Timeout: Range BC, TimeoutDis+, LTR-, OBFF Not Supported ARIFwd-
		DevCtl2: Completion Timeout: 50us to 50ms, TimeoutDis-, LTR-, OBFF Disabled ARIFwd-
		LnkCtl2: Target Link Speed: 5GT/s, EnterCompliance- SpeedDis-
			 Transmit Margin: Normal Operating Range, EnterModifiedCompliance- ComplianceSOS-
			 Compliance De-emphasis: -6dB
		LnkSta2: Current De-emphasis Level: -3.5dB, EqualizationComplete-, EqualizationPhase1-
			 EqualizationPhase2-, EqualizationPhase3-, LinkEqualizationRequest-
	Capabilities: [80] MSI: Enable- Count=1/1 Maskable- 64bit-
		Address: 00000000  Data: 0000
	Capabilities: [90] Subsystem: Intel Corporation 6 Series/C200 Series Chipset Family PCI Express Root Port 1
	Capabilities: [a0] Power Management version 2
		Flags: PMEClk- DSI- D1- D2- AuxCurrent=0mA PME(D0+,D1-,D2-,D3hot+,D3cold+)
		Status: D0 NoSoftRst- PME-Enable- DSel=0 DScale=0 PME-
	Kernel driver in use: pcieport

00:1c.4 PCI bridge: Intel Corporation 6 Series/C200 Series Chipset Family PCI Express Root Port 5 (rev b5) (prog-if 00 [Normal decode])
	Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx-
	Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
	Latency: 0, Cache Line Size: 64 bytes
	Bus: primary=00, secondary=07, subordinate=07, sec-latency=0
	I/O behind bridge: 0000e000-0000efff
	Memory behind bridge: f7b00000-f7bfffff
	Secondary status: 66MHz- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- <SERR- <PERR-
	BridgeCtl: Parity- SERR- NoISA- VGA- MAbort- >Reset- FastB2B-
		PriDiscTmr- SecDiscTmr- DiscTmrStat- DiscTmrSERREn-
	Capabilities: [40] Express (v2) Root Port (Slot+), MSI 00
		DevCap:	MaxPayload 128 bytes, PhantFunc 0, Latency L0s <64ns, L1 <1us
			ExtTag- RBE+ FLReset-
		DevCtl:	Report errors: Correctable- Non-Fatal- Fatal- Unsupported-
			RlxdOrd- ExtTag- PhantFunc- AuxPwr- NoSnoop-
			MaxPayload 128 bytes, MaxReadReq 128 bytes
		DevSta:	CorrErr+ UncorrErr- FatalErr- UnsuppReq- AuxPwr+ TransPend-
		LnkCap:	Port #5, Speed 5GT/s, Width x1, ASPM L0s L1, Latency L0 <512ns, L1 <4us
			ClockPM- Surprise- LLActRep+ BwNot-
		LnkCtl:	ASPM Disabled; RCB 64 bytes Disabled- Retrain- CommClk+
			ExtSynch- ClockPM- AutWidDis- BWInt- AutBWInt-
		LnkSta:	Speed 2.5GT/s, Width x1, TrErr- Train- SlotClk+ DLActive+ BWMgmt+ ABWMgmt-
		SltCap:	AttnBtn- PwrCtrl- MRL- AttnInd- PwrInd- HotPlug- Surprise-
			Slot #4, PowerLimit 10.000W; Interlock- NoCompl+
		SltCtl:	Enable: AttnBtn- PwrFlt- MRL- PresDet- CmdCplt- HPIrq- LinkChg-
			Control: AttnInd Unknown, PwrInd Unknown, Power- Interlock-
		SltSta:	Status: AttnBtn- PowerFlt- MRL- CmdCplt- PresDet+ Interlock-
			Changed: MRL- PresDet- LinkState-
		RootCtl: ErrCorrectable- ErrNon-Fatal- ErrFatal- PMEIntEna- CRSVisible-
		RootCap: CRSVisible-
		RootSta: PME ReqID 0000, PMEStatus- PMEPending-
		DevCap2: Completion Timeout: Range BC, TimeoutDis+, LTR-, OBFF Not Supported ARIFwd-
		DevCtl2: Completion Timeout: 50us to 50ms, TimeoutDis-, LTR-, OBFF Disabled ARIFwd-
		LnkCtl2: Target Link Speed: 5GT/s, EnterCompliance- SpeedDis-
			 Transmit Margin: Normal Operating Range, EnterModifiedCompliance- ComplianceSOS-
			 Compliance De-emphasis: -6dB
		LnkSta2: Current De-emphasis Level: -3.5dB, EqualizationComplete-, EqualizationPhase1-
			 EqualizationPhase2-, EqualizationPhase3-, LinkEqualizationRequest-
	Capabilities: [80] MSI: Enable- Count=1/1 Maskable- 64bit-
		Address: 00000000  Data: 0000
	Capabilities: [90] Subsystem: Intel Corporation 6 Series/C200 Series Chipset Family PCI Express Root Port 5
	Capabilities: [a0] Power Management version 2
		Flags: PMEClk- DSI- D1- D2- AuxCurrent=0mA PME(D0+,D1-,D2-,D3hot+,D3cold+)
		Status: D0 NoSoftRst- PME-Enable- DSel=0 DScale=0 PME-
	Kernel driver in use: pcieport

00:1c.6 PCI bridge: Intel Corporation 6 Series/C200 Series Chipset Family PCI Express Root Port 7 (rev b5) (prog-if 00 [Normal decode])
	Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx-
	Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
	Latency: 0, Cache Line Size: 64 bytes
	Bus: primary=00, secondary=08, subordinate=08, sec-latency=0
	Memory behind bridge: f7a00000-f7afffff
	Secondary status: 66MHz- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- <SERR- <PERR-
	BridgeCtl: Parity- SERR- NoISA- VGA- MAbort- >Reset- FastB2B-
		PriDiscTmr- SecDiscTmr- DiscTmrStat- DiscTmrSERREn-
	Capabilities: [40] Express (v2) Root Port (Slot+), MSI 00
		DevCap:	MaxPayload 128 bytes, PhantFunc 0, Latency L0s <64ns, L1 <1us
			ExtTag- RBE+ FLReset-
		DevCtl:	Report errors: Correctable- Non-Fatal- Fatal- Unsupported-
			RlxdOrd- ExtTag- PhantFunc- AuxPwr- NoSnoop-
			MaxPayload 128 bytes, MaxReadReq 128 bytes
		DevSta:	CorrErr- UncorrErr- FatalErr- UnsuppReq- AuxPwr+ TransPend-
		LnkCap:	Port #7, Speed 5GT/s, Width x1, ASPM L0s L1, Latency L0 <512ns, L1 <4us
			ClockPM- Surprise- LLActRep+ BwNot-
		LnkCtl:	ASPM Disabled; RCB 64 bytes Disabled- Retrain- CommClk+
			ExtSynch- ClockPM- AutWidDis- BWInt- AutBWInt-
		LnkSta:	Speed 5GT/s, Width x1, TrErr- Train- SlotClk+ DLActive+ BWMgmt+ ABWMgmt-
		SltCap:	AttnBtn- PwrCtrl- MRL- AttnInd- PwrInd- HotPlug- Surprise-
			Slot #6, PowerLimit 10.000W; Interlock- NoCompl+
		SltCtl:	Enable: AttnBtn- PwrFlt- MRL- PresDet- CmdCplt- HPIrq- LinkChg-
			Control: AttnInd Unknown, PwrInd Unknown, Power- Interlock-
		SltSta:	Status: AttnBtn- PowerFlt- MRL- CmdCplt- PresDet+ Interlock-
			Changed: MRL- PresDet- LinkState-
		RootCtl: ErrCorrectable- ErrNon-Fatal- ErrFatal- PMEIntEna- CRSVisible-
		RootCap: CRSVisible-
		RootSta: PME ReqID 0000, PMEStatus- PMEPending-
		DevCap2: Completion Timeout: Range BC, TimeoutDis+, LTR-, OBFF Not Supported ARIFwd-
		DevCtl2: Completion Timeout: 50us to 50ms, TimeoutDis-, LTR-, OBFF Disabled ARIFwd-
		LnkCtl2: Target Link Speed: 5GT/s, EnterCompliance- SpeedDis-
			 Transmit Margin: Normal Operating Range, EnterModifiedCompliance- ComplianceSOS-
			 Compliance De-emphasis: -6dB
		LnkSta2: Current De-emphasis Level: -6dB, EqualizationComplete-, EqualizationPhase1-
			 EqualizationPhase2-, EqualizationPhase3-, LinkEqualizationRequest-
	Capabilities: [80] MSI: Enable- Count=1/1 Maskable- 64bit-
		Address: 00000000  Data: 0000
	Capabilities: [90] Subsystem: Intel Corporation 6 Series/C200 Series Chipset Family PCI Express Root Port 7
	Capabilities: [a0] Power Management version 2
		Flags: PMEClk- DSI- D1- D2- AuxCurrent=0mA PME(D0+,D1-,D2-,D3hot+,D3cold+)
		Status: D0 NoSoftRst- PME-Enable- DSel=0 DScale=0 PME-
	Kernel driver in use: pcieport

00:1d.0 USB controller: Intel Corporation 6 Series/C200 Series Chipset Family USB Enhanced Host Controller #1 (rev 05) (prog-if 20 [EHCI])
	Subsystem: Intel Corporation 6 Series/C200 Series Chipset Family USB Enhanced Host Controller #1
	Control: I/O- Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx-
	Status: Cap+ 66MHz- UDF- FastB2B+ ParErr- DEVSEL=medium >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
	Latency: 0
	Interrupt: pin A routed to IRQ 23
	Region 0: Memory at f7f27000 (32-bit, non-prefetchable) [size=1K]
	Capabilities: [50] Power Management version 2
		Flags: PMEClk- DSI- D1- D2- AuxCurrent=375mA PME(D0+,D1-,D2-,D3hot+,D3cold+)
		Status: D0 NoSoftRst- PME-Enable- DSel=0 DScale=0 PME-
	Capabilities: [58] Debug port: BAR=1 offset=00a0
	Capabilities: [98] PCI Advanced Features
		AFCap: TP+ FLR+
		AFCtrl: FLR-
		AFStatus: TP-
	Kernel driver in use: ehci-pci
	Kernel modules: ehci_pci

00:1e.0 PCI bridge: Intel Corporation 82801 PCI Bridge (rev a5) (prog-if 01 [Subtractive decode])
	Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx-
	Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
	Latency: 0
	Bus: primary=00, secondary=09, subordinate=0a, sec-latency=32
	I/O behind bridge: 0000d000-0000dfff
	Memory behind bridge: f7800000-f79fffff
	Secondary status: 66MHz- FastB2B+ ParErr- DEVSEL=medium >TAbort- <TAbort- <MAbort- <SERR- <PERR-
	BridgeCtl: Parity- SERR- NoISA- VGA- MAbort- >Reset- FastB2B-
		PriDiscTmr- SecDiscTmr- DiscTmrStat- DiscTmrSERREn-
	Capabilities: [50] Subsystem: Intel Corporation 82801 PCI Bridge

00:1f.0 ISA bridge: Intel Corporation C206 Chipset Family LPC Controller (rev 05)
	Subsystem: Intel Corporation C206 Chipset Family LPC Controller
	Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx-
	Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=medium >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
	Latency: 0
	Capabilities: [e0] Vendor Specific Information: Len=0c <?>
	Kernel driver in use: lpc_ich
	Kernel modules: lpc_ich

00:1f.2 SATA controller: Intel Corporation 6 Series/C200 Series Chipset Family SATA AHCI Controller (rev 05) (prog-if 01 [AHCI 1.0])
	Subsystem: Intel Corporation 6 Series/C200 Series Chipset Family SATA AHCI Controller
	Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx+
	Status: Cap+ 66MHz+ UDF- FastB2B+ ParErr- DEVSEL=medium >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
	Latency: 0
	Interrupt: pin B routed to IRQ 53
	Region 0: I/O ports at f0d0 [size=8]
	Region 1: I/O ports at f0c0 [size=4]
	Region 2: I/O ports at f0b0 [size=8]
	Region 3: I/O ports at f0a0 [size=4]
	Region 4: I/O ports at f060 [size=32]
	Region 5: Memory at f7f26000 (32-bit, non-prefetchable) [size=2K]
	Capabilities: [80] MSI: Enable+ Count=1/1 Maskable- 64bit-
		Address: fee00318  Data: 0000
	Capabilities: [70] Power Management version 3
		Flags: PMEClk- DSI- D1- D2- AuxCurrent=0mA PME(D0-,D1-,D2-,D3hot+,D3cold-)
		Status: D0 NoSoftRst+ PME-Enable- DSel=0 DScale=0 PME-
	Capabilities: [a8] SATA HBA v1.0 BAR4 Offset=00000004
	Capabilities: [b0] PCI Advanced Features
		AFCap: TP+ FLR+
		AFCtrl: FLR-
		AFStatus: TP-
	Kernel driver in use: ahci

00:1f.3 SMBus: Intel Corporation 6 Series/C200 Series Chipset Family SMBus Controller (rev 05)
	Subsystem: Intel Corporation 6 Series/C200 Series Chipset Family SMBus Controller
	Control: I/O+ Mem+ BusMaster- SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx-
	Status: Cap- 66MHz- UDF- FastB2B+ ParErr- DEVSEL=medium >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
	Interrupt: pin C routed to IRQ 18
	Region 0: Memory at f7f25000 (64-bit, non-prefetchable) [size=256]
	Region 4: I/O ports at f040 [size=32]
	Kernel modules: i2c_i801

02:00.0 PCI bridge: ASMedia Technology Inc. ASM1083/1085 PCIe to PCI Bridge (rev 01) (prog-if 00 [Normal decode])
	Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx-
	Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
	Latency: 0, Cache Line Size: 64 bytes
	Bus: primary=02, secondary=03, subordinate=03, sec-latency=32
	Memory behind bridge: f7e00000-f7efffff
	Secondary status: 66MHz+ FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort+ <SERR- <PERR-
	BridgeCtl: Parity- SERR- NoISA- VGA- MAbort- >Reset- FastB2B-
		PriDiscTmr- SecDiscTmr- DiscTmrStat- DiscTmrSERREn-
	Capabilities: [50] MSI: Enable- Count=1/1 Maskable- 64bit+
		Address: 0000000000000000  Data: 0000
	Capabilities: [78] Power Management version 3
		Flags: PMEClk- DSI+ D1+ D2+ AuxCurrent=0mA PME(D0+,D1+,D2+,D3hot+,D3cold+)
		Status: D0 NoSoftRst+ PME-Enable- DSel=0 DScale=0 PME-
	Capabilities: [80] Express (v1) PCI/PCI-X Bridge, MSI 00
		DevCap:	MaxPayload 128 bytes, PhantFunc 0, Latency L0s <64ns, L1 <1us
			ExtTag- AttnBtn- AttnInd- PwrInd- RBE+ FLReset-
		DevCtl:	Report errors: Correctable- Non-Fatal- Fatal- Unsupported-
			RlxdOrd- ExtTag- PhantFunc- AuxPwr- NoSnoop+ BrConfRtry-
			MaxPayload 128 bytes, MaxReadReq 512 bytes
		DevSta:	CorrErr- UncorrErr+ FatalErr- UnsuppReq- AuxPwr- TransPend-
		LnkCap:	Port #1, Speed 2.5GT/s, Width x1, ASPM L0s L1, Latency L0 <2us, L1 <2us
			ClockPM- Surprise- LLActRep- BwNot-
		LnkCtl:	ASPM Disabled; Disabled- Retrain- CommClk-
			ExtSynch- ClockPM- AutWidDis- BWInt- AutBWInt-
		LnkSta:	Speed 2.5GT/s, Width x1, TrErr- Train- SlotClk- DLActive- BWMgmt- ABWMgmt-
	Capabilities: [c0] Subsystem: Device 0000:0000
	Capabilities: [100 v1] Virtual Channel
		Caps:	LPEVC=0 RefClk=100ns PATEntryBits=1
		Arb:	Fixed- WRR32- WRR64- WRR128-
		Ctrl:	ArbSelect=Fixed
		Status:	InProgress-
		VC0:	Caps:	PATOffset=00 MaxTimeSlots=1 RejSnoopTrans-
			Arb:	Fixed- WRR32- WRR64- WRR128- TWRR128- WRR256-
			Ctrl:	Enable+ ID=0 ArbSelect=Fixed TC/VC=01
			Status:	NegoPending- InProgress-
	Kernel driver in use: pciback

03:00.0 Unassigned class [ff00]: OpenVox Communication Co. Ltd. Device 0100 (rev 01)
	Subsystem: OpenVox Communication Co. Ltd. Device 0104
	Control: I/O- Mem- BusMaster- SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx-
	Status: Cap- 66MHz- UDF- FastB2B- ParErr- DEVSEL=slow >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
	Interrupt: pin A routed to IRQ 17
	Region 0: Memory at f7e00000 (32-bit, non-prefetchable) [disabled] [size=64K]
	Kernel driver in use: pciback

04:00.0 Network controller: Atheros Communications Inc. AR9300 Wireless LAN adaptor (rev 01)
	Subsystem: Atheros Communications Inc. Device 3114
	Control: I/O- Mem- BusMaster- SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx-
	Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
	Interrupt: pin A routed to IRQ 19
	Region 0: Memory at f7d00000 (64-bit, non-prefetchable) [disabled] [size=128K]
	Expansion ROM at f7d20000 [disabled] [size=64K]
	Capabilities: [40] Power Management version 3
		Flags: PMEClk- DSI- D1+ D2- AuxCurrent=375mA PME(D0+,D1+,D2-,D3hot+,D3cold-)
		Status: D0 NoSoftRst- PME-Enable- DSel=0 DScale=0 PME-
	Capabilities: [50] MSI: Enable- Count=1/4 Maskable+ 64bit+
		Address: 0000000000000000  Data: 0000
		Masking: 00000000  Pending: 00000000
	Capabilities: [70] Express (v2) Endpoint, MSI 00
		DevCap:	MaxPayload 128 bytes, PhantFunc 0, Latency L0s <1us, L1 <8us
			ExtTag- AttnBtn- AttnInd- PwrInd- RBE+ FLReset-
		DevCtl:	Report errors: Correctable- Non-Fatal- Fatal- Unsupported-
			RlxdOrd- ExtTag- PhantFunc- AuxPwr- NoSnoop-
			MaxPayload 128 bytes, MaxReadReq 512 bytes
		DevSta:	CorrErr+ UncorrErr- FatalErr- UnsuppReq+ AuxPwr- TransPend-
		LnkCap:	Port #0, Speed 2.5GT/s, Width x1, ASPM L0s L1, Latency L0 <2us, L1 <64us
			ClockPM- Surprise- LLActRep- BwNot-
		LnkCtl:	ASPM Disabled; RCB 64 bytes Disabled- Retrain- CommClk+
			ExtSynch- ClockPM- AutWidDis- BWInt- AutBWInt-
		LnkSta:	Speed 2.5GT/s, Width x1, TrErr- Train- SlotClk+ DLActive- BWMgmt- ABWMgmt-
		DevCap2: Completion Timeout: Not Supported, TimeoutDis+, LTR-, OBFF Not Supported
		DevCtl2: Completion Timeout: 50us to 50ms, TimeoutDis-, LTR-, OBFF Disabled
		LnkCtl2: Target Link Speed: 2.5GT/s, EnterCompliance- SpeedDis-
			 Transmit Margin: Normal Operating Range, EnterModifiedCompliance- ComplianceSOS-
			 Compliance De-emphasis: -6dB
		LnkSta2: Current De-emphasis Level: -6dB, EqualizationComplete-, EqualizationPhase1-
			 EqualizationPhase2-, EqualizationPhase3-, LinkEqualizationRequest-
	Capabilities: [100 v1] Advanced Error Reporting
		UESta:	DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF- MalfTLP- ECRC- UnsupReq- ACSViol-
		UEMsk:	DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF- MalfTLP- ECRC- UnsupReq- ACSViol-
		UESvrt:	DLP+ SDES+ TLP- FCP+ CmpltTO- CmpltAbrt- UnxCmplt- RxOF+ MalfTLP+ ECRC- UnsupReq- ACSViol-
		CESta:	RxErr- BadTLP- BadDLLP- Rollover- Timeout- NonFatalErr+
		CEMsk:	RxErr- BadTLP- BadDLLP- Rollover- Timeout- NonFatalErr+
		AERCap:	First Error Pointer: 00, GenCap- CGenEn- ChkCap- ChkEn-
	Capabilities: [140 v1] Virtual Channel
		Caps:	LPEVC=0 RefClk=100ns PATEntryBits=1
		Arb:	Fixed- WRR32- WRR64- WRR128-
		Ctrl:	ArbSelect=Fixed
		Status:	InProgress-
		VC0:	Caps:	PATOffset=00 MaxTimeSlots=1 RejSnoopTrans-
			Arb:	Fixed- WRR32- WRR64- WRR128- TWRR128- WRR256-
			Ctrl:	Enable+ ID=0 ArbSelect=Fixed TC/VC=01
			Status:	NegoPending- InProgress-
	Capabilities: [300 v1] Device Serial Number 00-00-00-00-00-00-00-00
	Kernel driver in use: pciback

05:00.0 PCI bridge: PLX Technology, Inc. PEX8112 x1 Lane PCI Express-to-PCI Bridge (rev aa) (prog-if 00 [Normal decode])
	Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx-
	Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
	Latency: 0, Cache Line Size: 64 bytes
	Bus: primary=05, secondary=06, subordinate=06, sec-latency=32
	Memory behind bridge: f7c00000-f7cfffff
	Secondary status: 66MHz+ FastB2B- ParErr- DEVSEL=medium >TAbort- <TAbort- <MAbort+ <SERR- <PERR-
	BridgeCtl: Parity- SERR- NoISA- VGA- MAbort- >Reset- FastB2B-
		PriDiscTmr- SecDiscTmr- DiscTmrStat- DiscTmrSERREn-
	Capabilities: [40] Power Management version 2
		Flags: PMEClk- DSI- D1+ D2- AuxCurrent=0mA PME(D0+,D1+,D2-,D3hot+,D3cold-)
		Status: D0 NoSoftRst- PME-Enable- DSel=0 DScale=0 PME-
	Capabilities: [50] MSI: Enable- Count=1/1 Maskable- 64bit+
		Address: 0000000000000000  Data: 0000
	Capabilities: [60] Express (v1) PCI/PCI-X Bridge, MSI 00
		DevCap:	MaxPayload 128 bytes, PhantFunc 0, Latency L0s <64ns, L1 <1us
			ExtTag- AttnBtn- AttnInd- PwrInd- RBE- FLReset-
		DevCtl:	Report errors: Correctable- Non-Fatal- Fatal- Unsupported-
			RlxdOrd- ExtTag- PhantFunc- AuxPwr- NoSnoop- BrConfRtry-
			MaxPayload 128 bytes, MaxReadReq 512 bytes
		DevSta:	CorrErr- UncorrErr+ FatalErr- UnsuppReq+ AuxPwr- TransPend-
		LnkCap:	Port #0, Speed 2.5GT/s, Width x1, ASPM L0s L1, Latency L0 <1us, L1 <16us
			ClockPM- Surprise- LLActRep- BwNot-
		LnkCtl:	ASPM Disabled; Disabled- Retrain- CommClk-
			ExtSynch- ClockPM- AutWidDis- BWInt- AutBWInt-
		LnkSta:	Speed 2.5GT/s, Width x1, TrErr- Train- SlotClk- DLActive- BWMgmt- ABWMgmt-
	Capabilities: [100 v1] Power Budgeting <?>
	Kernel driver in use: pciback

06:00.0 Communication controller: OpenVox Communication Co. Ltd. Device 0810 (rev 14)
	Subsystem: OpenVox Communication Co. Ltd. Device 0001
	Control: I/O- Mem- BusMaster- SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx-
	Status: Cap- 66MHz- UDF- FastB2B- ParErr- DEVSEL=slow >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
	Interrupt: pin A routed to IRQ 16
	Region 0: Memory at f7c00000 (32-bit, non-prefetchable) [disabled] [size=512K]
	Kernel driver in use: pciback

07:00.0 Ethernet controller: Intel Corporation 82574L Gigabit Network Connection
	Subsystem: Intel Corporation Device 0000
	Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx+
	Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
	Latency: 0, Cache Line Size: 64 bytes
	Interrupt: pin A routed to IRQ 16
	Region 0: Memory at f7b40000 (32-bit, non-prefetchable) [size=128K]
	Region 2: I/O ports at e000 [size=32]
	Region 3: Memory at f7b60000 (32-bit, non-prefetchable) [size=16K]
	Expansion ROM at f7b00000 [disabled] [size=256K]
	Capabilities: [c8] Power Management version 2
		Flags: PMEClk- DSI+ D1- D2- AuxCurrent=0mA PME(D0+,D1-,D2-,D3hot+,D3cold+)
		Status: D0 NoSoftRst- PME-Enable- DSel=0 DScale=1 PME-
	Capabilities: [d0] MSI: Enable- Count=1/1 Maskable- 64bit+
		Address: 0000000000000000  Data: 0000
	Capabilities: [e0] Express (v1) Endpoint, MSI 00
		DevCap:	MaxPayload 256 bytes, PhantFunc 0, Latency L0s <512ns, L1 <64us
			ExtTag- AttnBtn- AttnInd- PwrInd- RBE+ FLReset-
		DevCtl:	Report errors: Correctable+ Non-Fatal+ Fatal+ Unsupported+
			RlxdOrd- ExtTag- PhantFunc- AuxPwr- NoSnoop+
			MaxPayload 128 bytes, MaxReadReq 512 bytes
		DevSta:	CorrErr- UncorrErr- FatalErr- UnsuppReq- AuxPwr+ TransPend-
		LnkCap:	Port #0, Speed 2.5GT/s, Width x1, ASPM L0s L1, Latency L0 <128ns, L1 <64us
			ClockPM- Surprise- LLActRep- BwNot-
		LnkCtl:	ASPM Disabled; RCB 64 bytes Disabled- Retrain- CommClk+
			ExtSynch- ClockPM- AutWidDis- BWInt- AutBWInt-
		LnkSta:	Speed 2.5GT/s, Width x1, TrErr- Train- SlotClk+ DLActive- BWMgmt- ABWMgmt-
	Capabilities: [a0] MSI-X: Enable+ Count=3 Masked-
		Vector table: BAR=3 offset=00000000
		PBA: BAR=3 offset=00002000
	Capabilities: [100 v1] Advanced Error Reporting
		UESta:	DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF- MalfTLP- ECRC- UnsupReq- ACSViol-
		UEMsk:	DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF- MalfTLP- ECRC- UnsupReq- ACSViol-
		UESvrt:	DLP+ SDES- TLP- FCP+ CmpltTO- CmpltAbrt- UnxCmplt- RxOF+ MalfTLP+ ECRC- UnsupReq- ACSViol-
		CESta:	RxErr- BadTLP- BadDLLP- Rollover- Timeout- NonFatalErr-
		CEMsk:	RxErr- BadTLP- BadDLLP- Rollover- Timeout- NonFatalErr+
		AERCap:	First Error Pointer: 00, GenCap- CGenEn- ChkCap- ChkEn-
	Kernel driver in use: e1000e

08:00.0 USB controller: NEC Corporation uPD720200 USB 3.0 Host Controller (rev 03) (prog-if 30 [XHCI])
	Subsystem: NEC Corporation uPD720200 USB 3.0 Host Controller
	Control: I/O- Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx+
	Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
	Latency: 0, Cache Line Size: 64 bytes
	Interrupt: pin A routed to IRQ 18
	Region 0: Memory at f7a00000 (64-bit, non-prefetchable) [size=8K]
	Capabilities: [50] Power Management version 3
		Flags: PMEClk- DSI- D1- D2- AuxCurrent=0mA PME(D0+,D1-,D2-,D3hot+,D3cold-)
		Status: D0 NoSoftRst+ PME-Enable- DSel=0 DScale=0 PME-
	Capabilities: [70] MSI: Enable- Count=1/8 Maskable- 64bit+
		Address: 0000000000000000  Data: 0000
	Capabilities: [90] MSI-X: Enable+ Count=8 Masked-
		Vector table: BAR=0 offset=00001000
		PBA: BAR=0 offset=00001080
	Capabilities: [a0] Express (v2) Endpoint, MSI 00
		DevCap:	MaxPayload 128 bytes, PhantFunc 0, Latency L0s unlimited, L1 unlimited
			ExtTag- AttnBtn- AttnInd- PwrInd- RBE+ FLReset-
		DevCtl:	Report errors: Correctable- Non-Fatal- Fatal- Unsupported-
			RlxdOrd- ExtTag- PhantFunc- AuxPwr- NoSnoop+
			MaxPayload 128 bytes, MaxReadReq 512 bytes
		DevSta:	CorrErr- UncorrErr- FatalErr- UnsuppReq- AuxPwr- TransPend-
		LnkCap:	Port #0, Speed 5GT/s, Width x1, ASPM L0s L1, Latency L0 <4us, L1 unlimited
			ClockPM+ Surprise- LLActRep- BwNot-
		LnkCtl:	ASPM Disabled; RCB 64 bytes Disabled- Retrain- CommClk+
			ExtSynch- ClockPM- AutWidDis- BWInt- AutBWInt-
		LnkSta:	Speed 5GT/s, Width x1, TrErr- Train- SlotClk+ DLActive- BWMgmt- ABWMgmt-
		DevCap2: Completion Timeout: Not Supported, TimeoutDis+, LTR+, OBFF Not Supported
		DevCtl2: Completion Timeout: 50us to 50ms, TimeoutDis-, LTR-, OBFF Disabled
		LnkCtl2: Target Link Speed: 5GT/s, EnterCompliance- SpeedDis-
			 Transmit Margin: Normal Operating Range, EnterModifiedCompliance- ComplianceSOS-
			 Compliance De-emphasis: -6dB
		LnkSta2: Current De-emphasis Level: -6dB, EqualizationComplete-, EqualizationPhase1-
			 EqualizationPhase2-, EqualizationPhase3-, LinkEqualizationRequest-
	Capabilities: [100 v1] Advanced Error Reporting
		UESta:	DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF- MalfTLP- ECRC- UnsupReq- ACSViol-
		UEMsk:	DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF- MalfTLP- ECRC- UnsupReq- ACSViol-
		UESvrt:	DLP+ SDES+ TLP- FCP+ CmpltTO- CmpltAbrt- UnxCmplt- RxOF+ MalfTLP+ ECRC- UnsupReq- ACSViol-
		CESta:	RxErr- BadTLP- BadDLLP- Rollover- Timeout- NonFatalErr-
		CEMsk:	RxErr- BadTLP- BadDLLP- Rollover- Timeout- NonFatalErr+
		AERCap:	First Error Pointer: 00, GenCap- CGenEn- ChkCap- ChkEn-
	Capabilities: [140 v1] Device Serial Number ff-ff-ff-ff-ff-ff-ff-ff
	Capabilities: [150 v1] Latency Tolerance Reporting
		Max snoop latency: 0ns
		Max no snoop latency: 0ns
	Kernel driver in use: xhci_hcd
	Kernel modules: xhci_hcd

09:00.0 PCI bridge: Pericom Semiconductor PI7C8148A/PI7C8148B PCI-to-PCI Bridge (prog-if 00 [Normal decode])
	Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx-
	Status: Cap+ 66MHz+ UDF- FastB2B+ ParErr- DEVSEL=medium >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
	Latency: 32, Cache Line Size: 64 bytes
	Bus: primary=09, secondary=0a, subordinate=0a, sec-latency=32
	I/O behind bridge: 0000d000-0000dfff
	Memory behind bridge: f7800000-f78fffff
	Secondary status: 66MHz+ FastB2B+ ParErr- DEVSEL=medium >TAbort- <TAbort- <MAbort- <SERR- <PERR-
	BridgeCtl: Parity- SERR- NoISA- VGA- MAbort- >Reset- FastB2B-
		PriDiscTmr- SecDiscTmr- DiscTmrStat- DiscTmrSERREn-
	Capabilities: [80] Power Management version 2
		Flags: PMEClk- DSI- D1+ D2+ AuxCurrent=0mA PME(D0-,D1-,D2-,D3hot-,D3cold-)
		Status: D0 NoSoftRst- PME-Enable- DSel=0 DScale=0 PME-
		Bridge: PM- B3+
	Capabilities: [90] CompactPCI hot-swap <?>
	Capabilities: [a0] Vital Product Data
		Unknown small resource type 00, will not decode more.
	Kernel driver in use: pciback

09:02.0 Communication controller: Quicklogic Corporation COM-ON-AIR Dosch&Amand DECT (rev 32)
	Subsystem: Device 1786:0001
	Control: I/O- Mem- BusMaster- SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx-
	Status: Cap- 66MHz- UDF- FastB2B+ ParErr- DEVSEL=medium >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
	Interrupt: pin A routed to IRQ 18
	Region 0: Memory at f7900000 (32-bit, non-prefetchable) [disabled] [size=8K]
	Kernel driver in use: pciback

0a:08.0 Ethernet controller: Realtek Semiconductor Co., Ltd. RTL-8139/8139C/8139C+ (rev 10)
	Subsystem: Realtek Semiconductor Co., Ltd. RTL-8139/8139C/8139C+
	Control: I/O- Mem- BusMaster- SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx-
	Status: Cap+ 66MHz- UDF- FastB2B+ ParErr- DEVSEL=medium >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
	Interrupt: pin A routed to IRQ 16
	Region 0: I/O ports at d200 [disabled] [size=256]
	Region 1: Memory at f7842000 (32-bit, non-prefetchable) [disabled] [size=256]
	Expansion ROM at f7830000 [disabled] [size=64K]
	Capabilities: [50] Power Management version 2
		Flags: PMEClk- DSI- D1+ D2+ AuxCurrent=375mA PME(D0-,D1+,D2+,D3hot+,D3cold+)
		Status: D0 NoSoftRst- PME-Enable- DSel=0 DScale=0 PME-
	Kernel driver in use: pciback
	Kernel modules: 8139too

0a:09.0 Ethernet controller: Realtek Semiconductor Co., Ltd. RTL-8139/8139C/8139C+ (rev 10)
	Subsystem: Realtek Semiconductor Co., Ltd. RTL-8139/8139C/8139C+
	Control: I/O- Mem- BusMaster- SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx-
	Status: Cap+ 66MHz- UDF- FastB2B+ ParErr- DEVSEL=medium >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
	Interrupt: pin A routed to IRQ 17
	Region 0: I/O ports at d100 [disabled] [size=256]
	Region 1: Memory at f7841000 (32-bit, non-prefetchable) [disabled] [size=256]
	Expansion ROM at f7820000 [disabled] [size=64K]
	Capabilities: [50] Power Management version 2
		Flags: PMEClk- DSI- D1+ D2+ AuxCurrent=375mA PME(D0-,D1+,D2+,D3hot+,D3cold+)
		Status: D0 NoSoftRst- PME-Enable- DSel=0 DScale=0 PME-
	Kernel driver in use: pciback
	Kernel modules: 8139too

0a:0a.0 Ethernet controller: Realtek Semiconductor Co., Ltd. RTL-8139/8139C/8139C+ (rev 10)
	Subsystem: Realtek Semiconductor Co., Ltd. RTL-8139/8139C/8139C+
	Control: I/O- Mem- BusMaster- SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx-
	Status: Cap+ 66MHz- UDF- FastB2B+ ParErr- DEVSEL=medium >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
	Interrupt: pin A routed to IRQ 18
	Region 0: I/O ports at d000 [disabled] [size=256]
	Region 1: Memory at f7840000 (32-bit, non-prefetchable) [disabled] [size=256]
	Expansion ROM at f7810000 [disabled] [size=64K]
	Capabilities: [50] Power Management version 2
		Flags: PMEClk- DSI- D1+ D2+ AuxCurrent=375mA PME(D0-,D1+,D2+,D3hot+,D3cold+)
		Status: D0 NoSoftRst- PME-Enable- DSel=0 DScale=0 PME-
	Kernel driver in use: pciback
	Kernel modules: 8139too

0a:0b.0 Network controller: Atheros Communications Inc. AR922X Wireless Network Adapter (rev 01)
	Subsystem: Atheros Communications Inc. Device 2093
	Control: I/O- Mem- BusMaster- SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx-
	Status: Cap+ 66MHz+ UDF- FastB2B+ ParErr- DEVSEL=medium >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
	Interrupt: pin A routed to IRQ 19
	Region 0: Memory at f7800000 (32-bit, non-prefetchable) [disabled] [size=64K]
	Capabilities: [44] Power Management version 2
		Flags: PMEClk- DSI- D1- D2- AuxCurrent=100mA PME(D0+,D1-,D2-,D3hot+,D3cold-)
		Status: D0 NoSoftRst- PME-Enable- DSel=0 DScale=0 PME-
	Kernel driver in use: pciback


[-- Attachment #4: lspci-tv --]
[-- Type: text/plain, Size: 1964 bytes --]

-[0000:00]-+-00.0  Intel Corporation Xeon E3-1200 Processor Family DRAM Controller
           +-01.0-[01]--
           +-01.1-[02-03]----00.0-[03]----00.0  OpenVox Communication Co. Ltd. Device 0100
           +-02.0  Intel Corporation Xeon E3-1200 Processor Family Integrated Graphics Controller
           +-06.0-[04]----00.0  Atheros Communications Inc. AR9300 Wireless LAN adaptor
           +-16.0  Intel Corporation 6 Series/C200 Series Chipset Family MEI Controller #1
           +-16.3  Intel Corporation 6 Series/C200 Series Chipset Family KT Controller
           +-19.0  Intel Corporation 82579LM Gigabit Network Connection
           +-1a.0  Intel Corporation 6 Series/C200 Series Chipset Family USB Enhanced Host Controller #2
           +-1b.0  Intel Corporation 6 Series/C200 Series Chipset Family High Definition Audio Controller
           +-1c.0-[05-06]----00.0-[06]----00.0  OpenVox Communication Co. Ltd. Device 0810
           +-1c.4-[07]----00.0  Intel Corporation 82574L Gigabit Network Connection
           +-1c.6-[08]----00.0  NEC Corporation uPD720200 USB 3.0 Host Controller
           +-1d.0  Intel Corporation 6 Series/C200 Series Chipset Family USB Enhanced Host Controller #1
           +-1e.0-[09-0a]--+-00.0-[0a]--+-08.0  Realtek Semiconductor Co., Ltd. RTL-8139/8139C/8139C+
           |               |            +-09.0  Realtek Semiconductor Co., Ltd. RTL-8139/8139C/8139C+
           |               |            +-0a.0  Realtek Semiconductor Co., Ltd. RTL-8139/8139C/8139C+
           |               |            \-0b.0  Atheros Communications Inc. AR922X Wireless Network Adapter
           |               \-02.0  Quicklogic Corporation COM-ON-AIR Dosch&Amand DECT
           +-1f.0  Intel Corporation C206 Chipset Family LPC Controller
           +-1f.2  Intel Corporation 6 Series/C200 Series Chipset Family SATA AHCI Controller
           \-1f.3  Intel Corporation 6 Series/C200 Series Chipset Family SMBus Controller

[-- Attachment #5: Type: text/plain, Size: 126 bytes --]

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

^ permalink raw reply	[flat|nested] 28+ messages in thread

* Re: Powerdown problem on XEN | ACPI S5
  2013-08-14 19:39                 ` Atom2
@ 2013-08-14 20:18                   ` Andrew Cooper
  2013-08-14 20:24                     ` Atom2
  0 siblings, 1 reply; 28+ messages in thread
From: Andrew Cooper @ 2013-08-14 20:18 UTC (permalink / raw)
  To: Atom2; +Cc: xen-devel, Ian Campbell, Jan Beulich

On 14/08/13 20:39, Atom2 wrote:
> Hi Andrew,
> all requested information is attached, please see notes below.
>
> My ongoing & continued thanks anyway ...
>
> Am 14.08.13 21:18, schrieb Andrew Cooper:
> [...]
>> Hmm - curious.  That would indicate that the timeout occurs during the
>> very first attempt to shut down the iommu
>>
>> Can you grab the full log with "iommu=1,debug,verbose" ?
> Sure, please see attachment "XEN console (debug, verbose)"
>>
>> Can you also please attach "lspci -vv" and "lspci -tv" from dom0
> No problem - please see attachements "lspci-vv" and "lspci-tv"
>>
>> ~Andrew
>>

Ok thanks.

Do you mind confirming whether S5 works with "iommu=off" on the Xen
command line?

~Andrew

^ permalink raw reply	[flat|nested] 28+ messages in thread

* Re: Powerdown problem on XEN | ACPI S5
  2013-08-14 20:18                   ` Andrew Cooper
@ 2013-08-14 20:24                     ` Atom2
  2013-08-14 20:30                       ` Atom2
                                         ` (2 more replies)
  0 siblings, 3 replies; 28+ messages in thread
From: Atom2 @ 2013-08-14 20:24 UTC (permalink / raw)
  To: Andrew Cooper; +Cc: xen-devel, Ian Campbell, Jan Beulich

Am 14.08.13 22:18, schrieb Andrew Cooper:
[...]
>
> Ok thanks.
>
> Do you mind confirming whether S5 works with "iommu=off" on the Xen
> command line?
>
> ~Andrew
>
Yes, that works: The system powers off after issuing
	shutdown -h now
from the dom0.

^ permalink raw reply	[flat|nested] 28+ messages in thread

* Re: Powerdown problem on XEN | ACPI S5
  2013-08-14 20:24                     ` Atom2
@ 2013-08-14 20:30                       ` Atom2
  2013-08-14 20:34                       ` Ben Guthro
  2013-08-14 20:38                       ` Andrew Cooper
  2 siblings, 0 replies; 28+ messages in thread
From: Atom2 @ 2013-08-14 20:30 UTC (permalink / raw)
  To: Andrew Cooper; +Cc: xen-devel, Ian Campbell, Jan Beulich

[-- Attachment #1: Type: text/plain, Size: 642 bytes --]

And just to provide full information, attached please find the serial 
log file when iommu=off is set on the XEN command line via grub.

This obviously (and expectedly) turns off I/O virtualization.

Am 14.08.13 22:24, schrieb Atom2:
> Am 14.08.13 22:18, schrieb Andrew Cooper:
> [...]
>>
>> Ok thanks.
>>
>> Do you mind confirming whether S5 works with "iommu=off" on the Xen
>> command line?
>>
>> ~Andrew
>>
> Yes, that works: The system powers off after issuing
>      shutdown -h now
> from the dom0.
>
> _______________________________________________
> Xen-devel mailing list
> Xen-devel@lists.xen.org
> http://lists.xen.org/xen-devel

[-- Attachment #2: XEN console (iommu=off) --]
[-- Type: text/plain, Size: 9511 bytes --]

 \ \/ /___ _ __   | || |  |___ \  |___ \ 
  \  // _ \ '_ \  | || |_   __) |   __) |
  /  \  __/ | | | |__   _| / __/ _ / __/ 
 /_/\_\___|_| |_|    |_|(_)_____(_)_____|

(XEN) Xen version 4.2.2 (root@herrenhauspark.com) (gcc (Gentoo Hardened 4.6.3 p1.13, pie-0.5.2) 4.6.3) Sat Aug 17 20:43:43 CEST 2013
(XEN) Latest ChangeSet: unavailable
(XEN) Bootloader: GNU GRUB 0.97
(XEN) Command line: console=com1 loglvl=all com1=115200,8n1,0x2d8,10 dom0_mem=512M,max:512M dom0_max_vcpus=1 dom0_vcpus_pin iommu=off
(XEN) Video information:
(XEN)  VGA is text mode 80x25, font 8x16
(XEN)  VBE/DDC methods: V2; EDID transfer time: 1 seconds
(XEN) Disc information:
(XEN)  Found 2 MBR signatures
(XEN)  Found 2 EDD information structures
(XEN) Xen-e820 RAM map:
(XEN)  0000000000000000 - 0000000000099c00 (usable)
(XEN)  0000000000099c00 - 00000000000a0000 (reserved)
(XEN)  00000000000e0000 - 0000000000100000 (reserved)
(XEN)  0000000000100000 - 0000000020000000 (usable)
(XEN)  0000000020000000 - 0000000020200000 (reserved)
(XEN)  0000000020200000 - 0000000040000000 (usable)
(XEN)  0000000040000000 - 0000000040200000 (reserved)
(XEN)  0000000040200000 - 00000000db9f1000 (usable)
(XEN)  00000000db9f1000 - 00000000dc0db000 (reserved)
(XEN)  00000000dc0db000 - 00000000dc1fa000 (ACPI NVS)
(XEN)  00000000dc1fa000 - 00000000dc652000 (reserved)
(XEN)  00000000dc652000 - 00000000dc653000 (usable)
(XEN)  00000000dc653000 - 00000000dc696000 (ACPI NVS)
(XEN)  00000000dc696000 - 00000000dcdba000 (usable)
(XEN)  00000000dcdba000 - 00000000dcff2000 (reserved)
(XEN)  00000000dcff2000 - 00000000dd000000 (usable)
(XEN)  00000000dd800000 - 00000000dfa00000 (reserved)
(XEN)  00000000f8000000 - 00000000fc000000 (reserved)
(XEN)  00000000fec00000 - 00000000fec01000 (reserved)
(XEN)  00000000fed00000 - 00000000fed04000 (reserved)
(XEN)  00000000fed1c000 - 00000000fed20000 (reserved)
(XEN)  00000000fee00000 - 00000000fee01000 (reserved)
(XEN)  00000000ff000000 - 0000000100000000 (reserved)
(XEN)  0000000100000000 - 000000081e600000 (usable)
(XEN) ACPI: RSDP 000F0490, 0024 (r2 ALASKA)
(XEN) ACPI: XSDT DC1EA078, 0074 (r1 ALASKA    A M I  1072009 AMI     10013)
(XEN) ACPI: FACP DC1F4710, 00F4 (r4 ALASKA    A M I  1072009 AMI     10013)
(XEN) ACPI: DSDT DC1EA188, A587 (r2 ALASKA    A M I        1 INTL 20051117)
(XEN) ACPI: FACS DC1F8F80, 0040
(XEN) ACPI: APIC DC1F4808, 0092 (r3 ALASKA    A M I  1072009 AMI     10013)
(XEN) ACPI: FPDT DC1F48A0, 0044 (r1 ALASKA    A M I  1072009 AMI     10013)
(XEN) ACPI: MCFG DC1F48E8, 003C (r1 ALASKA    A M I  1072009 MSFT       97)
(XEN) ACPI: HPET DC1F4928, 0038 (r1 ALASKA    A M I  1072009 AMI.        5)
(XEN) ACPI: SSDT DC1F4960, 036D (r1 SataRe SataTabl     1000 INTL 20091112)
(XEN) ACPI: SSDT DC1F4CD0, 081E (r1  PmRef  Cpu0Ist     3000 INTL 20051117)
(XEN) ACPI: SSDT DC1F54F0, 0A92 (r1  PmRef    CpuPm     3000 INTL 20051117)
(XEN) ACPI: DMAR DC1F5F88, 00B0 (r1 INTEL      SNB         1 INTL        1)
(XEN) ACPI: ASF! DC1F6038, 00A5 (r32 INTEL       HCG        1 TFSM    F4240)
(XEN) System RAM: 32674MB (33458932kB)
(XEN) No NUMA configuration found
(XEN) Faking a node at 0000000000000000-000000081e600000
(XEN) Domain heap initialised
(XEN) found SMP MP-table at 000fd8d0
(XEN) DMI 2.7 present.
(XEN) Using APIC driver default
(XEN) ACPI: PM-Timer IO Port: 0x408
(XEN) ACPI: ACPI SLEEP INFO: pm1x_cnt[404,0], pm1x_evt[400,0]
(XEN) ACPI: 32/64X FACS address mismatch in FADT - dc1f8f80/0000000000000000, using 32
(XEN) ACPI:                  wakeup_vec[dc1f8f8c], vec_size[20]
(XEN) ACPI: Local APIC address 0xfee00000
(XEN) ACPI: LAPIC (acpi_id[0x01] lapic_id[0x00] enabled)
(XEN) Processor #0 6:10 APIC version 21
(XEN) ACPI: LAPIC (acpi_id[0x02] lapic_id[0x02] enabled)
(XEN) Processor #2 6:10 APIC version 21
(XEN) ACPI: LAPIC (acpi_id[0x03] lapic_id[0x04] enabled)
(XEN) Processor #4 6:10 APIC version 21
(XEN) ACPI: LAPIC (acpi_id[0x04] lapic_id[0x06] enabled)
(XEN) Processor #6 6:10 APIC version 21
(XEN) ACPI: LAPIC (acpi_id[0x05] lapic_id[0x01] enabled)
(XEN) Processor #1 6:10 APIC version 21
(XEN) ACPI: LAPIC (acpi_id[0x06] lapic_id[0x03] enabled)
(XEN) Processor #3 6:10 APIC version 21
(XEN) ACPI: LAPIC (acpi_id[0x07] lapic_id[0x05] enabled)
(XEN) Processor #5 6:10 APIC version 21
(XEN) ACPI: LAPIC (acpi_id[0x08] lapic_id[0x07] enabled)
(XEN) Processor #7 6:10 APIC version 21
(XEN) ACPI: LAPIC_NMI (acpi_id[0xff] high edge lint[0x1])
(XEN) ACPI: IOAPIC (id[0x02] address[0xfec00000] gsi_base[0])
(XEN) IOAPIC[0]: apic_id 2, version 32, address 0xfec00000, GSI 0-23
(XEN) ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl)
(XEN) ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level)
(XEN) ACPI: IRQ0 used by override.
(XEN) ACPI: IRQ2 used by override.
(XEN) ACPI: IRQ9 used by override.
(XEN) Enabling APIC mode:  Flat.  Using 1 I/O APICs
(XEN) ACPI: HPET id: 0x8086a701 base: 0xfed00000
(XEN) ERST table was not found
(XEN) Using ACPI (MADT) for SMP configuration information
(XEN) SMP: Allowing 8 CPUs (0 hotplug CPUs)
(XEN) IRQ limits: 24 GSI, 1528 MSI/MSI-X
(XEN) Not enabling x2APIC: depends on iommu_supports_eim.
(XEN) Using scheduler: SMP Credit Scheduler (credit)
(XEN) Detected 2394.615 MHz processor.
(XEN) Initing memory sharing.
(XEN) xstate_init: using cntxt_size: 0x340 and states: 0x7
(XEN) mce_intel.c:1238: MCA Capability: BCAST 1 SER 0 CMCI 1 firstbank 0 extended MCE MSR 0
(XEN) Intel machine check reporting enabled
(XEN) PCI: MCFG configuration 0: base f8000000 segment 0000 buses 00 - 3f
(XEN) PCI: MCFG area at f8000000 reserved in E820
(XEN) PCI: Using MCFG for segment 0000 bus 00-3f
(XEN) I/O virtualisation disabled
(XEN) Enabled directed EOI with ioapic_ack_old on!
(XEN) ENABLING IO-APIC IRQs
(XEN)  -> Using old ACK method
(XEN) ..TIMER: vector=0xF0 apic1=0 pin1=2 apic2=-1 pin2=-1
(XEN) TSC deadline timer enabled
(XEN) Platform timer is 14.318MHz HPET
(XEN) Allocated console ring of 64 KiB.
(XEN) VMX: Supported advanced features:
(XEN)  - APIC MMIO access virtualisation
(XEN)  - APIC TPR shadow
(XEN)  - Extended Page Tables (EPT)
(XEN)  - Virtual-Processor Identifiers (VPID)
(XEN)  - Virtual NMI
(XEN)  - MSR direct-access bitmap
(XEN)  - Unrestricted Guest
(XEN) HVM: ASIDs enabled.
(XEN) HVM: VMX enabled
(XEN) HVM: Hardware Assisted Paging (HAP) detected
(XEN) HVM: HAP page sizes: 4kB, 2MB
(XEN) Brought up 8 CPUs
(XEN) ACPI sleep modes: S3
(XEN) mcheck_poll: Machine check polling timer started.
(XEN) *** LOADING DOMAIN 0 ***
(XEN)  Xen  kernel: 64-bit, lsb, compat32
(XEN)  Dom0 kernel: 64-bit, PAE, lsb, paddr 0x1000000 -> 0x1c00000
(XEN) PHYSICAL MEMORY ARRANGEMENT:
(XEN)  Dom0 alloc.:   0000000804000000->0000000806000000 (122880 pages to be allocated)
(XEN) VIRTUAL MEMORY ARRANGEMENT:
(XEN)  Loaded kernel: ffffffff81000000->ffffffff81c00000
(XEN)  Init. ramdisk: ffffffff81c00000->ffffffff81c00000
(XEN)  Phys-Mach map: ffffffff81c00000->ffffffff81d00000
(XEN)  Start info:    ffffffff81d00000->ffffffff81d004b4
(XEN)  Page tables:   ffffffff81d01000->ffffffff81d14000
(XEN)  Boot stack:    ffffffff81d14000->ffffffff81d15000
(XEN)  TOTAL:         ffffffff80000000->ffffffff82000000
(XEN)  ENTRY ADDRESS: ffffffff815df1e0
(XEN) Dom0 has maximum 1 VCPUs
(XEN) Scrubbing Free RAM: ..............................................................................................................................................................................................................................................................................................................................done.
(XEN) Initial low memory virq threshold set at 0x4000 pages.
(XEN) Std. Loglevel: All
(XEN) Guest Loglevel: Nothing (Rate-limited: Errors and warnings)
(XEN) *** Serial input -> DOM0 (type 'CTRL-a' three times to switch input to Xen)
(XEN) Freed 268kB init memory.
(XEN) PCI add device 0000:00:00.0
(XEN) PCI add device 0000:00:01.0
(XEN) PCI add device 0000:00:01.1
(XEN) PCI add device 0000:00:02.0
(XEN) PCI add device 0000:00:06.0
(XEN) PCI add device 0000:00:16.0
(XEN) PCI add device 0000:00:16.3
(XEN) PCI add device 0000:00:19.0
(XEN) PCI add device 0000:00:1a.0
(XEN) PCI add device 0000:00:1b.0
(XEN) PCI add device 0000:00:1c.0
(XEN) PCI add device 0000:00:1c.4
(XEN) PCI add device 0000:00:1c.6
(XEN) PCI add device 0000:00:1d.0
(XEN) PCI add device 0000:00:1e.0
(XEN) PCI add device 0000:00:1f.0
(XEN) PCI add device 0000:00:1f.2
(XEN) PCI add device 0000:00:1f.3
(XEN) PCI add device 0000:02:00.0
(XEN) PCI add device 0000:03:00.0
(XEN) PCI add device 0000:04:00.0
(XEN) PCI add device 0000:05:00.0
(XEN) PCI add device 0000:06:00.0
(XEN) PCI add device 0000:07:00.0
(XEN) PCI add device 0000:08:00.0
(XEN) PCI add device 0000:09:00.0
(XEN) PCI add device 0000:09:02.0
(XEN) PCI add device 0000:0a:08.0
(XEN) PCI add device 0000:0a:09.0
(XEN) PCI add device 0000:0a:0a.0
(XEN) PCI add device 0000:0a:0b.0
(XEN) traps.c:2584:d0 Domain attempted WRMSR 00000000000001fc from 0x000000000004005f to 0x000000000004005d.
(XEN) Monitor-Mwait will be used to enter C1 state
(XEN) Monitor-Mwait will be used to enter C2 state
(XEN) Monitor-Mwait will be used to enter C3 state
(XEN) Preparing system for ACPI S5 state.
(XEN) Disabling non-boot CPUs ...
(XEN) Entering ACPI S5 state.
(XEN) After local_irq_save
(XEN) After spin_debug_disable
(XEN) After time_suspend
(XEN) After li8259_suspend
(XEN) After ioapic_suspend
(XEN) After iommu_suspend
(XEN) After lapic_suspend
(XEN) Before ACPI_FLUSH_CPU_CACHE
(XEN) After ACPI_FLUSH_CPU_CACHE

[-- Attachment #3: Type: text/plain, Size: 126 bytes --]

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

^ permalink raw reply	[flat|nested] 28+ messages in thread

* Re: Powerdown problem on XEN | ACPI S5
  2013-08-14 20:24                     ` Atom2
  2013-08-14 20:30                       ` Atom2
@ 2013-08-14 20:34                       ` Ben Guthro
  2013-08-14 20:37                         ` Konrad Rzeszutek Wilk
  2013-08-14 20:38                       ` Andrew Cooper
  2 siblings, 1 reply; 28+ messages in thread
From: Ben Guthro @ 2013-08-14 20:34 UTC (permalink / raw)
  To: Atom2; +Cc: Andrew Cooper, Ian Campbell, Jan Beulich, xen-devel


[-- Attachment #1.1: Type: text/plain, Size: 1031 bytes --]

This hang looks exactly like this one that I posted about in June:
http://www.gossamer-threads.com/lists/xen/devel/284943?do=post_view_threaded#284943

This ended up being an interaction with the i915 driver in the kernel, and
pvops use of PAT.

If you revert the following linux commit:
http://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.git/commit/?id=c79c49826270b8b0061b2fca840fc3f013c8a78a

and then apply:
https://lkml.org/lkml/2012/2/10/229

You may get good results.
It helped me with a similar problem, at least.

Ben



On Wed, Aug 14, 2013 at 4:24 PM, Atom2 <ariel.atom2@web2web.at> wrote:

> Am 14.08.13 22:18, schrieb Andrew Cooper:
> [...]
>
>>
>> Ok thanks.
>>
>> Do you mind confirming whether S5 works with "iommu=off" on the Xen
>> command line?
>>
>> ~Andrew
>>
>>  Yes, that works: The system powers off after issuing
>         shutdown -h now
> from the dom0.
>
>
> ______________________________**_________________
> Xen-devel mailing list
> Xen-devel@lists.xen.org
> http://lists.xen.org/xen-devel
>

[-- Attachment #1.2: Type: text/html, Size: 2171 bytes --]

[-- Attachment #2: Type: text/plain, Size: 126 bytes --]

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

^ permalink raw reply	[flat|nested] 28+ messages in thread

* Re: Powerdown problem on XEN | ACPI S5
  2013-08-14 20:34                       ` Ben Guthro
@ 2013-08-14 20:37                         ` Konrad Rzeszutek Wilk
  2013-08-14 21:56                           ` Atom2
  0 siblings, 1 reply; 28+ messages in thread
From: Konrad Rzeszutek Wilk @ 2013-08-14 20:37 UTC (permalink / raw)
  To: Ben Guthro; +Cc: Andrew Cooper, xen-devel, Atom2, Jan Beulich, Ian Campbell

On Wed, Aug 14, 2013 at 04:34:21PM -0400, Ben Guthro wrote:
> This hang looks exactly like this one that I posted about in June:
> http://www.gossamer-threads.com/lists/xen/devel/284943?do=post_view_threaded#284943
> 
> This ended up being an interaction with the i915 driver in the kernel, and
> pvops use of PAT.
> 
> If you revert the following linux commit:
> http://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.git/commit/?id=c79c49826270b8b0061b2fca840fc3f013c8a78a

And probably also need to revert
8eaffa67b43e99ae581622c5133e20b0f48bcef1
?
> 
> and then apply:
> https://lkml.org/lkml/2012/2/10/229
> 
> You may get good results.
> It helped me with a similar problem, at least.
> 
> Ben
> 
> 
> 
> On Wed, Aug 14, 2013 at 4:24 PM, Atom2 <ariel.atom2@web2web.at> wrote:
> 
> > Am 14.08.13 22:18, schrieb Andrew Cooper:
> > [...]
> >
> >>
> >> Ok thanks.
> >>
> >> Do you mind confirming whether S5 works with "iommu=off" on the Xen
> >> command line?
> >>
> >> ~Andrew
> >>
> >>  Yes, that works: The system powers off after issuing
> >         shutdown -h now
> > from the dom0.
> >
> >
> > ______________________________**_________________
> > Xen-devel mailing list
> > Xen-devel@lists.xen.org
> > http://lists.xen.org/xen-devel
> >

> _______________________________________________
> Xen-devel mailing list
> Xen-devel@lists.xen.org
> http://lists.xen.org/xen-devel

^ permalink raw reply	[flat|nested] 28+ messages in thread

* Re: Powerdown problem on XEN | ACPI S5
  2013-08-14 20:37                         ` Konrad Rzeszutek Wilk
@ 2013-08-14 21:56                           ` Atom2
  2013-08-15  1:58                             ` Ben Guthro
  2013-08-15 13:40                             ` Konrad Rzeszutek Wilk
  0 siblings, 2 replies; 28+ messages in thread
From: Atom2 @ 2013-08-14 21:56 UTC (permalink / raw)
  To: Konrad Rzeszutek Wilk
  Cc: Andrew Cooper, Ian Campbell, Ben Guthro, Jan Beulich, xen-devel

Hi Ben & Konrad,
thanks for your suggestions. I have to admit, I don't fully understand 
the commit system yet and don't know how to apply that to my gentoo 
sources. But I read through the thread that Ben has posted and thought, 
given that Ben had linked the issue to the Intel i915 driver, I'll try 
something that crossed my mind:

lspci lists the Intel IGD as follows:
00:02.0 VGA compatible controller: Intel Corporation Xeon E3-1200 
Processor Family Integrated Graphics Controller (rev 09)

So my thought process was if the IGD is to blame, I'll just exclude that 
from the dom0 by adding it to the xen-pciback.hide list on the module 
line of grub. This line now reads as follows (NOTE: all other devices 
except the first one (00:02.0) have been there already before my latest 
try):

module  /boot/gentoo-3.9.5-hardened-xen rootfstype=ext4 root=/dev/sda6 \

xen-pciback.hide=(00:02.0)(00:1b.0)(02:00.0)(03:00.0)(04:00.0)(05:00.0)(06:00.0)(09:00.0)(09:02.0)(0a:08.0)(0a:09.0)(0a:0a.0
)(0a:0b.0)

Clearly this meant that after the dom0 (gentoo hardened) was started, 
there was no more console output available. So I had to use ssh to 
login. But once I started a (remote) 'shutdown -h now' from the login 
bash and ... the system actually powerd off.

I guess this really proves that the (in-kernel) i915-driver which, given 
that the IGD is no longer available, could not have been active in the 
dom0, is to blame for the powerdown issue.

I am not sure whether that brings us any further in resolving the issue, 
but it might make sense to involve someone responsibel for the Intel IGD 
driver? Also I could not find much documentation for the i915.modeset 
kernel parameter- probably there's something in there that could help?

What are your thoughts on this.

And as always thanks for your input so far and also thanks in advance 
for your continued help and ideas.

Am 14.08.13 22:37, schrieb Konrad Rzeszutek Wilk:
> On Wed, Aug 14, 2013 at 04:34:21PM -0400, Ben Guthro wrote:
>> This hang looks exactly like this one that I posted about in June:
>> http://www.gossamer-threads.com/lists/xen/devel/284943?do=post_view_threaded#284943
>>
>> This ended up being an interaction with the i915 driver in the kernel, and
>> pvops use of PAT.
>>
>> If you revert the following linux commit:
>> http://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.git/commit/?id=c79c49826270b8b0061b2fca840fc3f013c8a78a
>
> And probably also need to revert
> 8eaffa67b43e99ae581622c5133e20b0f48bcef1
> ?
>>
>> and then apply:
>> https://lkml.org/lkml/2012/2/10/229
>>
>> You may get good results.
>> It helped me with a similar problem, at least.
>>
>> Ben
>>
>>
>>
>> On Wed, Aug 14, 2013 at 4:24 PM, Atom2 <ariel.atom2@web2web.at> wrote:
>>
>>> Am 14.08.13 22:18, schrieb Andrew Cooper:
>>> [...]
>>>
>>>>
>>>> Ok thanks.
>>>>
>>>> Do you mind confirming whether S5 works with "iommu=off" on the Xen
>>>> command line?
>>>>
>>>> ~Andrew
>>>>
>>>>   Yes, that works: The system powers off after issuing
>>>          shutdown -h now
>>> from the dom0.
>>>
>>>
>>> ______________________________**_________________
>>> Xen-devel mailing list
>>> Xen-devel@lists.xen.org
>>> http://lists.xen.org/xen-devel
>>>
>
>> _______________________________________________
>> Xen-devel mailing list
>> Xen-devel@lists.xen.org
>> http://lists.xen.org/xen-devel
>
>
> _______________________________________________
> Xen-devel mailing list
> Xen-devel@lists.xen.org
> http://lists.xen.org/xen-devel
>

^ permalink raw reply	[flat|nested] 28+ messages in thread

* Re: Powerdown problem on XEN | ACPI S5
  2013-08-14 21:56                           ` Atom2
@ 2013-08-15  1:58                             ` Ben Guthro
  2013-08-15 19:28                               ` Atom2
  2013-08-15 13:40                             ` Konrad Rzeszutek Wilk
  1 sibling, 1 reply; 28+ messages in thread
From: Ben Guthro @ 2013-08-15  1:58 UTC (permalink / raw)
  To: Atom2; +Cc: Andrew Cooper, Ian Campbell, xen-devel, Jan Beulich


[-- Attachment #1.1: Type: text/plain, Size: 4639 bytes --]

Hi,

I admit, I don't know how the gentoo build system works, but the general
idea here is that you want to revert those 2 commits, and apply the third.

If you don't have a git tree, you can download the two commits from these
two links
http://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.git/patch/?id=c79c49826270b8b0061b2fca840fc3f013c8a78a
http://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.git/patch/?id=8eaffa67b43e99ae581622c5133e20b0f48bcef1

You'll want to apply them in reverse
patch -p1 -R < c79c498.patch
patch -p1 -R < 8eaffa67.patch

Then apply the patch from
https://lkml.org/lkml/2012/2/10/229


I hope this helps.

Ben

On Wed, Aug 14, 2013 at 5:56 PM, Atom2 <ariel.atom2@web2web.at> wrote:

> Hi Ben & Konrad,
> thanks for your suggestions. I have to admit, I don't fully understand the
> commit system yet and don't know how to apply that to my gentoo sources.
> But I read through the thread that Ben has posted and thought, given that
> Ben had linked the issue to the Intel i915 driver, I'll try something that
> crossed my mind:
>
> lspci lists the Intel IGD as follows:
> 00:02.0 VGA compatible controller: Intel Corporation Xeon E3-1200
> Processor Family Integrated Graphics Controller (rev 09)
>
> So my thought process was if the IGD is to blame, I'll just exclude that
> from the dom0 by adding it to the xen-pciback.hide list on the module line
> of grub. This line now reads as follows (NOTE: all other devices except the
> first one (00:02.0) have been there already before my latest try):
>
> module  /boot/gentoo-3.9.5-hardened-**xen rootfstype=ext4 root=/dev/sda6 \
>
> xen-pciback.hide=(00:02.0)(00:**1b.0)(02:00.0)(03:00.0)(04:00.**
> 0)(05:00.0)(06:00.0)(09:00.0)(**09:02.0)(0a:08.0)(0a:09.0)(0a:**0a.0
> )(0a:0b.0)
>
> Clearly this meant that after the dom0 (gentoo hardened) was started,
> there was no more console output available. So I had to use ssh to login.
> But once I started a (remote) 'shutdown -h now' from the login bash and ...
> the system actually powerd off.
>
> I guess this really proves that the (in-kernel) i915-driver which, given
> that the IGD is no longer available, could not have been active in the
> dom0, is to blame for the powerdown issue.
>
> I am not sure whether that brings us any further in resolving the issue,
> but it might make sense to involve someone responsibel for the Intel IGD
> driver? Also I could not find much documentation for the i915.modeset
> kernel parameter- probably there's something in there that could help?
>
> What are your thoughts on this.
>
> And as always thanks for your input so far and also thanks in advance for
> your continued help and ideas.
>
> Am 14.08.13 22:37, schrieb Konrad Rzeszutek Wilk:
>
>  On Wed, Aug 14, 2013 at 04:34:21PM -0400, Ben Guthro wrote:
>>
>>> This hang looks exactly like this one that I posted about in June:
>>> http://www.gossamer-threads.**com/lists/xen/devel/284943?do=**
>>> post_view_threaded#284943<http://www.gossamer-threads.com/lists/xen/devel/284943?do=post_view_threaded#284943>
>>>
>>> This ended up being an interaction with the i915 driver in the kernel,
>>> and
>>> pvops use of PAT.
>>>
>>> If you revert the following linux commit:
>>> http://git.kernel.org/cgit/**linux/kernel/git/torvalds/**
>>> linux.git/commit/?id=**c79c49826270b8b0061b2fca840fc3**f013c8a78a<http://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.git/commit/?id=c79c49826270b8b0061b2fca840fc3f013c8a78a>
>>>
>>
>> And probably also need to revert
>> 8eaffa67b43e99ae581622c5133e20**b0f48bcef1
>> ?
>>
>>>
>>> and then apply:
>>> https://lkml.org/lkml/2012/2/**10/229<https://lkml.org/lkml/2012/2/10/229>
>>>
>>> You may get good results.
>>> It helped me with a similar problem, at least.
>>>
>>> Ben
>>>
>>>
>>>
>>> On Wed, Aug 14, 2013 at 4:24 PM, Atom2 <ariel.atom2@web2web.at> wrote:
>>>
>>>  Am 14.08.13 22:18, schrieb Andrew Cooper:
>>>> [...]
>>>>
>>>>
>>>>> Ok thanks.
>>>>>
>>>>> Do you mind confirming whether S5 works with "iommu=off" on the Xen
>>>>> command line?
>>>>>
>>>>> ~Andrew
>>>>>
>>>>>   Yes, that works: The system powers off after issuing
>>>>>
>>>>          shutdown -h now
>>>> from the dom0.
>>>>
>>>>
>>>> ______________________________****_________________
>>>> Xen-devel mailing list
>>>> Xen-devel@lists.xen.org
>>>> http://lists.xen.org/xen-devel
>>>>
>>>>
>>  ______________________________**_________________
>>> Xen-devel mailing list
>>> Xen-devel@lists.xen.org
>>> http://lists.xen.org/xen-devel
>>>
>>
>>
>> ______________________________**_________________
>> Xen-devel mailing list
>> Xen-devel@lists.xen.org
>> http://lists.xen.org/xen-devel
>>
>>

[-- Attachment #1.2: Type: text/html, Size: 6919 bytes --]

[-- Attachment #2: Type: text/plain, Size: 126 bytes --]

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

^ permalink raw reply	[flat|nested] 28+ messages in thread

* Re: Powerdown problem on XEN | ACPI S5
  2013-08-15  1:58                             ` Ben Guthro
@ 2013-08-15 19:28                               ` Atom2
  2013-08-15 20:26                                 ` Konrad Rzeszutek Wilk
  0 siblings, 1 reply; 28+ messages in thread
From: Atom2 @ 2013-08-15 19:28 UTC (permalink / raw)
  To: Ben Guthro; +Cc: Andrew Cooper, Ian Campbell, xen-devel, Jan Beulich

Hi guys,
thanks for your further input.

Following through Ben's mail below and Konrad's later mail suggesting 
the same, I tried to get these patches in. I'd however require your help 
before I feel I can safely proceed.

Please see below:

Am 15.08.13 03:58, schrieb Ben Guthro:
[...]
> I admit, I don't know how the gentoo build system works, but the general
> idea here is that you want to revert those 2 commits, and apply the third.
>
> If you don't have a git tree, you can download the two commits from
> these two links
> http://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.git/patch/?id=c79c49826270b8b0061b2fca840fc3f013c8a78a
> http://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.git/patch/?id=8eaffa67b43e99ae581622c5133e20b0f48bcef1
>
> You'll want to apply them in reverse
After consultation with the manual I decided to give it a dry-run before 
and check with you guys first. First of all, I assume I'm righht that 
this is a patch to the *linux kernel* and not the xen-sources as I could 
not find the referenced files in the xen tree.

> patch -p1 -R < c79c498.patch
vm-host # patch --dry-run -p1 -R < c79c498.patch
patching file arch/x86/xen/enlighten.c
Hunk #2 succeeded at 1431 (offset 14 lines).

I am slightly worried about the last message, not so much about the 
offset, but rather only the "Hunk #2" success. Why is there no "Hunk #1" 
when there's a "Hunk #2"?

> patch -p1 -R < 8eaffa67.patch
vm-host # patch --dry-run -p1 -R < 8eaffa67.patch
patching file arch/x86/xen/enlighten.c
Hunk #1 succeeded at 1367 (offset 226 lines).
patching file arch/x86/xen/mmu.c
Hunk #1 succeeded at 434 (offset 19 lines).
Hunk #2 succeeded at 482 (offset 19 lines).
Hunk #3 succeeded at 495 (offset 19 lines).

That seems to be o.k. from my understanding?
>
> Then apply the patch from
> https://lkml.org/lkml/2012/2/10/229
For this patch I copied the complete text from the https address above 
and copied it to a file named 229.patch. Then I issued the following 
command:
vm-host # patch --dry-run -p1 -R < 229.patch
patching file arch/x86/include/asm/pgtable.h
Unreversed patch detected!  Ignore -R? [n]

I am not sure what to make out of this? Could you please provide some input.

Thanks and sorry for those probably dumb questions. I'm new to this 
(automated) patching thing, and with a little help, the first time 
usually works out well.

^ permalink raw reply	[flat|nested] 28+ messages in thread

* Re: Powerdown problem on XEN | ACPI S5
  2013-08-15 19:28                               ` Atom2
@ 2013-08-15 20:26                                 ` Konrad Rzeszutek Wilk
  2013-08-15 21:39                                   ` Atom2
  0 siblings, 1 reply; 28+ messages in thread
From: Konrad Rzeszutek Wilk @ 2013-08-15 20:26 UTC (permalink / raw)
  To: Atom2; +Cc: Andrew Cooper, Ian Campbell, Ben Guthro, Jan Beulich, xen-devel

[-- Attachment #1: Type: text/plain, Size: 3318 bytes --]

On Thu, Aug 15, 2013 at 09:28:24PM +0200, Atom2 wrote:
> Hi guys,
> thanks for your further input.
> 
> Following through Ben's mail below and Konrad's later mail
> suggesting the same, I tried to get these patches in. I'd however
> require your help before I feel I can safely proceed.
> 
> Please see below:
> 
> Am 15.08.13 03:58, schrieb Ben Guthro:
> [...]
> >I admit, I don't know how the gentoo build system works, but the general
> >idea here is that you want to revert those 2 commits, and apply the third.
> >
> >If you don't have a git tree, you can download the two commits from
> >these two links
> >http://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.git/patch/?id=c79c49826270b8b0061b2fca840fc3f013c8a78a
> >http://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.git/patch/?id=8eaffa67b43e99ae581622c5133e20b0f48bcef1
> >
> >You'll want to apply them in reverse
> After consultation with the manual I decided to give it a dry-run
> before and check with you guys first. First of all, I assume I'm
> righht that this is a patch to the *linux kernel* and not the
> xen-sources as I could not find the referenced files in the xen
> tree.

Right. You also need to compile the kernel. Usually I pluck the
/boot/config-my-exisitng-kernel-vresion and put it in the linux
directory as .config.

> 
> >patch -p1 -R < c79c498.patch
> vm-host # patch --dry-run -p1 -R < c79c498.patch
> patching file arch/x86/xen/enlighten.c
> Hunk #2 succeeded at 1431 (offset 14 lines).
> 
> I am slightly worried about the last message, not so much about the
> offset, but rather only the "Hunk #2" success. Why is there no "Hunk
> #1" when there's a "Hunk #2"?
> 
> >patch -p1 -R < 8eaffa67.patch
> vm-host # patch --dry-run -p1 -R < 8eaffa67.patch
> patching file arch/x86/xen/enlighten.c
> Hunk #1 succeeded at 1367 (offset 226 lines).
> patching file arch/x86/xen/mmu.c
> Hunk #1 succeeded at 434 (offset 19 lines).
> Hunk #2 succeeded at 482 (offset 19 lines).
> Hunk #3 succeeded at 495 (offset 19 lines).
> 
> That seems to be o.k. from my understanding?
> >
> >Then apply the patch from
> >https://lkml.org/lkml/2012/2/10/229
> For this patch I copied the complete text from the https address
> above and copied it to a file named 229.patch. Then I issued the
> following command:
> vm-host # patch --dry-run -p1 -R < 229.patch
> patching file arch/x86/include/asm/pgtable.h
> Unreversed patch detected!  Ignore -R? [n]

Note that you had been using --dry-run which means that the changes
did NOT go in effect.
> 
> I am not sure what to make out of this? Could you please provide some input.

Attaching the full part thanks to Martin Cerveny <martin@c-home.cz>
doing it in another thread (about the Nvidia and CUDA).

You basically want those changes that the diff file has.

After the patching, if you run git diff you should see a similar
result to what the attached patch had.

Then just do 'make -j3141567891238901948248092840932480932; sudo make modules_install; sudo make
install;sudo grub2-mkconfig -o /boot/grub2/grub.cfg' and reboot the new
kernel.

> Thanks and sorry for those probably dumb questions. I'm new to this
> (automated) patching thing, and with a little help, the first time
> usually works out well.

P.S.
No need to do the -j31415.. It should be just the amount of CPUs
you have.
> 

[-- Attachment #2: linux-3.9.fc18-PAT.patch --]
[-- Type: text/plain, Size: 3882 bytes --]

diff -uNrp linux-3.9.11-200.fc18.x86_64/arch/x86/include/asm/pgtable.h linux-3.9.11-200.PAT.fc18.x86_64/arch/x86/include/asm/pgtable.h
--- linux-3.9.11-200.fc18.x86_64/arch/x86/include/asm/pgtable.h	2013-08-13 22:35:49.372205382 +0200
+++ linux-3.9.11-200.fc18.x86_64/arch/x86/include/asm/pgtable.h	2013-08-13 22:58:23.212956533 +0200
@@ -353,6 +353,11 @@ static inline pgprot_t pgprot_modify(pgp
 	return __pgprot(preservebits | addbits);
 }
 
+static inline pgprot_t pte_attrs(pte_t pte)
+{
+	return __pgprot(pte_val(pte) & PTE_FLAGS_MASK);
+}
+
 #define pte_pgprot(x) __pgprot(pte_flags(x) & PTE_FLAGS_MASK)
 
 #define canon_pgprot(p) __pgprot(massage_pgprot(p))
diff -uNrp linux-3.9.11-200.fc18.x86_64/arch/x86/mm/pageattr.c linux-3.9.11-200.PAT.fc18.x86_64/arch/x86/mm/pageattr.c
--- linux-3.9.11-200.fc18.x86_64/arch/x86/mm/pageattr.c	2013-08-13 22:35:49.377206198 +0200
+++ linux-3.9.11-200.fc18.x86_64/arch/x86/mm/pageattr.c	2013-08-13 22:58:23.213956696 +0200
@@ -708,7 +708,7 @@ repeat:
 
 	if (level == PG_LEVEL_4K) {
 		pte_t new_pte;
-		pgprot_t new_prot = pte_pgprot(old_pte);
+		pgprot_t new_prot = pte_attrs(old_pte);
 		unsigned long pfn = pte_pfn(old_pte);
 
 		pgprot_val(new_prot) &= ~pgprot_val(cpa->mask_clr);
diff -uNrp linux-3.9.11-200.fc18.x86_64/arch/x86/xen/enlighten.c linux-3.9.11-200.PAT.fc18.x86_64/arch/x86/xen/enlighten.c
--- linux-3.9.11-200.fc18.x86_64/arch/x86/xen/enlighten.c	2013-08-13 22:35:49.404210605 +0200
+++ linux-3.9.11-200.fc18.x86_64/arch/x86/xen/enlighten.c	2013-08-13 22:52:28.505074079 +0200
@@ -67,7 +67,6 @@
 #include <asm/hypervisor.h>
 #include <asm/mwait.h>
 #include <asm/pci_x86.h>
-#include <asm/pat.h>
 
 #ifdef CONFIG_ACPI
 #include <linux/acpi.h>
@@ -1371,9 +1370,7 @@ asmlinkage void __init xen_start_kernel(
 
 	/* Prevent unwanted bits from being set in PTEs. */
 	__supported_pte_mask &= ~_PAGE_GLOBAL;
-#if 0
 	if (!xen_initial_domain())
-#endif
 		__supported_pte_mask &= ~(_PAGE_PWT | _PAGE_PCD);
 
 	__supported_pte_mask |= _PAGE_IOMAP;
@@ -1433,14 +1430,7 @@ asmlinkage void __init xen_start_kernel(
 	 */
 	acpi_numa = -1;
 #endif
-#ifdef CONFIG_X86_PAT
-	/*
-	 * For right now disable the PAT. We should remove this once
-	 * git commit 8eaffa67b43e99ae581622c5133e20b0f48bcef1
-	 * (xen/pat: Disable PAT support for now) is reverted.
-	 */
-	pat_enabled = 0;
-#endif
+
 	/* Don't do the full vcpu_info placement stuff until we have a
 	   possible map and a non-dummy shared_info. */
 	per_cpu(xen_vcpu, 0) = &HYPERVISOR_shared_info->vcpu_info[0];
diff -uNrp linux-3.9.11-200.fc18.x86_64/arch/x86/xen/mmu.c linux-3.9.11-200.PAT.fc18.x86_64/arch/x86/xen/mmu.c
--- linux-3.9.11-200.fc18.x86_64/arch/x86/xen/mmu.c	2013-08-13 22:35:49.404210605 +0200
+++ linux-3.9.11-200.fc18.x86_64/arch/x86/xen/mmu.c	2013-08-13 22:47:19.885598037 +0200
@@ -434,13 +434,13 @@ static pteval_t iomap_pte(pteval_t val)
 static pteval_t xen_pte_val(pte_t pte)
 {
 	pteval_t pteval = pte.pte;
-#if 0
+
 	/* If this is a WC pte, convert back from Xen WC to Linux WC */
 	if ((pteval & (_PAGE_PAT | _PAGE_PCD | _PAGE_PWT)) == _PAGE_PAT) {
 		WARN_ON(!pat_enabled);
 		pteval = (pteval & ~_PAGE_PAT) | _PAGE_PWT;
 	}
-#endif
+
 	if (xen_initial_domain() && (pteval & _PAGE_IOMAP))
 		return pteval;
 
@@ -482,7 +482,7 @@ void xen_set_pat(u64 pat)
 static pte_t xen_make_pte(pteval_t pte)
 {
 	phys_addr_t addr = (pte & PTE_PFN_MASK);
-#if 0
+
 	/* If Linux is trying to set a WC pte, then map to the Xen WC.
 	 * If _PAGE_PAT is set, then it probably means it is really
 	 * _PAGE_PSE, so avoid fiddling with the PAT mapping and hope
@@ -495,7 +495,7 @@ static pte_t xen_make_pte(pteval_t pte)
 		if ((pte & (_PAGE_PCD | _PAGE_PWT)) == _PAGE_PWT)
 			pte = (pte & ~(_PAGE_PCD | _PAGE_PWT)) | _PAGE_PAT;
 	}
-#endif
+
 	/*
 	 * Unprivileged domains are allowed to do IOMAPpings for
 	 * PCI passthrough, but not map ISA space.  The ISA

[-- Attachment #3: Type: text/plain, Size: 126 bytes --]

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

^ permalink raw reply	[flat|nested] 28+ messages in thread

* Re: Powerdown problem on XEN | ACPI S5
  2013-08-15 20:26                                 ` Konrad Rzeszutek Wilk
@ 2013-08-15 21:39                                   ` Atom2
  2013-08-16 12:24                                     ` Konrad Rzeszutek Wilk
  2013-12-11 21:52                                     ` Konrad Rzeszutek Wilk
  0 siblings, 2 replies; 28+ messages in thread
From: Atom2 @ 2013-08-15 21:39 UTC (permalink / raw)
  To: Konrad Rzeszutek Wilk
  Cc: Andrew Cooper, Ian Campbell, Ben Guthro, Jan Beulich, xen-devel

Hi guys,
the good news is that the latest patched kernel now powers down my 
machine as expected. Thanks for all your input and help.

The console is still working and there's no need to hide the Intel IGD 
from dom0 to get proper powerdown.

I am however still unclear what this may mean further down the line? Are 
those patches someting I have to manually apply for every new kernel 
release that I'm going to install?

Or would those patches be something that makes it into the upstream 
kernel sources to then again drip downstream allowing me to use my 
distribution's sources (gentoo in my case) without change in the future?

Are there any negative knock-on effects / reduced functionalities to be 
expected from the patches I have applied?

Being kernel patches I assume they at least should have no effect on 
upgrading from XEN 4.2.2 to newer versions in the future.


Also just for reference in case someone else faces the same problem and 
stumbles across this thread please find a few comments / clarifications 
below to the questions I had raised and to Konrad's subsequent answers.

Am 15.08.13 22:26, schrieb Konrad Rzeszutek Wilk:
> On Thu, Aug 15, 2013 at 09:28:24PM +0200, Atom2 wrote:
>> Hi guys,
>> thanks for your further input.
>>
>> Following through Ben's mail below and Konrad's later mail
>> suggesting the same, I tried to get these patches in. I'd however
>> require your help before I feel I can safely proceed.
>>
>> Please see below:
>>
>> Am 15.08.13 03:58, schrieb Ben Guthro:
>> [...]
>>> I admit, I don't know how the gentoo build system works, but the general
>>> idea here is that you want to revert those 2 commits, and apply the third.
>>>
>>> If you don't have a git tree, you can download the two commits from
>>> these two links
>>> http://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.git/patch/?id=c79c49826270b8b0061b2fca840fc3f013c8a78a
>>> http://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.git/patch/?id=8eaffa67b43e99ae581622c5133e20b0f48bcef1
>>>
>>> You'll want to apply them in reverse
>> After consultation with the manual I decided to give it a dry-run
>> before and check with you guys first. First of all, I assume I'm
>> righht that this is a patch to the *linux kernel* and not the
>> xen-sources as I could not find the referenced files in the xen
>> tree.
>
> Right. You also need to compile the kernel. Usually I pluck the
> /boot/config-my-exisitng-kernel-vresion and put it in the linux
> directory as .config.
Extracting .config from a kernel image requires the kernel configuration 
option CONFIG_IKCONFIG. One can then either extract the .config through 
scripts/extract-ikconfig (located under the linux directory) or if 
additionally CONFIG_IKCONFIG_PROC is configured, by accessing 
/proc/config.gz.

In my case (and most likely for all gentoo users) it was even easier as 
I had originally built the running kernel myself and .config was readily 
available in the right directory anyways.
>
>>
>>> patch -p1 -R < c79c498.patch
>> vm-host # patch --dry-run -p1 -R < c79c498.patch
>> patching file arch/x86/xen/enlighten.c
>> Hunk #2 succeeded at 1431 (offset 14 lines).
>>
>> I am slightly worried about the last message, not so much about the
>> offset, but rather only the "Hunk #2" success. Why is there no "Hunk
>> #1" when there's a "Hunk #2"?
That "Hunk #2" message seems to be harmless as a check of my patched 
sources against Konrad's diff attachement suggests. Still don't know 
where it comes from though.

>>
>>> patch -p1 -R < 8eaffa67.patch
>> vm-host # patch --dry-run -p1 -R < 8eaffa67.patch
>> patching file arch/x86/xen/enlighten.c
>> Hunk #1 succeeded at 1367 (offset 226 lines).
>> patching file arch/x86/xen/mmu.c
>> Hunk #1 succeeded at 434 (offset 19 lines).
>> Hunk #2 succeeded at 482 (offset 19 lines).
>> Hunk #3 succeeded at 495 (offset 19 lines).
>>
>> That seems to be o.k. from my understanding?
A check against Konrad's diff attachment after running the final patch 
command again confirmed everything o.k.

>>>
>>> Then apply the patch from
>>> https://lkml.org/lkml/2012/2/10/229
>> For this patch I copied the complete text from the https address
>> above and copied it to a file named 229.patch. Then I issued the
>> following command:
>> vm-host # patch --dry-run -p1 -R < 229.patch
>> patching file arch/x86/include/asm/pgtable.h
>> Unreversed patch detected!  Ignore -R? [n]
>
> Note that you had been using --dry-run which means that the changes
> did NOT go in effect.
>>
>> I am not sure what to make out of this? Could you please provide some input.
The issue was not the --dry-run (which I was aware of), but rather the 
-R option. This patch does not need to be *reversed* (the -R), but 
rather *applied* (as Ben had already suggested in his e-Mail). And that 
was what the message actually meant ...

I have also added a -b option to all patch commands (and clearly removed 
the --dry-run option for all patches) to create a backup copy just in 
case ...

>
> Attaching the full part thanks to Martin Cerveny <martin@c-home.cz>
> doing it in another thread (about the Nvidia and CUDA).
>
> You basically want those changes that the diff file has.
>
> After the patching, if you run git diff you should see a similar
> result to what the attached patch had.
>
> Then just do 'make -j3141567891238901948248092840932480932; sudo make modules_install; sudo make
> install;sudo grub2-mkconfig -o /boot/grub2/grub.cfg' and reboot the new
> kernel.
I had to do this slightly differently, not only because I use grub 
instead of grub2 - but that's something Konrad could not possibly have 
been aware of.

>
>> Thanks and sorry for those probably dumb questions. I'm new to this
>> (automated) patching thing, and with a little help, the first time
>> usually works out well.
>
> P.S.
> No need to do the -j31415.. It should be just the amount of CPUs
> you have.
Yeah, in my case it was just a -j9 using a 4-core CPU with hyperthreading
>>

^ permalink raw reply	[flat|nested] 28+ messages in thread

* Re: Powerdown problem on XEN | ACPI S5
  2013-08-15 21:39                                   ` Atom2
@ 2013-08-16 12:24                                     ` Konrad Rzeszutek Wilk
  2013-12-11 21:52                                     ` Konrad Rzeszutek Wilk
  1 sibling, 0 replies; 28+ messages in thread
From: Konrad Rzeszutek Wilk @ 2013-08-16 12:24 UTC (permalink / raw)
  To: Atom2, stefan.bader
  Cc: Andrew Cooper, Ian Campbell, Ben Guthro, Jan Beulich, xen-devel

On Thu, Aug 15, 2013 at 11:39:14PM +0200, Atom2 wrote:
> Hi guys,
> the good news is that the latest patched kernel now powers down my
> machine as expected. Thanks for all your input and help.
> 
> The console is still working and there's no need to hide the Intel
> IGD from dom0 to get proper powerdown.
> 
> I am however still unclear what this may mean further down the line?
> Are those patches someting I have to manually apply for every new
> kernel release that I'm going to install?

For right now, yes.
> 
> Or would those patches be something that makes it into the upstream
> kernel sources to then again drip downstream allowing me to use my
> distribution's sources (gentoo in my case) without change in the
> future?

That is the goal. The way that the x86 maintainer wanted this to be done is
to have some sort of bit lookup "box". Were the generic code does some
form of pte_wc(pte) and we lookup in the PAT for the index bit for WC.
And vice-versa to pte_wb(pte). The complication is that there is also
the PSE bit to think off. It is not as simple as it sounds.

CC-ing Stefan Bader who took a stab at it. Maybe he had a chance to
look in this a bit more.

(sorry if this is confusing, the WC stands for Write Combine -
while WB is WriteBack. WC is heavily used in graphics).
> 
> Are there any negative knock-on effects / reduced functionalities to
> be expected from the patches I have applied?

No. The opposite actually.
> 
> Being kernel patches I assume they at least should have no effect on
> upgrading from XEN 4.2.2 to newer versions in the future.

Correct.
> 
> 
> Also just for reference in case someone else faces the same problem
> and stumbles across this thread please find a few comments /
> clarifications below to the questions I had raised and to Konrad's
> subsequent answers.

Thanks!
> 
> Am 15.08.13 22:26, schrieb Konrad Rzeszutek Wilk:
> >On Thu, Aug 15, 2013 at 09:28:24PM +0200, Atom2 wrote:
> >>Hi guys,
> >>thanks for your further input.
> >>
> >>Following through Ben's mail below and Konrad's later mail
> >>suggesting the same, I tried to get these patches in. I'd however
> >>require your help before I feel I can safely proceed.
> >>
> >>Please see below:
> >>
> >>Am 15.08.13 03:58, schrieb Ben Guthro:
> >>[...]
> >>>I admit, I don't know how the gentoo build system works, but the general
> >>>idea here is that you want to revert those 2 commits, and apply the third.
> >>>
> >>>If you don't have a git tree, you can download the two commits from
> >>>these two links
> >>>http://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.git/patch/?id=c79c49826270b8b0061b2fca840fc3f013c8a78a
> >>>http://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.git/patch/?id=8eaffa67b43e99ae581622c5133e20b0f48bcef1
> >>>
> >>>You'll want to apply them in reverse
> >>After consultation with the manual I decided to give it a dry-run
> >>before and check with you guys first. First of all, I assume I'm
> >>righht that this is a patch to the *linux kernel* and not the
> >>xen-sources as I could not find the referenced files in the xen
> >>tree.
> >
> >Right. You also need to compile the kernel. Usually I pluck the
> >/boot/config-my-exisitng-kernel-vresion and put it in the linux
> >directory as .config.
> Extracting .config from a kernel image requires the kernel
> configuration option CONFIG_IKCONFIG. One can then either extract
> the .config through scripts/extract-ikconfig (located under the
> linux directory) or if additionally CONFIG_IKCONFIG_PROC is
> configured, by accessing /proc/config.gz.
> 
> In my case (and most likely for all gentoo users) it was even easier
> as I had originally built the running kernel myself and .config was
> readily available in the right directory anyways.
> >
> >>
> >>>patch -p1 -R < c79c498.patch
> >>vm-host # patch --dry-run -p1 -R < c79c498.patch
> >>patching file arch/x86/xen/enlighten.c
> >>Hunk #2 succeeded at 1431 (offset 14 lines).
> >>
> >>I am slightly worried about the last message, not so much about the
> >>offset, but rather only the "Hunk #2" success. Why is there no "Hunk
> >>#1" when there's a "Hunk #2"?
> That "Hunk #2" message seems to be harmless as a check of my patched
> sources against Konrad's diff attachement suggests. Still don't know
> where it comes from though.
> 
> >>
> >>>patch -p1 -R < 8eaffa67.patch
> >>vm-host # patch --dry-run -p1 -R < 8eaffa67.patch
> >>patching file arch/x86/xen/enlighten.c
> >>Hunk #1 succeeded at 1367 (offset 226 lines).
> >>patching file arch/x86/xen/mmu.c
> >>Hunk #1 succeeded at 434 (offset 19 lines).
> >>Hunk #2 succeeded at 482 (offset 19 lines).
> >>Hunk #3 succeeded at 495 (offset 19 lines).
> >>
> >>That seems to be o.k. from my understanding?
> A check against Konrad's diff attachment after running the final
> patch command again confirmed everything o.k.
> 
> >>>
> >>>Then apply the patch from
> >>>https://lkml.org/lkml/2012/2/10/229
> >>For this patch I copied the complete text from the https address
> >>above and copied it to a file named 229.patch. Then I issued the
> >>following command:
> >>vm-host # patch --dry-run -p1 -R < 229.patch
> >>patching file arch/x86/include/asm/pgtable.h
> >>Unreversed patch detected!  Ignore -R? [n]
> >
> >Note that you had been using --dry-run which means that the changes
> >did NOT go in effect.
> >>
> >>I am not sure what to make out of this? Could you please provide some input.
> The issue was not the --dry-run (which I was aware of), but rather
> the -R option. This patch does not need to be *reversed* (the -R),
> but rather *applied* (as Ben had already suggested in his e-Mail).
> And that was what the message actually meant ...
> 
> I have also added a -b option to all patch commands (and clearly
> removed the --dry-run option for all patches) to create a backup
> copy just in case ...
> 
> >
> >Attaching the full part thanks to Martin Cerveny <martin@c-home.cz>
> >doing it in another thread (about the Nvidia and CUDA).
> >
> >You basically want those changes that the diff file has.
> >
> >After the patching, if you run git diff you should see a similar
> >result to what the attached patch had.
> >
> >Then just do 'make -j3141567891238901948248092840932480932; sudo make modules_install; sudo make
> >install;sudo grub2-mkconfig -o /boot/grub2/grub.cfg' and reboot the new
> >kernel.
> I had to do this slightly differently, not only because I use grub
> instead of grub2 - but that's something Konrad could not possibly
> have been aware of.
> 
> >
> >>Thanks and sorry for those probably dumb questions. I'm new to this
> >>(automated) patching thing, and with a little help, the first time
> >>usually works out well.
> >
> >P.S.
> >No need to do the -j31415.. It should be just the amount of CPUs
> >you have.
> Yeah, in my case it was just a -j9 using a 4-core CPU with hyperthreading
> >>

^ permalink raw reply	[flat|nested] 28+ messages in thread

* Re: Powerdown problem on XEN | ACPI S5
  2013-08-15 21:39                                   ` Atom2
  2013-08-16 12:24                                     ` Konrad Rzeszutek Wilk
@ 2013-12-11 21:52                                     ` Konrad Rzeszutek Wilk
  1 sibling, 0 replies; 28+ messages in thread
From: Konrad Rzeszutek Wilk @ 2013-12-11 21:52 UTC (permalink / raw)
  To: Atom2; +Cc: Andrew Cooper, Ian Campbell, Ben Guthro, Jan Beulich, xen-devel

On Thu, Aug 15, 2013 at 11:39:14PM +0200, Atom2 wrote:
> Hi guys,
> the good news is that the latest patched kernel now powers down my
> machine as expected. Thanks for all your input and help.
> 
> The console is still working and there's no need to hide the Intel
> IGD from dom0 to get proper powerdown.
> 
> I am however still unclear what this may mean further down the line?
> Are those patches someting I have to manually apply for every new
> kernel release that I'm going to install?
> 
> Or would those patches be something that makes it into the upstream
> kernel sources to then again drip downstream allowing me to use my
> distribution's sources (gentoo in my case) without change in the
> future?
> 
> Are there any negative knock-on effects / reduced functionalities to
> be expected from the patches I have applied?

Could you refresh my memory of which ones? Can you just do a git diff
please on your Linux tree?
> 
> Being kernel patches I assume they at least should have no effect on
> upgrading from XEN 4.2.2 to newer versions in the future.
> 
> 
> Also just for reference in case someone else faces the same problem
> and stumbles across this thread please find a few comments /
> clarifications below to the questions I had raised and to Konrad's
> subsequent answers.
> 
> Am 15.08.13 22:26, schrieb Konrad Rzeszutek Wilk:
> >On Thu, Aug 15, 2013 at 09:28:24PM +0200, Atom2 wrote:
> >>Hi guys,
> >>thanks for your further input.
> >>
> >>Following through Ben's mail below and Konrad's later mail
> >>suggesting the same, I tried to get these patches in. I'd however
> >>require your help before I feel I can safely proceed.
> >>
> >>Please see below:
> >>
> >>Am 15.08.13 03:58, schrieb Ben Guthro:
> >>[...]
> >>>I admit, I don't know how the gentoo build system works, but the general
> >>>idea here is that you want to revert those 2 commits, and apply the third.
> >>>
> >>>If you don't have a git tree, you can download the two commits from
> >>>these two links
> >>>http://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.git/patch/?id=c79c49826270b8b0061b2fca840fc3f013c8a78a
> >>>http://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.git/patch/?id=8eaffa67b43e99ae581622c5133e20b0f48bcef1
> >>>
> >>>You'll want to apply them in reverse
> >>After consultation with the manual I decided to give it a dry-run
> >>before and check with you guys first. First of all, I assume I'm
> >>righht that this is a patch to the *linux kernel* and not the
> >>xen-sources as I could not find the referenced files in the xen
> >>tree.
> >
> >Right. You also need to compile the kernel. Usually I pluck the
> >/boot/config-my-exisitng-kernel-vresion and put it in the linux
> >directory as .config.
> Extracting .config from a kernel image requires the kernel
> configuration option CONFIG_IKCONFIG. One can then either extract
> the .config through scripts/extract-ikconfig (located under the
> linux directory) or if additionally CONFIG_IKCONFIG_PROC is
> configured, by accessing /proc/config.gz.
> 
> In my case (and most likely for all gentoo users) it was even easier
> as I had originally built the running kernel myself and .config was
> readily available in the right directory anyways.
> >
> >>
> >>>patch -p1 -R < c79c498.patch
> >>vm-host # patch --dry-run -p1 -R < c79c498.patch
> >>patching file arch/x86/xen/enlighten.c
> >>Hunk #2 succeeded at 1431 (offset 14 lines).
> >>
> >>I am slightly worried about the last message, not so much about the
> >>offset, but rather only the "Hunk #2" success. Why is there no "Hunk
> >>#1" when there's a "Hunk #2"?
> That "Hunk #2" message seems to be harmless as a check of my patched
> sources against Konrad's diff attachement suggests. Still don't know
> where it comes from though.
> 
> >>
> >>>patch -p1 -R < 8eaffa67.patch
> >>vm-host # patch --dry-run -p1 -R < 8eaffa67.patch
> >>patching file arch/x86/xen/enlighten.c
> >>Hunk #1 succeeded at 1367 (offset 226 lines).
> >>patching file arch/x86/xen/mmu.c
> >>Hunk #1 succeeded at 434 (offset 19 lines).
> >>Hunk #2 succeeded at 482 (offset 19 lines).
> >>Hunk #3 succeeded at 495 (offset 19 lines).
> >>
> >>That seems to be o.k. from my understanding?
> A check against Konrad's diff attachment after running the final
> patch command again confirmed everything o.k.
> 
> >>>
> >>>Then apply the patch from
> >>>https://lkml.org/lkml/2012/2/10/229
> >>For this patch I copied the complete text from the https address
> >>above and copied it to a file named 229.patch. Then I issued the
> >>following command:
> >>vm-host # patch --dry-run -p1 -R < 229.patch
> >>patching file arch/x86/include/asm/pgtable.h
> >>Unreversed patch detected!  Ignore -R? [n]
> >
> >Note that you had been using --dry-run which means that the changes
> >did NOT go in effect.
> >>
> >>I am not sure what to make out of this? Could you please provide some input.
> The issue was not the --dry-run (which I was aware of), but rather
> the -R option. This patch does not need to be *reversed* (the -R),
> but rather *applied* (as Ben had already suggested in his e-Mail).
> And that was what the message actually meant ...
> 
> I have also added a -b option to all patch commands (and clearly
> removed the --dry-run option for all patches) to create a backup
> copy just in case ...
> 
> >
> >Attaching the full part thanks to Martin Cerveny <martin@c-home.cz>
> >doing it in another thread (about the Nvidia and CUDA).
> >
> >You basically want those changes that the diff file has.
> >
> >After the patching, if you run git diff you should see a similar
> >result to what the attached patch had.
> >
> >Then just do 'make -j3141567891238901948248092840932480932; sudo make modules_install; sudo make
> >install;sudo grub2-mkconfig -o /boot/grub2/grub.cfg' and reboot the new
> >kernel.
> I had to do this slightly differently, not only because I use grub
> instead of grub2 - but that's something Konrad could not possibly
> have been aware of.
> 
> >
> >>Thanks and sorry for those probably dumb questions. I'm new to this
> >>(automated) patching thing, and with a little help, the first time
> >>usually works out well.
> >
> >P.S.
> >No need to do the -j31415.. It should be just the amount of CPUs
> >you have.
> Yeah, in my case it was just a -j9 using a 4-core CPU with hyperthreading
> >>

^ permalink raw reply	[flat|nested] 28+ messages in thread

* Re: Powerdown problem on XEN | ACPI S5
  2013-08-14 21:56                           ` Atom2
  2013-08-15  1:58                             ` Ben Guthro
@ 2013-08-15 13:40                             ` Konrad Rzeszutek Wilk
  1 sibling, 0 replies; 28+ messages in thread
From: Konrad Rzeszutek Wilk @ 2013-08-15 13:40 UTC (permalink / raw)
  To: Atom2; +Cc: Andrew Cooper, Ian Campbell, Ben Guthro, Jan Beulich, xen-devel

On Wed, Aug 14, 2013 at 11:56:10PM +0200, Atom2 wrote:
> Hi Ben & Konrad,
> thanks for your suggestions. I have to admit, I don't fully
> understand the commit system yet and don't know how to apply that to
> my gentoo sources. But I read through the thread that Ben has posted
> and thought, given that Ben had linked the issue to the Intel i915
> driver, I'll try something that crossed my mind:
> 
> lspci lists the Intel IGD as follows:
> 00:02.0 VGA compatible controller: Intel Corporation Xeon E3-1200
> Processor Family Integrated Graphics Controller (rev 09)
> 
> So my thought process was if the IGD is to blame, I'll just exclude
> that from the dom0 by adding it to the xen-pciback.hide list on the
> module line of grub. This line now reads as follows (NOTE: all other
> devices except the first one (00:02.0) have been there already
> before my latest try):
> 
> module  /boot/gentoo-3.9.5-hardened-xen rootfstype=ext4 root=/dev/sda6 \
> 
> xen-pciback.hide=(00:02.0)(00:1b.0)(02:00.0)(03:00.0)(04:00.0)(05:00.0)(06:00.0)(09:00.0)(09:02.0)(0a:08.0)(0a:09.0)(0a:0a.0
> )(0a:0b.0)
> 
> Clearly this meant that after the dom0 (gentoo hardened) was
> started, there was no more console output available. So I had to use
> ssh to login. But once I started a (remote) 'shutdown -h now' from
> the login bash and ... the system actually powerd off.
> 
> I guess this really proves that the (in-kernel) i915-driver which,
> given that the IGD is no longer available, could not have been
> active in the dom0, is to blame for the powerdown issue.

Perhaps. If this theory is correct then if you boot baremetal (so no
Xen) and with 'pat=0' on the command line you should see a similar
issue. That is theory.

But it would be really really fantastic if we could confirm that indeed
having those three patches (well, two reverts and one patch) indeed
fix the issue - without you resorting to disabling it altogether.

> 
> I am not sure whether that brings us any further in resolving the
> issue, but it might make sense to involve someone responsibel for
> the Intel IGD driver? Also I could not find much documentation for
> the i915.modeset kernel parameter- probably there's something in
> there that could help?

If you have it set to zero it will make X not work. In effect it should
have the same behavior as if you used the xen-pciback.hide=..
> 
> What are your thoughts on this.

Lets make sure that the patches work for you. If they do then you don't
have to use the work-around (disabling either iommu or not using IGD).
> 
> And as always thanks for your input so far and also thanks in
> advance for your continued help and ideas.
> 
> Am 14.08.13 22:37, schrieb Konrad Rzeszutek Wilk:
> >On Wed, Aug 14, 2013 at 04:34:21PM -0400, Ben Guthro wrote:
> >>This hang looks exactly like this one that I posted about in June:
> >>http://www.gossamer-threads.com/lists/xen/devel/284943?do=post_view_threaded#284943
> >>
> >>This ended up being an interaction with the i915 driver in the kernel, and
> >>pvops use of PAT.
> >>
> >>If you revert the following linux commit:
> >>http://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.git/commit/?id=c79c49826270b8b0061b2fca840fc3f013c8a78a
> >
> >And probably also need to revert
> >8eaffa67b43e99ae581622c5133e20b0f48bcef1
> >?
> >>
> >>and then apply:
> >>https://lkml.org/lkml/2012/2/10/229
> >>
> >>You may get good results.
> >>It helped me with a similar problem, at least.
> >>
> >>Ben
> >>
> >>
> >>
> >>On Wed, Aug 14, 2013 at 4:24 PM, Atom2 <ariel.atom2@web2web.at> wrote:
> >>
> >>>Am 14.08.13 22:18, schrieb Andrew Cooper:
> >>>[...]
> >>>
> >>>>
> >>>>Ok thanks.
> >>>>
> >>>>Do you mind confirming whether S5 works with "iommu=off" on the Xen
> >>>>command line?
> >>>>
> >>>>~Andrew
> >>>>
> >>>>  Yes, that works: The system powers off after issuing
> >>>         shutdown -h now
> >>>from the dom0.
> >>>
> >>>
> >>>______________________________**_________________
> >>>Xen-devel mailing list
> >>>Xen-devel@lists.xen.org
> >>>http://lists.xen.org/xen-devel
> >>>
> >
> >>_______________________________________________
> >>Xen-devel mailing list
> >>Xen-devel@lists.xen.org
> >>http://lists.xen.org/xen-devel
> >
> >
> >_______________________________________________
> >Xen-devel mailing list
> >Xen-devel@lists.xen.org
> >http://lists.xen.org/xen-devel
> >

^ permalink raw reply	[flat|nested] 28+ messages in thread

* Re: Powerdown problem on XEN | ACPI S5
  2013-08-14 20:24                     ` Atom2
  2013-08-14 20:30                       ` Atom2
  2013-08-14 20:34                       ` Ben Guthro
@ 2013-08-14 20:38                       ` Andrew Cooper
  2013-08-14 20:54                         ` Atom2
  2 siblings, 1 reply; 28+ messages in thread
From: Andrew Cooper @ 2013-08-14 20:38 UTC (permalink / raw)
  To: Atom2; +Cc: xen-devel, Ian Campbell, Jan Beulich

On 14/08/13 21:24, Atom2 wrote:
> Am 14.08.13 22:18, schrieb Andrew Cooper:
> [...]
>>
>> Ok thanks.
>>
>> Do you mind confirming whether S5 works with "iommu=off" on the Xen
>> command line?
>>
>> ~Andrew
>>
> Yes, that works: The system powers off after issuing
>     shutdown -h now
> from the dom0.

So it is certainly an iommu interaction issue, which was sadly suspected
given that we have seen similar problems in the past.

Curiously, there are two IOMMUs on the system.  I am not familiar enough
with Cougar Point chipsets to know how they are layed out.  Perhaps the
ACPI tables might have more information.

ACPI interaction with Xen is tricky at the best of times.  Xen as no AML
interpreter, so relies on Linux in dom0 to most of the ACPI legwork.

My best guess at the moment is that something in the ACPI code for S5 is
turning off enough of the PCH that Xen can no longer talk to the one of
the IOMMUs.

~Andrew

^ permalink raw reply	[flat|nested] 28+ messages in thread

* Re: Powerdown problem on XEN | ACPI S5
  2013-08-14 20:38                       ` Andrew Cooper
@ 2013-08-14 20:54                         ` Atom2
  2013-08-14 21:11                           ` Andrew Cooper
  2013-08-15  8:12                           ` Jan Beulich
  0 siblings, 2 replies; 28+ messages in thread
From: Atom2 @ 2013-08-14 20:54 UTC (permalink / raw)
  To: Andrew Cooper; +Cc: xen-devel, Ian Campbell, Jan Beulich

Andrew,
that does not sound too promising at the moment. Is there anything else 
I could provide to come to a resolution given that I/O virtualization is 
what the system is supposed to do.

You mention ACPI tables - I have no clue how to provide those, but I am 
more than happy to do what I can?

Ian (Campbell) originally diverted me to Jan (Beulich) who seemed to 
have an idea before you thankfully jumped in. Jan's idea seemed to also 
revolve around ACPI - although not tables, but registers:

quote from Jan Beulich:
 > It would be particularly interesting to know whether perhaps
 > some of the ACPI registers live in memory space on that
 > system - I already have a patch queued up (but not submitted
 > yet) that fixes problems in that case.

@Jan: would it make sense to go down that route?

Thanks again to everybody for their help so far.


Am 14.08.13 22:38, schrieb Andrew Cooper:
> On 14/08/13 21:24, Atom2 wrote:
>> Am 14.08.13 22:18, schrieb Andrew Cooper:
>> [...]
>>>
>>> Ok thanks.
>>>
>>> Do you mind confirming whether S5 works with "iommu=off" on the Xen
>>> command line?
>>>
>>> ~Andrew
>>>
>> Yes, that works: The system powers off after issuing
>>      shutdown -h now
>> from the dom0.
>
> So it is certainly an iommu interaction issue, which was sadly suspected
> given that we have seen similar problems in the past.
>
> Curiously, there are two IOMMUs on the system.  I am not familiar enough
> with Cougar Point chipsets to know how they are layed out.  Perhaps the
> ACPI tables might have more information.
I'm just curios, but where did you see two IOMMUs?
>
> ACPI interaction with Xen is tricky at the best of times.  Xen as no AML
> interpreter, so relies on Linux in dom0 to most of the ACPI legwork.
>
> My best guess at the moment is that something in the ACPI code for S5 is
> turning off enough of the PCH that Xen can no longer talk to the one of
> the IOMMUs.
>
> ~Andrew
>

^ permalink raw reply	[flat|nested] 28+ messages in thread

* Re: Powerdown problem on XEN | ACPI S5
  2013-08-14 20:54                         ` Atom2
@ 2013-08-14 21:11                           ` Andrew Cooper
  2013-08-15  8:12                           ` Jan Beulich
  1 sibling, 0 replies; 28+ messages in thread
From: Andrew Cooper @ 2013-08-14 21:11 UTC (permalink / raw)
  To: Atom2; +Cc: xen-devel, Ian Campbell, Jan Beulich

On 14/08/13 21:54, Atom2 wrote:
> Andrew,
> that does not sound too promising at the moment. Is there anything
> else I could provide to come to a resolution given that I/O
> virtualization is what the system is supposed to do.
>
> You mention ACPI tables - I have no clue how to provide those, but I
> am more than happy to do what I can?
>
> Ian (Campbell) originally diverted me to Jan (Beulich) who seemed to
> have an idea before you thankfully jumped in. Jan's idea seemed to
> also revolve around ACPI - although not tables, but registers:
>
> quote from Jan Beulich:
> > It would be particularly interesting to know whether perhaps
> > some of the ACPI registers live in memory space on that
> > system - I already have a patch queued up (but not submitted
> > yet) that fixes problems in that case.
>
> @Jan: would it make sense to go down that route?
>
> Thanks again to everybody for their help so far.

I would first start with the suggestion from Ben & Konrad, especially as
this smells the same in terms of breakage.

If that fails, you can get at the acpi tables using `acpidump` which
looks to live in the pmtools package in gentoo.

~Andrew

>
>
> Am 14.08.13 22:38, schrieb Andrew Cooper:
>> On 14/08/13 21:24, Atom2 wrote:
>>> Am 14.08.13 22:18, schrieb Andrew Cooper:
>>> [...]
>>>>
>>>> Ok thanks.
>>>>
>>>> Do you mind confirming whether S5 works with "iommu=off" on the Xen
>>>> command line?
>>>>
>>>> ~Andrew
>>>>
>>> Yes, that works: The system powers off after issuing
>>>      shutdown -h now
>>> from the dom0.
>>
>> So it is certainly an iommu interaction issue, which was sadly suspected
>> given that we have seen similar problems in the past.
>>
>> Curiously, there are two IOMMUs on the system.  I am not familiar enough
>> with Cougar Point chipsets to know how they are layed out.  Perhaps the
>> ACPI tables might have more information.
> I'm just curios, but where did you see two IOMMUs?

(XEN) PCI: Using MCFG for segment 0000 bus 00-3f
(XEN) Intel VT-d iommu 0 supported page sizes: 4kB.
(XEN) Intel VT-d iommu 1 supported page sizes: 4kB.
(XEN) Intel VT-d Snoop Control not enabled.
(XEN) Intel VT-d Dom0 DMA Passthrough not enabled.
(XEN) Intel VT-d Queued Invalidation enabled.
(XEN) Intel VT-d Interrupt Remapping enabled.
(XEN) Intel VT-d Shared EPT tables not enabled.
(XEN) I/O virtualisation enabled
(XEN)  - Dom0 mode: Relaxed

I added the iommu index into that loop because we found multi-socket
servers with multiple iommus, but I was not expecting to see two iommus
on a single socket workstation chipset.

>>
>> ACPI interaction with Xen is tricky at the best of times.  Xen as no AML
>> interpreter, so relies on Linux in dom0 to most of the ACPI legwork.
>>
>> My best guess at the moment is that something in the ACPI code for S5 is
>> turning off enough of the PCH that Xen can no longer talk to the one of
>> the IOMMUs.
>>
>> ~Andrew
>>

^ permalink raw reply	[flat|nested] 28+ messages in thread

* Re: Powerdown problem on XEN | ACPI S5
  2013-08-14 20:54                         ` Atom2
  2013-08-14 21:11                           ` Andrew Cooper
@ 2013-08-15  8:12                           ` Jan Beulich
  2013-08-15  8:16                             ` Atom2
  1 sibling, 1 reply; 28+ messages in thread
From: Jan Beulich @ 2013-08-15  8:12 UTC (permalink / raw)
  To: Andrew Cooper, Atom2; +Cc: xen-devel, Ian Campbell

>>> On 14.08.13 at 22:54, Atom2 <ariel.atom2@web2web.at> wrote:
> Ian (Campbell) originally diverted me to Jan (Beulich) who seemed to 
> have an idea before you thankfully jumped in. Jan's idea seemed to also 
> revolve around ACPI - although not tables, but registers:
> 
> quote from Jan Beulich:
>  > It would be particularly interesting to know whether perhaps
>  > some of the ACPI registers live in memory space on that
>  > system - I already have a patch queued up (but not submitted
>  > yet) that fixes problems in that case.
> 
> @Jan: would it make sense to go down that route?

No, that's clearly not an issue here, given the more complete
logs you sent meanwhile.

Jan

^ permalink raw reply	[flat|nested] 28+ messages in thread

* Re: Powerdown problem on XEN | ACPI S5
  2013-08-15  8:12                           ` Jan Beulich
@ 2013-08-15  8:16                             ` Atom2
  0 siblings, 0 replies; 28+ messages in thread
From: Atom2 @ 2013-08-15  8:16 UTC (permalink / raw)
  To: Jan Beulich; +Cc: Andrew Cooper, Ian Campbell, xen-devel

Jan,
many thanks for that clarification.

Am 15.08.13 10:12, schrieb Jan Beulich:
>>>> On 14.08.13 at 22:54, Atom2 <ariel.atom2@web2web.at> wrote:
>> Ian (Campbell) originally diverted me to Jan (Beulich) who seemed to
>> have an idea before you thankfully jumped in. Jan's idea seemed to also
>> revolve around ACPI - although not tables, but registers:
>>
>> quote from Jan Beulich:
>>   > It would be particularly interesting to know whether perhaps
>>   > some of the ACPI registers live in memory space on that
>>   > system - I already have a patch queued up (but not submitted
>>   > yet) that fixes problems in that case.
>>
>> @Jan: would it make sense to go down that route?
>
> No, that's clearly not an issue here, given the more complete
> logs you sent meanwhile.
>
> Jan
>

^ permalink raw reply	[flat|nested] 28+ messages in thread

end of thread, other threads:[~2013-12-11 21:52 UTC | newest]

Thread overview: 28+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2013-08-14  8:48 Powerdown problem on XEN | ACPI S5 Atom2
2013-08-14 10:30 ` Jan Beulich
2013-08-14 13:52   ` Atom2
2013-08-14 14:00     ` Andrew Cooper
2013-08-14 17:00       ` Atom2
2013-08-14 17:30         ` Andrew Cooper
2013-08-14 18:40           ` Atom2
2013-08-14 19:10             ` Atom2
2013-08-14 19:18               ` Andrew Cooper
2013-08-14 19:39                 ` Atom2
2013-08-14 20:18                   ` Andrew Cooper
2013-08-14 20:24                     ` Atom2
2013-08-14 20:30                       ` Atom2
2013-08-14 20:34                       ` Ben Guthro
2013-08-14 20:37                         ` Konrad Rzeszutek Wilk
2013-08-14 21:56                           ` Atom2
2013-08-15  1:58                             ` Ben Guthro
2013-08-15 19:28                               ` Atom2
2013-08-15 20:26                                 ` Konrad Rzeszutek Wilk
2013-08-15 21:39                                   ` Atom2
2013-08-16 12:24                                     ` Konrad Rzeszutek Wilk
2013-12-11 21:52                                     ` Konrad Rzeszutek Wilk
2013-08-15 13:40                             ` Konrad Rzeszutek Wilk
2013-08-14 20:38                       ` Andrew Cooper
2013-08-14 20:54                         ` Atom2
2013-08-14 21:11                           ` Andrew Cooper
2013-08-15  8:12                           ` Jan Beulich
2013-08-15  8:16                             ` Atom2

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).