From mboxrd@z Thu Jan 1 00:00:00 1970 From: Georg Bege Subject: Fwd: Re: Fwd: Re: Xen 4.3 / 4.4 - concurrent APIs, VGA Passthru Date: Sun, 03 Aug 2014 12:48:49 +0200 Message-ID: <53DE1391.2080205@ninth-art.de> References: <53DE1385.70006@ninth-art.de> Reply-To: therion@ninth-art.de Mime-Version: 1.0 Content-Type: multipart/mixed; boundary="===============1789051814667122766==" Return-path: In-Reply-To: <53DE1385.70006@ninth-art.de> List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Sender: xen-devel-bounces@lists.xen.org Errors-To: xen-devel-bounces@lists.xen.org To: xen-devel@lists.xensource.com List-Id: xen-devel@lists.xenproject.org This is a multi-part message in MIME format. --===============1789051814667122766== Content-Type: multipart/alternative; boundary="------------030208040707040908050801" This is a multi-part message in MIME format. --------------030208040707040908050801 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 7bit -------- Original-Nachricht -------- Betreff: Re: [Xen-devel] Fwd: Re: Xen 4.3 / 4.4 - concurrent APIs, VGA Passthru Datum: Sun, 03 Aug 2014 12:48:37 +0200 Von: Georg Bege Antwort an: therion@ninth-art.de An: Gordan Bobic Hi I've tried the proposed flags - I got some additional dmesg: 6.000171] Kernel command line: root=ZFS=warehouse/gentoo dozfs=force intel_iommu=on pcie_aspm=off pcie_ports=compat xen-pciback.hide=(00:1b.0)(02:00.0)(03:00.0)(03:00.1) [ 6.370424] pciback 0000:03:00.0: no compatible bridge window for [mem 0xfff80000-0xffffffff pref] [ 6.370481] pci 0000:04:00.0: no compatible bridge window for [mem 0xfff80000-0xffffffff pref] [ 6.370538] pci 0000:05:00.0: no compatible bridge window for [mem 0xfff80000-0 ... [ 6.000235] PCIe ASPM is disabled However that didnt truly help out. I might wonder if this has something to do with the fact that Im not able to enable interrupt remapping. Thus I also have pci_msitranslate = 0 - if I put it on 1, I get wired distortions in the DomU (with stdvga). Up to now, I thought that I do not really need interrupt remapping... maybe this has also something to do with the 'err code 43' while running Xen 4.3.x? regards, Georg PS: xl dmesg output below. --------------------------------- [xl dmesg output] --------------------------------- Xen 4.4.0 (XEN) Xen version 4.4.0 (@ninth-art.net) (x86_64-pc-linux-gnu-gcc (Gentoo 4.8.3 p1.1, pie-0.5.9) 4.8.3) debug=n Sun Aug 3 08:49:40 CEST 2014 (XEN) Latest ChangeSet: (XEN) Bootloader: GRUB 2.00 (XEN) Command line: dom0_max_vcpus=7 dom0_mem=16384M iommu=dom0-passthrough unrestricted_guest=1 msi=1 (XEN) Video information: (XEN) VGA is text mode 80x25, font 8x16 (XEN) VBE/DDC methods: V2; EDID transfer time: 1 seconds (XEN) Disc information: (XEN) Found 7 MBR signatures (XEN) Found 6 EDD information structures (XEN) Xen-e820 RAM map: (XEN) 0000000000000000 - 000000000008f000 (usable) (XEN) 000000000008f000 - 00000000000a0000 (reserved) (XEN) 00000000000e0000 - 0000000000100000 (reserved) (XEN) 0000000000100000 - 00000000b847a000 (usable) (XEN) 00000000b847a000 - 00000000b855e000 (ACPI NVS) (XEN) 00000000b855e000 - 00000000b89ac000 (usable) (XEN) 00000000b89ac000 - 00000000b8aac000 (ACPI NVS) (XEN) 00000000b8aac000 - 00000000b8ddd000 (usable) (XEN) 00000000b8ddd000 - 00000000b8e5d000 (reserved) (XEN) 00000000b8e5d000 - 00000000bb6b9000 (usable) (XEN) 00000000bb6b9000 - 00000000bb6bf000 (reserved) (XEN) 00000000bb6bf000 - 00000000bb71b000 (usable) (XEN) 00000000bb71b000 - 00000000bb7bf000 (ACPI NVS) (XEN) 00000000bb7bf000 - 00000000bb7de000 (usable) (XEN) 00000000bb7de000 - 00000000bb7f1000 (ACPI data) (XEN) 00000000bb7f1000 - 00000000bb7f2000 (usable) (XEN) 00000000bb7f2000 - 00000000bb7ff000 (ACPI data) (XEN) 00000000bb7ff000 - 00000000bb800000 (usable) (XEN) 00000000bb800000 - 00000000c0000000 (reserved) (XEN) 00000000f0000000 - 0000000100000000 (reserved) (XEN) 0000000100000000 - 0000000840000000 (usable) (XEN) ACPI: RSDP 000FE020, 0024 (r2 WORTMA) (XEN) ACPI: XSDT BB7FD120, 006C (r1 WORTMA WORTMANN 229 1000013) (XEN) ACPI: FACP BB7FC000, 00F4 (r3 WORTMA WORTMANN 229 MSFT 100000D) (XEN) ACPI: DSDT BB7F7000, 4F75 (r2 WORTMA WORTMANN 229 MSFT 100000D) (XEN) ACPI: FACS BB725000, 0040 (XEN) ACPI: APIC BB7F6000, 0138 (r2 WORTMA WORTMANN 229 MSFT 100000D) (XEN) ACPI: MCFG BB7F5000, 003C (r1 WORTMA WORTMANN 229 MSFT 100000D) (XEN) ACPI: ASF! BB7F4000, 00A5 (r32 WORTMA WORTMANN 229 MSFT 100000D) (XEN) ACPI: HPET BB7F3000, 0038 (r1 WORTMA WORTMANN 229 MSFT 100000D) (XEN) ACPI: SLIC BB7F2000, 0176 (r1 WORTMA WORTMANN 229 MSFT 100000D) (XEN) ACPI: ASPT BB7F0000, 0034 (r7 WORTMA WORTMANN 229 MSFT 100000D) (XEN) ACPI: SSDT BB7E0000, F054 (r1 WORTMA SSDT PM 229 MSFT 100000D) (XEN) ACPI: DMAR BB7DE000, 00E8 (r1 WORTMA WORTMANN 229 MSFT 100000D) (XEN) System RAM: 32692MB (33476996kB) (XEN) Domain heap initialised (XEN) Processor #0 6:13 APIC version 21 (XEN) Processor #1 6:13 APIC version 21 (XEN) Processor #2 6:13 APIC version 21 (XEN) Processor #3 6:13 APIC version 21 (XEN) Processor #4 6:13 APIC version 21 (XEN) Processor #5 6:13 APIC version 21 (XEN) Processor #6 6:13 APIC version 21 (XEN) Processor #7 6:13 APIC version 21 (XEN) Processor #8 6:13 APIC version 21 (XEN) Processor #9 6:13 APIC version 21 (XEN) Processor #10 6:13 APIC version 21 (XEN) Processor #11 6:13 APIC version 21 (XEN) IOAPIC[0]: apic_id 2, version 32, address 0xfec00000, GSI 0-23 (XEN) IOAPIC[1]: apic_id 1, version 32, address 0xfec3f000, GSI 24-47 (XEN) Enabling APIC mode: Phys. Using 2 I/O APICs (XEN) Failed to enable Interrupt Remapping: Will not enable x2APIC. (XEN) Using scheduler: SMP Credit Scheduler (credit) (XEN) Detected 2400.159 MHz processor. (XEN) Initing memory sharing. (XEN) xstate_init: using cntxt_size: 0x340 and states: 0x7 (XEN) Intel VT-d iommu 0 supported page sizes: 4kB, 2MB, 1GB. (XEN) Intel VT-d iommu 1 supported page sizes: 4kB, 2MB, 1GB. (XEN) Intel VT-d Snoop Control not enabled. (XEN) Intel VT-d Dom0 DMA Passthrough enabled. (XEN) Intel VT-d Queued Invalidation enabled. (XEN) Intel VT-d Interrupt Remapping enabled. (XEN) Intel VT-d Shared EPT tables enabled. (XEN) [VT-D] Platform firmware does not support interrupt remapping (XEN) [VT-D]iommu.c:2113: Interrupt Remapping not enabled (XEN) I/O virtualisation enabled (XEN) - Dom0 mode: Passthrough (XEN) Interrupt remapping disabled (XEN) Enabled directed EOI with ioapic_ack_old on! (XEN) ENABLING IO-APIC IRQs (XEN) -> Using old ACK method (XEN) Platform timer is 14.318MHz HPET (XEN) Allocated console ring of 32 KiB. (XEN) VMX: Supported advanced features: (XEN) - APIC MMIO access virtualisation (XEN) - APIC TPR shadow (XEN) - Extended Page Tables (EPT) (XEN) - Virtual-Processor Identifiers (VPID) (XEN) - Virtual NMI (XEN) - MSR direct-access bitmap (XEN) - Unrestricted Guest (XEN) HVM: ASIDs enabled. (XEN) HVM: VMX enabled (XEN) HVM: Hardware Assisted Paging (HAP) detected (XEN) HVM: HAP page sizes: 4kB, 2MB, 1GB (XEN) Brought up 12 CPUs (XEN) verify_tsc_reliability: TSC warp detected, disabling TSC_RELIABLE (XEN) *** LOADING DOMAIN 0 *** (XEN) Xen kernel: 64-bit, lsb, compat32 (XEN) Dom0 kernel: 64-bit, PAE, lsb, paddr 0x1000000 -> 0x1add000 (XEN) PHYSICAL MEMORY ARRANGEMENT: (XEN) Dom0 alloc.: 0000000824000000->0000000828000000 (4177164 pages to be allocated) (XEN) Init. ramdisk: 000000083fd0c000->000000083ffff080 (XEN) VIRTUAL MEMORY ARRANGEMENT: (XEN) Loaded kernel: ffffffff81000000->ffffffff81add000 (XEN) Init. ramdisk: ffffffff81add000->ffffffff81dd0080 (XEN) Phys-Mach map: ffffffff81dd1000->ffffffff83dd1000 (XEN) Start info: ffffffff83dd1000->ffffffff83dd14b4 (XEN) Page tables: ffffffff83dd2000->ffffffff83df5000 (XEN) Boot stack: ffffffff83df5000->ffffffff83df6000 (XEN) TOTAL: ffffffff80000000->ffffffff84000000 (XEN) ENTRY ADDRESS: ffffffff8155e1f0 (XEN) Dom0 has maximum 7 VCPUs (XEN) Scrubbing Free RAM: ...............................................................................................................................................................done. (XEN) Initial low memory virq threshold set at 0x4000 pages. (XEN) Std. Loglevel: Errors and warnings (XEN) Guest Loglevel: Nothing (Rate-limited: Errors and warnings) (XEN) Xen is relinquishing VGA console. (XEN) *** Serial input -> DOM0 (type 'CTRL-a' three times to switch input to Xen) (XEN) Freed 276kB init memory. Am 03.08.2014 11:40, schrieb Gordan Bobic: > > Have you tried an older Nvidia driver? I'm running the latest 331.x > driver on both XP and Win7 (both x64). > > The slowness might be indicative of an interrupt related problem. The > boot parameters I'm running with are as follows: > > xen.gz: dom0_vcpus_pin iommu=dom0-passthrough unrestricted_guest=1 msi=1 > vmlinuz: intel_iommu=on pcie_aspm=off pcie_ports=compat > > Note: The pcie_aspm=off pcie_ports=compat are to work around a bug on > my motherboard that causes a massive interrupt flood due to a device > on ICH10 flapping between hotplug states and creating an interrupt > storm that makes the entire machine permanently unusable as soon as > the kernel starts probing devices. You probably don't need those two > options but they shouldn't do any harm while you're troubleshooting. > > Gordan --------------030208040707040908050801 Content-Type: text/html; charset=UTF-8 Content-Transfer-Encoding: 7bit


-------- Original-Nachricht --------
Betreff: Re: [Xen-devel] Fwd: Re: Xen 4.3 / 4.4 - concurrent APIs, VGA Passthru
Datum: Sun, 03 Aug 2014 12:48:37 +0200
Von: Georg Bege <therion@ninth-art.de>
Antwort an: therion@ninth-art.de
An: Gordan Bobic <gordan@bobich.net> 


Hi

I've tried the proposed flags - I got some additional dmesg:
    6.000171] Kernel command line: root=ZFS=warehouse/gentoo dozfs=force
intel_iommu=on pcie_aspm=off pcie_ports=compat
xen-pciback.hide=(00:1b.0)(02:00.0)(03:00.0)(03:00.1)
[    6.370424] pciback 0000:03:00.0: no compatible bridge window for
[mem 0xfff80000-0xffffffff pref]
[    6.370481] pci 0000:04:00.0: no compatible bridge window for [mem
0xfff80000-0xffffffff pref]
[    6.370538] pci 0000:05:00.0: no compatible bridge window for [mem
0xfff80000-0
...
[    6.000235] PCIe ASPM is disabled

However that didnt truly help out.
I might wonder if this has something to do with the fact that Im not
able to enable interrupt remapping.
Thus I also have pci_msitranslate = 0 - if I put it on 1, I get wired
distortions in the DomU (with stdvga).

Up to now, I thought that I do not really need interrupt remapping...
maybe this has also something to do with the 'err code 43' while running
Xen 4.3.x?

regards,
Georg

PS: xl dmesg output below.

--------------------------------- [xl dmesg output]
---------------------------------
 Xen 4.4.0
(XEN) Xen version 4.4.0 (@ninth-art.net) (x86_64-pc-linux-gnu-gcc
(Gentoo 4.8.3 p1.1, pie-0.5.9) 4.8.3) debug=n Sun Aug  3 08:49:40 CEST 2014
(XEN) Latest ChangeSet:
(XEN) Bootloader: GRUB 2.00
(XEN) Command line: dom0_max_vcpus=7 dom0_mem=16384M
iommu=dom0-passthrough unrestricted_guest=1 msi=1
(XEN) Video information:
(XEN)  VGA is text mode 80x25, font 8x16
(XEN)  VBE/DDC methods: V2; EDID transfer time: 1 seconds
(XEN) Disc information:
(XEN)  Found 7 MBR signatures
(XEN)  Found 6 EDD information structures
(XEN) Xen-e820 RAM map:
(XEN)  0000000000000000 - 000000000008f000 (usable)
(XEN)  000000000008f000 - 00000000000a0000 (reserved)
(XEN)  00000000000e0000 - 0000000000100000 (reserved)
(XEN)  0000000000100000 - 00000000b847a000 (usable)
(XEN)  00000000b847a000 - 00000000b855e000 (ACPI NVS)
(XEN)  00000000b855e000 - 00000000b89ac000 (usable)
(XEN)  00000000b89ac000 - 00000000b8aac000 (ACPI NVS)
(XEN)  00000000b8aac000 - 00000000b8ddd000 (usable)
(XEN)  00000000b8ddd000 - 00000000b8e5d000 (reserved)
(XEN)  00000000b8e5d000 - 00000000bb6b9000 (usable)
(XEN)  00000000bb6b9000 - 00000000bb6bf000 (reserved)
(XEN)  00000000bb6bf000 - 00000000bb71b000 (usable)
(XEN)  00000000bb71b000 - 00000000bb7bf000 (ACPI NVS)
(XEN)  00000000bb7bf000 - 00000000bb7de000 (usable)
(XEN)  00000000bb7de000 - 00000000bb7f1000 (ACPI data)
(XEN)  00000000bb7f1000 - 00000000bb7f2000 (usable)
(XEN)  00000000bb7f2000 - 00000000bb7ff000 (ACPI data)
(XEN)  00000000bb7ff000 - 00000000bb800000 (usable)
(XEN)  00000000bb800000 - 00000000c0000000 (reserved)
(XEN)  00000000f0000000 - 0000000100000000 (reserved)
(XEN)  0000000100000000 - 0000000840000000 (usable)
(XEN) ACPI: RSDP 000FE020, 0024 (r2 WORTMA)
(XEN) ACPI: XSDT BB7FD120, 006C (r1 WORTMA WORTMANN      229       1000013)
(XEN) ACPI: FACP BB7FC000, 00F4 (r3 WORTMA WORTMANN      229 MSFT  100000D)
(XEN) ACPI: DSDT BB7F7000, 4F75 (r2 WORTMA WORTMANN      229 MSFT  100000D)
(XEN) ACPI: FACS BB725000, 0040
(XEN) ACPI: APIC BB7F6000, 0138 (r2 WORTMA WORTMANN      229 MSFT  100000D)
(XEN) ACPI: MCFG BB7F5000, 003C (r1 WORTMA WORTMANN      229 MSFT  100000D)
(XEN) ACPI: ASF! BB7F4000, 00A5 (r32 WORTMA WORTMANN      229 MSFT  100000D)
(XEN) ACPI: HPET BB7F3000, 0038 (r1 WORTMA WORTMANN      229 MSFT  100000D)
(XEN) ACPI: SLIC BB7F2000, 0176 (r1 WORTMA WORTMANN      229 MSFT  100000D)
(XEN) ACPI: ASPT BB7F0000, 0034 (r7 WORTMA WORTMANN      229 MSFT  100000D)
(XEN) ACPI: SSDT BB7E0000, F054 (r1 WORTMA SSDT  PM      229 MSFT  100000D)
(XEN) ACPI: DMAR BB7DE000, 00E8 (r1 WORTMA WORTMANN      229 MSFT  100000D)
(XEN) System RAM: 32692MB (33476996kB)
(XEN) Domain heap initialised
(XEN) Processor #0 6:13 APIC version 21
(XEN) Processor #1 6:13 APIC version 21
(XEN) Processor #2 6:13 APIC version 21
(XEN) Processor #3 6:13 APIC version 21
(XEN) Processor #4 6:13 APIC version 21
(XEN) Processor #5 6:13 APIC version 21
(XEN) Processor #6 6:13 APIC version 21
(XEN) Processor #7 6:13 APIC version 21
(XEN) Processor #8 6:13 APIC version 21
(XEN) Processor #9 6:13 APIC version 21
(XEN) Processor #10 6:13 APIC version 21
(XEN) Processor #11 6:13 APIC version 21
(XEN) IOAPIC[0]: apic_id 2, version 32, address 0xfec00000, GSI 0-23
(XEN) IOAPIC[1]: apic_id 1, version 32, address 0xfec3f000, GSI 24-47
(XEN) Enabling APIC mode:  Phys.  Using 2 I/O APICs
(XEN) Failed to enable Interrupt Remapping: Will not enable x2APIC.
(XEN) Using scheduler: SMP Credit Scheduler (credit)
(XEN) Detected 2400.159 MHz processor.
(XEN) Initing memory sharing.
(XEN) xstate_init: using cntxt_size: 0x340 and states: 0x7
(XEN) Intel VT-d iommu 0 supported page sizes: 4kB, 2MB, 1GB.
(XEN) Intel VT-d iommu 1 supported page sizes: 4kB, 2MB, 1GB.
(XEN) Intel VT-d Snoop Control not enabled.
(XEN) Intel VT-d Dom0 DMA Passthrough enabled.
(XEN) Intel VT-d Queued Invalidation enabled.
(XEN) Intel VT-d Interrupt Remapping enabled.
(XEN) Intel VT-d Shared EPT tables enabled.
(XEN) [VT-D] Platform firmware does not support interrupt remapping
(XEN) [VT-D]iommu.c:2113: Interrupt Remapping not enabled
(XEN) I/O virtualisation enabled
(XEN)  - Dom0 mode: Passthrough
(XEN) Interrupt remapping disabled
(XEN) Enabled directed EOI with ioapic_ack_old on!
(XEN) ENABLING IO-APIC IRQs
(XEN)  -> Using old ACK method
(XEN) Platform timer is 14.318MHz HPET
(XEN) Allocated console ring of 32 KiB.
(XEN) VMX: Supported advanced features:
(XEN)  - APIC MMIO access virtualisation
(XEN)  - APIC TPR shadow
(XEN)  - Extended Page Tables (EPT)
(XEN)  - Virtual-Processor Identifiers (VPID)
(XEN)  - Virtual NMI
(XEN)  - MSR direct-access bitmap
(XEN)  - Unrestricted Guest
(XEN) HVM: ASIDs enabled.
(XEN) HVM: VMX enabled
(XEN) HVM: Hardware Assisted Paging (HAP) detected
(XEN) HVM: HAP page sizes: 4kB, 2MB, 1GB
(XEN) Brought up 12 CPUs
(XEN) verify_tsc_reliability: TSC warp detected, disabling TSC_RELIABLE
(XEN) *** LOADING DOMAIN 0 ***
(XEN)  Xen  kernel: 64-bit, lsb, compat32
(XEN)  Dom0 kernel: 64-bit, PAE, lsb, paddr 0x1000000 -> 0x1add000
(XEN) PHYSICAL MEMORY ARRANGEMENT:
(XEN)  Dom0 alloc.:   0000000824000000->0000000828000000 (4177164 pages
to be allocated)
(XEN)  Init. ramdisk: 000000083fd0c000->000000083ffff080
(XEN) VIRTUAL MEMORY ARRANGEMENT:
(XEN)  Loaded kernel: ffffffff81000000->ffffffff81add000
(XEN)  Init. ramdisk: ffffffff81add000->ffffffff81dd0080
(XEN)  Phys-Mach map: ffffffff81dd1000->ffffffff83dd1000
(XEN)  Start info:    ffffffff83dd1000->ffffffff83dd14b4
(XEN)  Page tables:   ffffffff83dd2000->ffffffff83df5000
(XEN)  Boot stack:    ffffffff83df5000->ffffffff83df6000
(XEN)  TOTAL:         ffffffff80000000->ffffffff84000000
(XEN)  ENTRY ADDRESS: ffffffff8155e1f0
(XEN) Dom0 has maximum 7 VCPUs
(XEN) Scrubbing Free RAM:
...............................................................................................................................................................done.
(XEN) Initial low memory virq threshold set at 0x4000 pages.
(XEN) Std. Loglevel: Errors and warnings
(XEN) Guest Loglevel: Nothing (Rate-limited: Errors and warnings)
(XEN) Xen is relinquishing VGA console.
(XEN) *** Serial input -> DOM0 (type 'CTRL-a' three times to switch
input to Xen)
(XEN) Freed 276kB init memory.

Am 03.08.2014 11:40, schrieb Gordan Bobic:
>
> Have you tried an older Nvidia driver? I'm running the latest 331.x
> driver on both XP and Win7 (both x64).
>
> The slowness might be indicative of an interrupt related problem. The
> boot parameters I'm running with are as follows:
>
> xen.gz: dom0_vcpus_pin iommu=dom0-passthrough unrestricted_guest=1 msi=1
> vmlinuz: intel_iommu=on pcie_aspm=off pcie_ports=compat
>
> Note: The pcie_aspm=off pcie_ports=compat are to work around a bug on
> my motherboard that causes a massive interrupt flood due to a device
> on ICH10 flapping between hotplug states and creating an interrupt
> storm that makes the entire machine permanently unusable as soon as
> the kernel starts probing devices. You probably don't need those two
> options but they shouldn't do any harm while you're troubleshooting.
>
> Gordan


--------------030208040707040908050801-- --===============1789051814667122766== Content-Type: text/plain; charset="us-ascii" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Content-Disposition: inline _______________________________________________ Xen-devel mailing list Xen-devel@lists.xen.org http://lists.xen.org/xen-devel --===============1789051814667122766==--