From mboxrd@z Thu Jan 1 00:00:00 1970 From: Ward Vandewege Subject: Re: AMD_IOV: IO_PAGE_FALT trying to pass through Mellanox ConnectX HCA (debian testing) Date: Mon, 31 Jan 2011 14:51:54 -0500 Message-ID: <20110131195154.GA19572@countzero.vandewege.net> References: <20110128185809.GA24465@countzero.vandewege.net> <20110128192742.GB23494@dumpdata.com> <20110128203858.GA26814@countzero.vandewege.net> <20110131184503.GB11939@dumpdata.com> Mime-Version: 1.0 Content-Type: multipart/mixed; boundary="VrqPEDrXMn8OVzN4" Return-path: Content-Disposition: inline In-Reply-To: <20110131184503.GB11939@dumpdata.com> List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Sender: xen-devel-bounces@lists.xensource.com Errors-To: xen-devel-bounces@lists.xensource.com To: Konrad Rzeszutek Wilk Cc: xen-devel@lists.xensource.com List-Id: xen-devel@lists.xenproject.org --VrqPEDrXMn8OVzN4 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Hi Konrad, On Mon, Jan 31, 2011 at 01:45:03PM -0500, Konrad Rzeszutek Wilk wrote: > > Hmm, I do have it: > > Indeed you do. Good! > > > > > # xm dmesg |grep iommu > > (XEN) Command line: placeholder iommu=pv,verbose,amd_iommu_debug > > > > But maybe it's not being picked up? > > You should see something passthrough in the log.. thought that might > be only if you are using Intel VT-d? Not sure. This seems related: (XEN) HVM: ASIDs enabled. (XEN) HVM: SVM enabled (XEN) HVM: Hardware Assisted Paging detected. (XEN) AMD-Vi: IOMMU 0 Enabled. (XEN) I/O virtualisation enabled (XEN) - Dom0 mode: Relaxed (XEN) Total of 16 processors activated. (XEN) ENABLING IO-APIC IRQs (XEN) -> Using new ACK method I've attached the full xm dmesg output. > > > you might need to make sure your driver is using the VM_IO flag. > > > > > > There was some discussion on LKML about this and they proposed > > > a patch that wasn't neccessary. Don't remember the details but I can > > > look that up next week. > > Found it.. it was from Vivien but in another thread: > http://www.mail-archive.com/linux-rdma@vger.kernel.org/msg06980.html Ah. Is your devel/p2m-identity.v4.5 still the one I should test with to see if it fixes this problem? I see you've got newer versions (up to v4.7) now too. Or has this patch meanwhile been pushed into the kernel? Thanks, Ward. --VrqPEDrXMn8OVzN4 Content-Type: text/plain; charset=us-ascii Content-Disposition: attachment; filename="xm.dmesg2" (XEN) Xen version 4.0.1 (Debian 4.0.1-2) (waldi@debian.org) (gcc version 4.4.5 (Debian 4.4.5-10) ) Wed Jan 12 14:04:06 UTC 2011 (XEN) Bootloader: GRUB 1.98+20100804-13 (XEN) Command line: iommu=pv,verbose,amd_iommu_debug (XEN) Video information: (XEN) VGA is text mode 80x25, font 8x16 (XEN) VBE/DDC methods: none; EDID transfer time: 0 seconds (XEN) EDID info not retrieved because no DDC retrieval method detected (XEN) Disc information: (XEN) Found 2 MBR signatures (XEN) Found 2 EDD information structures (XEN) Xen-e820 RAM map: (XEN) 0000000000000000 - 000000000009ac00 (usable) (XEN) 000000000009ac00 - 00000000000a0000 (reserved) (XEN) 00000000000e6000 - 0000000000100000 (reserved) (XEN) 0000000000100000 - 00000000dfe90000 (usable) (XEN) 00000000dfe9e000 - 00000000dfea0000 type 9 (XEN) 00000000dfea0000 - 00000000dfeb2000 (ACPI data) (XEN) 00000000dfeb2000 - 00000000dfee0000 (ACPI NVS) (XEN) 00000000dfee0000 - 00000000f0000000 (reserved) (XEN) 00000000ffe00000 - 0000000100000000 (reserved) (XEN) 0000000100000000 - 0000000820000000 (usable) (XEN) ACPI: RSDP 000F9E90, 0024 (r2 ACPIAM) (XEN) ACPI: XSDT DFEA0100, 008C (r1 SMCI 20101029 MSFT 97) (XEN) ACPI: FACP DFEA0290, 00F4 (r4 102910 FACP1824 20101029 MSFT 97) (XEN) ACPI: DSDT DFEA0670, 56DB (r2 1AA11 1AA11000 0 INTL 20051117) (XEN) ACPI: FACS DFEB2000, 0040 (XEN) ACPI: APIC DFEA0390, 0118 (r2 102910 APIC1824 20101029 MSFT 97) (XEN) ACPI: MCFG DFEA04B0, 003C (r1 102910 OEMMCFG 20101029 MSFT 97) (XEN) ACPI: OEMB DFEB2040, 0075 (r1 102910 OEMB1824 20101029 MSFT 97) (XEN) ACPI: HPET DFEAA670, 0038 (r1 102910 OEMHPET 20101029 MSFT 97) (XEN) ACPI: IVRS DFEAA6B0, 00B8 (r1 AMD RD890S 202031 AMD 0) (XEN) ACPI: SRAT DFEAA770, 0220 (r2 AMD F10 1 AMD 1) (XEN) ACPI: SLIT DFEAA990, 003C (r1 AMD F10 1 AMD 1) (XEN) ACPI: SSDT DFEAA9D0, 2854 (r1 A M I POWERNOW 1 AMD 1) (XEN) ACPI: EINJ DFEAD230, 0130 (r1 AMIER AMI_EINJ 20101029 MSFT 97) (XEN) ACPI: BERT DFEAD3C0, 0030 (r1 AMIER AMI_BERT 20101029 MSFT 97) (XEN) ACPI: ERST DFEAD3F0, 01B0 (r1 AMIER AMI_ERST 20101029 MSFT 97) (XEN) ACPI: HEST DFEAD5A0, 00A8 (r1 AMIER ABC_HEST 20101029 MSFT 97) (XEN) System RAM: 32766MB (33552552kB) (XEN) Reserving non-aligned node boundary @ mfn 0x620000 (XEN) Domain heap initialised DMA width 31 bits (XEN) Processor #16 0:9 APIC version 16 (XEN) Processor #17 0:9 APIC version 16 (XEN) Processor #18 0:9 APIC version 16 (XEN) Processor #19 0:9 APIC version 16 (XEN) Processor #20 0:9 APIC version 16 (XEN) Processor #21 0:9 APIC version 16 (XEN) Processor #22 0:9 APIC version 16 (XEN) Processor #23 0:9 APIC version 16 (XEN) Processor #32 0:9 APIC version 16 (XEN) Processor #33 0:9 APIC version 16 (XEN) Processor #34 0:9 APIC version 16 (XEN) Processor #35 0:9 APIC version 16 (XEN) Processor #36 0:9 APIC version 16 (XEN) Processor #37 0:9 APIC version 16 (XEN) Processor #38 0:9 APIC version 16 (XEN) Processor #39 0:9 APIC version 16 (XEN) IOAPIC[0]: apic_id 0, version 33, address 0xfec00000, GSI 0-23 (XEN) Enabling APIC mode: Phys. Using 1 I/O APICs (XEN) Using scheduler: SMP Credit Scheduler (credit) (XEN) Detected 2000.166 MHz processor. (XEN) Initing memory sharing. (XEN) HVM: ASIDs enabled. (XEN) HVM: SVM enabled (XEN) HVM: Hardware Assisted Paging detected. (XEN) AMD-Vi: IOMMU 0 Enabled. (XEN) I/O virtualisation enabled (XEN) - Dom0 mode: Relaxed (XEN) Total of 16 processors activated. (XEN) ENABLING IO-APIC IRQs (XEN) -> Using new ACK method (XEN) TSC is reliable, synchronization unnecessary (XEN) Platform timer appears to have unexpectedly wrapped 10 or more times. (XEN) Platform timer is 14.318MHz HPET (XEN) Allocated console ring of 32 KiB. (XEN) do_IRQ: 1.231 No irq handler for vector (irq -1) (XEN) Brought up 16 CPUs (XEN) do_IRQ: 2.231 No irq handler for vector (irq -1) (XEN) do_IRQ: 3.231 No irq handler for vector (irq -1) (XEN) do_IRQ: 5.231 No irq handler for vector (irq -1) (XEN) do_IRQ: 7.231 No irq handler for vector (irq -1) (XEN) do_IRQ: 4.231 No irq handler for vector (irq -1) (XEN) do_IRQ: 6.231 No irq handler for vector (irq -1) (XEN) do_IRQ: 15.231 No irq handler for vector (irq -1) (XEN) do_IRQ: 13.231 No irq handler for vector (irq -1) (XEN) do_IRQ: 12.231 No irq handler for vector (irq -1) (XEN) do_IRQ: 14.231 No irq handler for vector (irq -1) (XEN) do_IRQ: 8.231 No irq handler for vector (irq -1) (XEN) do_IRQ: 10.231 No irq handler for vector (irq -1) (XEN) do_IRQ: 11.231 No irq handler for vector (irq -1) (XEN) do_IRQ: 9.231 No irq handler for vector (irq -1) (XEN) *** LOADING DOMAIN 0 *** (XEN) Xen kernel: 64-bit, lsb, compat32 (XEN) Dom0 kernel: 64-bit, PAE, lsb, paddr 0x1000000 -> 0x16b8000 (XEN) PHYSICAL MEMORY ARRANGEMENT: (XEN) Dom0 alloc.: 0000000210000000->0000000218000000 (8213477 pages to be allocated) (XEN) VIRTUAL MEMORY ARRANGEMENT: (XEN) Loaded kernel: ffffffff81000000->ffffffff816b8000 (XEN) Init. ramdisk: ffffffff816b8000->ffffffff83ae5c00 (XEN) Phys-Mach map: ffffffff83ae6000->ffffffff879cff28 (XEN) Start info: ffffffff879d0000->ffffffff879d04b4 (XEN) Page tables: ffffffff879d1000->ffffffff87a12000 (XEN) Boot stack: ffffffff87a12000->ffffffff87a13000 (XEN) TOTAL: ffffffff80000000->ffffffff87c00000 (XEN) ENTRY ADDRESS: ffffffff81508200 (XEN) Dom0 has maximum 16 VCPUs (XEN) Scrubbing Free RAM: .done. (XEN) Xen trace buffers: disabled (XEN) Std. Loglevel: Errors and warnings (XEN) Guest Loglevel: Nothing (Rate-limited: Errors and warnings) (XEN) Xen is relinquishing VGA console. (XEN) *** Serial input -> DOM0 (type 'CTRL-a' three times to switch input to Xen) (XEN) Freed 176kB init memory. (XEN) traps.c:2308:d0 Domain attempted WRMSR 00000000c0010004 from 00004668:6b28938f to 00000000:00000000. (XEN) traps.c:2308:d0 Domain attempted WRMSR 00000000c0010000 from 0000030d:c1990e3a to 00000000:00430076. (XEN) AMD_IOV: IO_PAGE_FALT: domain:1, device id:0x200, fault address:0x32c8000 (XEN) AMD_IOV: IO_PAGE_FALT: domain:1, device id:0x200, fault address:0x32c8040 (XEN) AMD_IOV: IO_PAGE_FALT: domain:1, device id:0x200, fault address:0x32c8080 (XEN) AMD_IOV: IO_PAGE_FALT: domain:1, device id:0x200, fault address:0x32c80c0 --VrqPEDrXMn8OVzN4 Content-Type: text/plain; charset="us-ascii" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Content-Disposition: inline _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel --VrqPEDrXMn8OVzN4--