From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from [140.186.70.92] (port=33232 helo=eggs.gnu.org) by lists.gnu.org with esmtp (Exim 4.43) id 1OTLBi-0001zd-VN for qemu-devel@nongnu.org; Mon, 28 Jun 2010 16:44:22 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.69) (envelope-from ) id 1OTL5n-0006AJ-3q for qemu-devel@nongnu.org; Mon, 28 Jun 2010 16:38:12 -0400 Received: from mail-vw0-f45.google.com ([209.85.212.45]:64192) by eggs.gnu.org with esmtp (Exim 4.69) (envelope-from ) id 1OTL5m-0006A4-Tp for qemu-devel@nongnu.org; Mon, 28 Jun 2010 16:38:11 -0400 Received: by vws14 with SMTP id 14so1208259vws.4 for ; Mon, 28 Jun 2010 13:38:08 -0700 (PDT) MIME-Version: 1.0 Sender: camm@ualberta.ca In-Reply-To: <4C270E25.7070409@redhat.com> References: <4C175E30.2030605@redhat.com> <4C270E25.7070409@redhat.com> Date: Mon, 28 Jun 2010 14:38:08 -0600 Message-ID: From: Cam Macdonell Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable Subject: [Qemu-devel] Re: Unusual physical address when using 64-bit BAR List-Id: qemu-devel.nongnu.org List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , To: Avi Kivity Cc: "qemu-devel@nongnu.org Developers" , "Michael S. Tsirkin" On Sun, Jun 27, 2010 at 2:39 AM, Avi Kivity wrote: > On 06/25/2010 12:51 AM, Cam Macdonell wrote: >> >> On Tue, Jun 15, 2010 at 5:04 AM, Avi Kivity =A0wrote: >> >>> >>> On 06/11/2010 08:31 PM, Cam Macdonell wrote: >>> >>>> >>>> On Mon, Apr 19, 2010 at 10:41 AM, Cam Macdonell >>>> =A0wrote: >>>> >>>> >>>>> >>>>> Hi, >>>>> >>>>> I'm trying to use a 64-bit BAR for my shared memory device. =A0In sim= ply >>>>> changing the memory type in pci_register_bar() to >>>>> PCI_BASE_ADDRESS_MEM_TYPE_64 I get an unusual physical address for >>>>> that BAR (and my driver crashes in pci_ioremap). >>>>> >>>>> from lspci: >>>>> >>>>> 00:04.0 RAM memory: Qumranet, Inc. Device 1110 >>>>> =A0 =A0 =A0 =A0Subsystem: Qumranet, Inc. Device 1100 >>>>> =A0 =A0 =A0 =A0Flags: fast devsel, IRQ 10 >>>>> =A0 =A0 =A0 =A0Memory at f1020000 (32-bit, non-prefetchable) [size=3D= 1K] >>>>> =A0 =A0 =A0 =A0Memory at f1021000 (32-bit, non-prefetchable) [size=3D= 4K] >>>>> =A0 =A0 =A0 =A0Memory at c20000000000 (64-bit, non-prefetchable) [siz= e=3D1024M] >>>>> =A0 =A0 =A0 =A0Capabilities: >>>>> 00: f4 1a 10 11 03 00 10 00 00 00 00 05 00 00 00 00 >>>>> 10: 00 00 02 f1 00 10 02 f1 04 00 00 00 00 c2 00 00 >>>>> 20: 00 00 00 00 00 00 00 00 00 00 00 00 f4 1a 00 11 >>>>> 30: 00 00 00 00 40 00 00 00 00 00 00 00 0b 01 00 00 >>>>> >>>>> with DEBUG_MEMREG, I see >>>>> >>>>> kvm_unregister_memory_area:666 Unregistering memory region >>>>> c20000000000 (40000000) >>>>> kvm_destroy_phys_mem:649 slot 7 start c20000000000 len 0 flags 0 >>>>> IVSHMEM: addr =3D 3221225472 size =3D 1073741824 >>>>> kvm_register_phys_mem:605 memory: gpa: c200c0000000, size: 40000000, >>>>> uaddr: 7f6dd7ffe000, slot: 7, flags: 0 >>>>> kvm_unregister_memory_area:666 Unregistering memory region >>>>> c200c0000000 (40000000) >>>>> kvm_destroy_phys_mem:649 slot 7 start c200c0000000 len 0 flags 0 >>>>> IVSHMEM: addr =3D 0 size =3D 1073741824 >>>>> kvm_register_phys_mem:605 memory: gpa: c20000000000, size: 40000000, >>>>> uaddr: 7f6dd7ffe000, slot: 7, flags: 0 >>>>> kvm_unregister_memory_area:666 Unregistering memory region >>>>> c20000000000 (40000000) >>>>> kvm_destroy_phys_mem:649 slot 7 start c20000000000 len 0 flags 0 >>>>> IVSHMEM: addr =3D 0 size =3D 1073741824 >>>>> kvm_register_phys_mem:605 memory: gpa: ffffffff00000000, size: >>>>> 40000000, uaddr: 7f6dd7ffe000, slot: 7, flags: 0 >>>>> kvm_unregister_memory_area:666 Unregistering memory region >>>>> ffffffff00000000 (40000000) >>>>> kvm_destroy_phys_mem:649 slot 7 start ffffffff00000000 len 0 flags 0 >>>>> IVSHMEM: addr =3D 0 size =3D 1073741824 >>>>> kvm_register_phys_mem:605 memory: gpa: c20000000000, size: 40000000, >>>>> uaddr: 7f6dd7ffe000, slot: 7, flags: 0 >>>>> >>>>> (the IVSHMEM lines are my debug statements) >>>>> >>>>> address sizes =A0 : 40 bits physical, 48 bits virtual =A0(guest) >>>>> address sizes =A0 : 38 bits physical, 48 bits virtual =A0(host) >>>>> >>>>> >>>>> >>>> >>>> Hi, I happened to run into this problem again when trying to use a >>>> 64-bit BAR. =A0I did a bit more digging and the test that is failing i= s >>>> called from arch/x86/mm/ioremap.c in the guest and here it is. >>>> >>>> static inline int phys_addr_valid(resource_size_t addr) >>>> { >>>> #ifdef CONFIG_PHYS_ADDR_T_64BIT >>>> =A0 =A0 =A0 =A0return !(addr>> =A0 =A0boot_cpu_data.x86_phys_bits); >>>> #else >>>> =A0 =A0 =A0 =A0return 1; >>>> #endif >>>> } >>>> >>>> the value of addr (in this case the 48-bit virtual address >>>> c20000000000) is shifted to the right shift by >>>> boot_cpu_data.x86_phys_bits (which is 40 bits, the physical address >>>> size), so a non-zero value is returned which causes the test to fail >>>> and generates the "invalid physical address" error in the guest. >>>> >>>> Any help is appreciated as to whether this is a Qemu or guest kernel >>>> issue. >>>> >>>> >>> >>> The guest kernel should never have generated an address that is bigger >>> than >>> cpu_phys_bits in the first place. =A0What's the value for cpu_phys_bits= in >>> the >>> guest? (/proc/cpuinfo, 'address sizes :' line). >>> >> >> Sorry I missed your reply until now. =A0The guest address sizes are as >> follows: >> >> address sizes =A0 : 40 bits physical, 48 bits virtual >> > > So the address c20000000000 is illegal. > >>> Is this really the address the guest programmed, or is qemu >>> misinterpreting >>> it? >>> > > Well, what's the answer? You're going to have to give me a hint on how to determine that. lspci in the guest shows the following Memory at c20000000000 (64-bit, non-prefetchable) [size=3D1024M] does that demonstrate a guest generated address? Cam > > -- > error compiling committee.c: too many arguments to function > >