From mboxrd@z Thu Jan 1 00:00:00 1970 From: Simon Martin Subject: Re: Problems accessing passthrough PCI device Date: Wed, 19 Nov 2014 12:12:09 -0300 Message-ID: <6916092.20141119121209@gmail.com> References: <198478230.20141113102921@gmail.com> <5464C971020000780004739B@mail.emea.novell.com> <196307380.20141113120732@gmail.com> <5464E1D9020000780004746B@mail.emea.novell.com> <429773295.20141113144907@gmail.com> <5465CB080200007800047845@mail.emea.novell.com> <19872515.20141118132405@gmail.com> <546B86990200007800048DBF@mail.emea.novell.com> Mime-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Return-path: In-Reply-To: <546B86990200007800048DBF@mail.emea.novell.com> List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Sender: xen-devel-bounces@lists.xen.org Errors-To: xen-devel-bounces@lists.xen.org To: Jan Beulich , Konrad Rzeszutek Wilk Cc: xen-devel@lists.xen.org List-Id: xen-devel@lists.xenproject.org Hello Jan and Konrad, Tuesday, November 18, 2014, 1:49:13 PM, you wrote: >> >> I've just checked this with lspci. I see that the IO is being enabled. > Memory you mean. Yes. Sorry. >> Any other idea on why I might be reading back 0xff for all PCI >> memory area reads? The lspci output follows. > Since this isn't behind a bridge - no, not really. Did you try this with > any other device for comparison purposes? This is getting more interesting. It seems that something is overwriting the pci-back configuration data. Starting from a fresh reboot I checked the Dom0 pci configuration and got this: root@smartin-xen:~# lspci -s 00:19.0 -x 00:19.0 Ethernet controller: Intel Corporation Device 1559 (rev 04) 00: 86 80 59 15 00 00 10 00 04 00 00 02 00 00 00 00 10: 00 00 d0 f7 00 c0 d3 f7 81 f0 00 00 00 00 00 00 20: 00 00 00 00 00 00 00 00 00 00 00 00 86 80 54 20 30: 00 00 00 00 c8 00 00 00 00 00 00 00 05 01 00 00 I then start/stop my DomU and checked the Dom0 pci configuration again and got this: root@smartin-xen:~# lspci -s 00:19.0 -x 00:19.0 Ethernet controller: Intel Corporation Device 1559 (rev 04) 00: 86 80 59 15 00 00 10 00 04 00 00 02 00 00 00 00 10: 00 00 00 00 00 00 00 00 01 00 00 00 00 00 00 00 20: 00 00 00 00 00 00 00 00 00 00 00 00 86 80 54 20 30: 00 00 00 00 c8 00 00 00 00 00 00 00 05 01 00 00 Inside my DomU I added code to print the PCI configuration registers and what I get after restarting the DomU is: (d18) 14:57:04.042 src/e1000e.c@00150: 00: 86 80 59 15 00 00 10 00 04 00 00 02 00 00 00 00 (d18) 14:57:04.042 src/e1000e.c@00150: 10: 00 00 d0 f7 00 c0 d3 f7 81 f0 00 00 00 00 00 00 (d18) 14:57:04.042 src/e1000e.c@00150: 20: 00 00 00 00 00 00 00 00 00 00 00 00 86 80 54 20 (d18) 14:57:04.043 src/e1000e.c@00150: 30: 00 00 00 00 c8 00 00 00 00 00 00 00 14 01 00 00 (d18) 14:57:04.043 src/e1000e.c@00324: Enable PCI Memory Access (d18) 14:57:05.043 src/e1000e.c@00150: 00: 86 80 59 15 03 00 10 00 04 00 00 02 00 00 00 00 (d18) 14:57:05.044 src/e1000e.c@00150: 10: 00 00 d0 f7 00 c0 d3 f7 81 f0 00 00 00 00 00 00 (d18) 14:57:05.044 src/e1000e.c@00150: 20: 00 00 00 00 00 00 00 00 00 00 00 00 86 80 54 20 (d18) 14:57:05.045 src/e1000e.c@00150: 30: 00 00 00 00 c8 00 00 00 00 00 00 00 14 01 00 00 As you can see the pci configuration read from the pci-back driver by my DomU is different to the data in the Dom0 pci configuration! Just before leaving my DomU I disable the pci memory access and this is what I see (d18) 15:01:02.051 src/e1000e.c@00150: 00: 86 80 59 15 03 00 10 00 04 00 00 02 00 00 00 00 (d18) 15:01:02.051 src/e1000e.c@00150: 10: 00 00 d0 f7 00 c0 d3 f7 81 f0 00 00 00 00 00 00 (d18) 15:01:02.051 src/e1000e.c@00150: 20: 00 00 00 00 00 00 00 00 00 00 00 00 86 80 54 20 (d18) 15:01:02.052 src/e1000e.c@00150: 30: 00 00 00 00 c8 00 00 00 00 00 00 00 14 01 00 00 (d18) 15:01:02.052 src/e1000e.c@00541: Disable PCI Memory Access (d18) 15:01:02.052 src/e1000e.c@00150: 00: 86 80 59 15 00 00 10 00 04 00 00 02 00 00 00 00 (d18) 15:01:02.052 src/e1000e.c@00150: 10: 00 00 d0 f7 00 c0 d3 f7 81 f0 00 00 00 00 00 00 (d18) 15:01:02.052 src/e1000e.c@00150: 20: 00 00 00 00 00 00 00 00 00 00 00 00 86 80 54 20 (d18) 15:01:02.053 src/e1000e.c@00150: 30: 00 00 00 00 c8 00 00 00 00 00 00 00 14 01 00 00 As you can see the data is consistent with just writing 0000 to the pci control register. This is the output from the debug version of the xen-pciback module. [ 5429.351231] pciback 0000:00:19.0: enabling device (0000 -> 0003) [ 5429.351367] xen: registering gsi 20 triggering 0 polarity 1 [ 5429.351373] Already setup the GSI :20 [ 5429.351387] pciback 0000:00:19.0: xen-pciback[0000:00:19.0]: #20 on disable-> enable [ 5429.351436] pciback 0000:00:19.0: xen-pciback[0000:00:19.0]: #20 on enabled [ 5434.360078] pciback 0000:00:19.0: xen-pciback[0000:00:19.0]: #20 off enable-> disable [ 5434.360116] pciback 0000:00:19.0: xen-pciback[0000:00:19.0]: #0 off disabled [ 5434.361491] xen-pciback pci-20-0: fe state changed 5 [ 5434.362473] xen-pciback pci-20-0: fe state changed 6 [ 5434.363540] xen-pciback pci-20-0: fe state changed 0 [ 5434.363544] xen-pciback pci-20-0: frontend is gone! unregister device [ 5434.467359] pciback 0000:00:19.0: resetting virtual configuration space [ 5434.467376] pciback 0000:00:19.0: free-ing dynamically allocated virtual configuration space fields Does this make any sense to you? -- Best regards, Simon mailto:furryfuttock@gmail.com