Re: Problems accessing passthrough PCI device

[Simon Martin <furryfuttock@gmail.com>]

Hello Jan and Konrad,

Tuesday, November 18, 2014, 1:49:13 PM, you wrote:

>>
>> I've just checked this with lspci. I see that the IO is being enabled.

> Memory you mean.

Yes. Sorry.

>> Any   other   idea   on   why I might be reading back 0xff for all PCI
>> memory area reads? The lspci output follows.

> Since this isn't behind a bridge - no, not really. Did you try this with
> any other device for comparison purposes?

This   is  getting  more  interesting.  It  seems  that  something  is
overwriting the pci-back configuration data.

Starting  from a fresh reboot I checked the Dom0 pci configuration and
got this:

root@smartin-xen:~# lspci -s 00:19.0 -x
00:19.0 Ethernet controller: Intel Corporation Device 1559 (rev 04)
00: 86 80 59 15 00 00 10 00 04 00 00 02 00 00 00 00
10: 00 00 d0 f7 00 c0 d3 f7 81 f0 00 00 00 00 00 00
20: 00 00 00 00 00 00 00 00 00 00 00 00 86 80 54 20
30: 00 00 00 00 c8 00 00 00 00 00 00 00 05 01 00 00

I then start/stop my DomU and checked the Dom0 pci configuration again
and got this:

root@smartin-xen:~# lspci -s 00:19.0 -x
00:19.0 Ethernet controller: Intel Corporation Device 1559 (rev 04)
00: 86 80 59 15 00 00 10 00 04 00 00 02 00 00 00 00
10: 00 00 00 00 00 00 00 00 01 00 00 00 00 00 00 00
20: 00 00 00 00 00 00 00 00 00 00 00 00 86 80 54 20
30: 00 00 00 00 c8 00 00 00 00 00 00 00 05 01 00 00

Inside  my  DomU I added code to print the PCI configuration registers
and what I get after restarting the DomU is:

(d18) 14:57:04.042 src/e1000e.c@00150: 00: 86 80 59 15 00 00 10 00 04 00 00 02 00 00 00 00
(d18) 14:57:04.042 src/e1000e.c@00150: 10: 00 00 d0 f7 00 c0 d3 f7 81 f0 00 00 00 00 00 00
(d18) 14:57:04.042 src/e1000e.c@00150: 20: 00 00 00 00 00 00 00 00 00 00 00 00 86 80 54 20
(d18) 14:57:04.043 src/e1000e.c@00150: 30: 00 00 00 00 c8 00 00 00 00 00 00 00 14 01 00 00
(d18) 14:57:04.043 src/e1000e.c@00324: Enable PCI Memory Access
(d18) 14:57:05.043 src/e1000e.c@00150: 00: 86 80 59 15 03 00 10 00 04 00 00 02 00 00 00 00
(d18) 14:57:05.044 src/e1000e.c@00150: 10: 00 00 d0 f7 00 c0 d3 f7 81 f0 00 00 00 00 00 00
(d18) 14:57:05.044 src/e1000e.c@00150: 20: 00 00 00 00 00 00 00 00 00 00 00 00 86 80 54 20
(d18) 14:57:05.045 src/e1000e.c@00150: 30: 00 00 00 00 c8 00 00 00 00 00 00 00 14 01 00 00

As  you can see the pci configuration read from the pci-back driver by
my DomU is different to the data in the Dom0 pci configuration!

Just  before  leaving my DomU I disable the pci memory access and this
is what I see

(d18) 15:01:02.051 src/e1000e.c@00150: 00: 86 80 59 15 03 00 10 00 04 00 00 02 00 00 00 00
(d18) 15:01:02.051 src/e1000e.c@00150: 10: 00 00 d0 f7 00 c0 d3 f7 81 f0 00 00 00 00 00 00
(d18) 15:01:02.051 src/e1000e.c@00150: 20: 00 00 00 00 00 00 00 00 00 00 00 00 86 80 54 20
(d18) 15:01:02.052 src/e1000e.c@00150: 30: 00 00 00 00 c8 00 00 00 00 00 00 00 14 01 00 00
(d18) 15:01:02.052 src/e1000e.c@00541: Disable PCI Memory Access
(d18) 15:01:02.052 src/e1000e.c@00150: 00: 86 80 59 15 00 00 10 00 04 00 00 02 00 00 00 00
(d18) 15:01:02.052 src/e1000e.c@00150: 10: 00 00 d0 f7 00 c0 d3 f7 81 f0 00 00 00 00 00 00
(d18) 15:01:02.052 src/e1000e.c@00150: 20: 00 00 00 00 00 00 00 00 00 00 00 00 86 80 54 20
(d18) 15:01:02.053 src/e1000e.c@00150: 30: 00 00 00 00 c8 00 00 00 00 00 00 00 14 01 00 00

As  you  can  see the data is consistent with just writing 0000 to the
pci control register.

This is the output from the debug version of the xen-pciback module.

[ 5429.351231] pciback 0000:00:19.0: enabling device (0000 -> 0003)
[ 5429.351367] xen: registering gsi 20 triggering 0 polarity 1
[ 5429.351373] Already setup the GSI :20
[ 5429.351387] pciback 0000:00:19.0: xen-pciback[0000:00:19.0]: #20 on  disable-> enable
[ 5429.351436] pciback 0000:00:19.0: xen-pciback[0000:00:19.0]: #20 on  enabled
[ 5434.360078] pciback 0000:00:19.0: xen-pciback[0000:00:19.0]: #20 off  enable-> disable
[ 5434.360116] pciback 0000:00:19.0: xen-pciback[0000:00:19.0]: #0 off  disabled
[ 5434.361491] xen-pciback pci-20-0: fe state changed 5
[ 5434.362473] xen-pciback pci-20-0: fe state changed 6
[ 5434.363540] xen-pciback pci-20-0: fe state changed 0
[ 5434.363544] xen-pciback pci-20-0: frontend is gone! unregister device
[ 5434.467359] pciback 0000:00:19.0: resetting virtual configuration space
[ 5434.467376] pciback 0000:00:19.0: free-ing dynamically allocated virtual configuration space fields

Does this make any sense to you?

-- 
Best regards,
 Simon                            mailto:furryfuttock@gmail.com