Re: Problems accessing passthrough PCI device

[Simon Martin <furryfuttock@gmail.com>]

Thanks Jan,

Thursday, November 13, 2014, 11:08:33 AM, you wrote:

>>>> On 13.11.14 at 14:29, <furryfuttock@gmail.com> wrote:
>> I am having 2 major problems at the moment.
>> 
>> 1.- Access to the PCI device from the PV will fail the second time I
>> create it UNLESS I call xl pci-assignable-remove/pci-assignable-add
>> between each creation. If I don't do this then all PCI accesses return
>> -1. I get the same if I disable memory access in the PCI configuration
>> register.
>> 
>> 1.1.- Is this expected behaviour?

> No. But did you verify the driver properly enables the device (i.e.
> doesn't make assumptions on its state)? Also iirc some kernel
> versions weren't properly resetting the device when coming back
> from guest use, but without you stating the software versions
> you're using that's just a remote possibility.

Yes,  the  first thing I do in the driver is set the PCI configuration
access  bits to 7 that should enable IO space, Memory Space and Master
BUS access.

As  a  test I disabled this and all reads to the PCI device return -1,
even the first one.

>> 1.3.-  xl  dmesg  and  dmesg  in Dom0 do not show anything. I have set
>> loglvl=all in the Xen command line.

> "iommu=debug"? But given the symptoms above I don't really think
> this is a problem with the IOMMU.

Added this to the command line and when the PV starts I now see:

(XEN) [VT-D]iommu.c:1593: d0:PCI: unmap 0000:00:19.0
(XEN) [VT-D]iommu.c:1456: d4:PCI: map 0000:00:19.0

And when the PV stops I see

(XEN) [VT-D]iommu.c:1593: d4:PCI: unmap 0000:00:19.0
(XEN) [VT-D]iommu.c:1456: d0:PCI: map 0000:00:19.0

No more that this.

>> 2.-  Whenever  I  perform a software reset on the PCI device (it is an
>> Intel  82546  Ethernet  NIC) the hypervisor crashes. There is no oops,
>> kernel  panic  or the like, just a crash. My development device has no
>> serial port so I can't do much debugging.

> A crash would imply you see something telling you it's a crash. If
> you see nothing, I'd rather call it a hang or spontaneous reboot.
> But even without serial port (assuming you can't even plug in a
> serial port PCI card) there are ways to get at eventual crash output
> (register+stack dump): For one, we've got USB debug port support.
> This requires a special cable and there not being an (internal) hub in
> between, but it's worth a consideration. And in the worst case,
> "vga=keep" allows the hypervisor to continue printing to the screen
> even post-boot. But that requires the video mode to not be changed
> from the one Xen uses at boot (i.e. you should not use DRM's KMS,
> and if you need to use X it would need to be configured to use the
> frame buffer driver rather than any accelerating one).

I  agree,  this looks more than a hang than a crash. I've just found a
link to the USB debug cable. I've ordered one but it will take a while
to  get  here  (I  live in Chile). I'll try to enable the vga keep, at
least to see if I can debug this.


-- 
Best regards,
 Simon                            mailto:furryfuttock@gmail.com