Re: Does xen-4.2.0 support VGA passthrough with the virtual machine created by xl command?

Re: Does xen-4.2.0 support VGA passthrough with the virtual machine created by xl command?

[Dr. Greg Wettstein]

On Nov 14,  3:40pm, "Dr. Greg Wettstein" wrote:
} Subject: Re: [Xen-devel] Does xen-4.2.0 support VGA passthrough with the v

Good morning, hope the day is going well for everyone.

> On Nov 13, 10:02am, Ian Campbell wrote:
> } Subject: Re: [Xen-devel] Does xen-4.2.0 support VGA passthrough with the v
> 
> Good afternoon, hope the week is going well for everyone.
> 
> > On Tue, 2012-11-13 at 06:30 +0000, Qian Hu wrote:
> > 
> > > With spice tool, I have to create a VM by xl command, and now I am
> > > wondering if it supports VGA passghrouth?
> 
> > This list is for the development of Xen. You';d probably have more
> > luck with these sorts of support requests on the xen-users list.
> 
> That would normally be the case but I'm suspicious there are issues
> with VGA passthrough in 4.2.0.

I just wanted to follow up to the list on the status of passthrough
issues.

We reverted our test machine back to the 2.6.32.45 kernel which we had
been using in production.  That kernel was based on Jeremy's GIT
tree.  Using xm and the updated ATI patches which I referenced in my
original mail passthrough works as it should.

Passthrough does not work with xl.  Windows started but fell into its
text mode rescue screen and registered a crash dump.  It flashed the
screen back and forth between a stipled blue/grey and totally black
screen a few times and then locked the physical machine up solidly.

On the next boot I thought about it but declined to register the crash
dump with Microsoft.... :-)

We then went back and tested the 3.4.18 kernel and with both xm and xl
the guest faults on the first attempt to do an I/O port access.  All
factors (windows image, hardware, xen guest config) are held identical
so the difference would seem to be linked to the PCI passthrough
implementation between the two kernels.  I've copied Konrad on the
note since he would seem to be the person most familiar with this
area.

I'm including below a diff between a successful qemu-dm passthrough
session (2.6.32.45) and an unsuccessful session (3.4.18).  It would
appear 3.4.18 is getting the both the I/O port and memory ranges
wrong.

Let me know if I can forward any additional information or run any
additional tests.

Have a good weekend.

Greg

qemu-dm log diff: ---------------------------------------------------------
1c1
< domid: 3
---
> domid: 1
3,5c3,5
< Watching /local/domain/0/device-model/3/logdirty/cmd
< Watching /local/domain/0/device-model/3/command
< Watching /local/domain/3/cpu
---
> Watching /local/domain/0/device-model/1/logdirty/cmd
> Watching /local/domain/0/device-model/1/command
> Watching /local/domain/1/cpu
9c9
< Guest uuid = 7fcefb13-d1ef-105b-e38c-1e1454411e80
---
> Guest uuid = eab6bbbb-4819-b970-a83c-03288a1541ad
14c14
< xs_read(/local/domain/0/device-model/3/xen_extended_power_mgmt): read error
---
> xs_read(/local/domain/0/device-model/1/xen_extended_power_mgmt): read error
18,20c18,20
< xs_read(/local/domain/3/log-throttling): read error
< qemu: ignoring not-understood drive `/local/domain/3/log-throttling'
< medium change watch on `/local/domain/3/log-throttling' - unknown device, ignored
---
> xs_read(/local/domain/1/log-throttling): read error
> qemu: ignoring not-understood drive `/local/domain/1/log-throttling'
> medium change watch on `/local/domain/1/log-throttling' - unknown device, ignored
69,106c69,77
< pt_iomem_map: e_phys=f1020000 maddr=c1a00000 type=0 len=131072 index=2 first_map=1
< pt_iomem_map: e_phys=f1060000 maddr=c1b22000 type=0 len=4096 index=0 first_map=1
< pt_ioport_map: e_phys=c600 pio_base=3000 len=256 index=4 first_map=1
< pt_ioport_map: e_phys=c600 pio_base=3000 len=256 index=4 first_map=0
< ati_gfx_init: guest_pio_bar = 0xc600, host_pio_bar = 0x3000, pio_size=0x100 guest_mmio_bar1=0xe0000000, guest_mmio_bar2=0x0
< platform_fixed_ioport: changed ro/rw state of ROM memory area. now is rw state.
< platform_fixed_ioport: changed ro/rw state of ROM memory area. now is ro state.
< pt_pci_read_config: [00:06:0] Error: Failed to read register with invalid access size alignment. [Offset:0eh][Length:4]
< pt_pci_read_config: [00:06:0] Error: Failed to read register with invalid access size alignment. [Offset:0eh][Length:4]
< pt_pci_read_config: [00:06:0] Error: Failed to read register with invalid access size alignment. [Offset:0eh][Length:4]
< pt_pci_read_config: [00:06:0] Error: Failed to read register with invalid access size alignment. [Offset:0eh][Length:4]
< pt_pci_read_config: [00:06:0] Error: Failed to read register with invalid access size alignment. [Offset:0eh][Length:4]
< pt_pci_read_config: [00:06:0] Error: Failed to read register with invalid access size alignment. [Offset:0eh][Length:4]
< pt_pci_read_config: [00:06:0] Error: Failed to read register with invalid access size alignment. [Offset:0eh][Length:4]
< pt_iomem_map: e_phys=ffffffff maddr=b0000000 type=8 len=268435456 index=0 first_map=0
< pt_iomem_map: e_phys=ffffffff maddr=c1a00000 type=0 len=131072 index=2 first_map=0
< pt_iomem_map: e_phys=e0000000 maddr=b0000000 type=8 len=268435456 index=0 first_map=0
< pt_iomem_map: e_phys=f1020000 maddr=c1a00000 type=0 len=131072 index=2 first_map=0
< pt_ioport_map: e_phys=c600 pio_base=3000 len=256 index=4 first_map=0
< pt_iomem_map: e_phys=ffffffff maddr=c1b22000 type=0 len=4096 index=0 first_map=0
< pt_iomem_map: e_phys=f1060000 maddr=c1b22000 type=0 len=4096 index=0 first_map=0
< pt_iomem_map: e_phys=ffffffff maddr=b0000000 type=8 len=268435456 index=0 first_map=0
< pt_iomem_map: e_phys=ffffffff maddr=c1a00000 type=0 len=131072 index=2 first_map=0
< pt_iomem_map: e_phys=e0000000 maddr=b0000000 type=8 len=268435456 index=0 first_map=0
< pt_iomem_map: e_phys=f1020000 maddr=c1a00000 type=0 len=131072 index=2 first_map=0
< pt_ioport_map: e_phys=c600 pio_base=3000 len=256 index=4 first_map=0
< pt_ioport_map: e_phys=c600 pio_base=3000 len=256 index=4 first_map=0
< pt_msgctrl_reg_write: guest enabling MSI, disable MSI-INTx translation
< pci_intx: intx=1
< pt_msi_disable: Unmap msi with pirq 37
< pt_msgctrl_reg_write: setup msi for dev 30
< pt_msi_setup: msi mapped with pirq 37
< pt_msi_update: Update msi with pirq 37 gvec b0 gflags 0
< pt_iomem_map: e_phys=ffffffff maddr=c1b22000 type=0 len=4096 index=0 first_map=0
< pt_iomem_map: e_phys=f1060000 maddr=c1b22000 type=0 len=4096 index=0 first_map=0
< pt_iomem_map: e_phys=ffffffff maddr=c1b22000 type=0 len=4096 index=0 first_map=0
< shutdown requested in cpu_handle_ioreq
< Issued domain 3 poweroff
---
> pt_iomem_map: e_phys=f1000000 maddr=c1a00000 type=0 len=131072 index=2 first_map=1
> pt_iomem_map: e_phys=f1040000 maddr=c1b22000 type=0 len=4096 index=0 first_map=1
> pt_ioport_map: e_phys=c700 pio_base=3000 len=256 index=4 first_map=1
> pt_ioport_map: e_phys=c700 pio_base=3000 len=256 index=4 first_map=0
> ati_gfx_init: guest_pio_bar = 0xc700, host_pio_bar = 0x3000, pio_size=0x100 guest_mmio_bar1=0xe0000000, guest_mmio_bar2=0x0
> ati_io_regs_read: Requested read of c74c/51020, mapped: 304c/12364
> ati_hw_in: port I/O: 304c, base: 3000, size: 100
> ati_hw_in: ioperm successful
> ati_hw_in: Read: 0
---------------------------------------------------------------------------

}-- End of excerpt from "Dr. Greg Wettstein"

As always,
Dr. G.W. Wettstein, Ph.D.   Enjellic Systems Development, LLC.
4206 N. 19th Ave.           Specializing in information infra-structure
Fargo, ND  58102            development.
PH: 701-281-1686
FAX: 701-281-3949           EMAIL: greg@enjellic.com
------------------------------------------------------------------------------
"Boy, it must not take much to make a phone work.  Looking at
 everthing else here it must be the same way with the INTERNET."
                                -- Francis 'Fritz' Wettstein

Re: Does xen-4.2.0 support VGA passthrough with the virtual machine created by xl command?

[Konrad Rzeszutek Wilk]

On Sat, Nov 17, 2012 at 09:17:36AM -0600, Dr. Greg Wettstein wrote:
> On Nov 14,  3:40pm, "Dr. Greg Wettstein" wrote:
> } Subject: Re: [Xen-devel] Does xen-4.2.0 support VGA passthrough with the v
> 
> Good morning, hope the day is going well for everyone.

Heya!
> 
> > On Nov 13, 10:02am, Ian Campbell wrote:
> > } Subject: Re: [Xen-devel] Does xen-4.2.0 support VGA passthrough with the v
> > 
> > Good afternoon, hope the week is going well for everyone.
> > 
> > > On Tue, 2012-11-13 at 06:30 +0000, Qian Hu wrote:
> > > 
> > > > With spice tool, I have to create a VM by xl command, and now I am
> > > > wondering if it supports VGA passghrouth?
> > 
> > > This list is for the development of Xen. You';d probably have more
> > > luck with these sorts of support requests on the xen-users list.
> > 
> > That would normally be the case but I'm suspicious there are issues
> > with VGA passthrough in 4.2.0.
> 
> I just wanted to follow up to the list on the status of passthrough
> issues.
> 
> We reverted our test machine back to the 2.6.32.45 kernel which we had
> been using in production.  That kernel was based on Jeremy's GIT
> tree.  Using xm and the updated ATI patches which I referenced in my
> original mail passthrough works as it should.
> 
> Passthrough does not work with xl.  Windows started but fell into its
> text mode rescue screen and registered a crash dump.  It flashed the
> screen back and forth between a stipled blue/grey and totally black
> screen a few times and then locked the physical machine up solidly.
> 
> On the next boot I thought about it but declined to register the crash
> dump with Microsoft.... :-)

Ha!
> 
> We then went back and tested the 3.4.18 kernel and with both xm and xl
> the guest faults on the first attempt to do an I/O port access.  All
> factors (windows image, hardware, xen guest config) are held identical
> so the difference would seem to be linked to the PCI passthrough
> implementation between the two kernels.  I've copied Konrad on the
> note since he would seem to be the person most familiar with this
> area.
> 
> I'm including below a diff between a successful qemu-dm passthrough
> session (2.6.32.45) and an unsuccessful session (3.4.18).  It would
> appear 3.4.18 is getting the both the I/O port and memory ranges
> wrong.

Hm, can you also provide the 'lspci -vvv' with the 2.6.32.45 and 3.4.18 kernel?

I am specifically looking to see if there are any:

[from git commit c341ca45ce56143804ef5a8f4db753e554e640b4]
Author: Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>
Date:   Tue Sep 25 16:48:24 2012 -0400

   
	.. snip.. 
    -       Region 0: [virtual] Memory at fe8c0000 (32-bit, non-prefetchable) [size=128K]
    -       Region 1: [virtual] Memory at fe800000 (32-bit, non-prefetchable) [size=512K]
    +       Region 0: Memory at fe8c0000 (32-bit, non-prefetchable) [size=128K]
    +       Region 1: Memory at fe800000 (32-bit, non-prefetchable) [size=512K]
            Region 2: I/O ports at c000 [size=32]
    -       Region 3: [virtual] Memory at fe8e0000 (32-bit, non-prefetchable) [size=16K]
    +       Region 3: Memory at fe8e0000 (32-bit, non-prefetchable) [size=16K]
    
    The [virtual] means that lspci read those entries from SysFS but when
    it read them from the device it got a different value (0xfffffff).
   

Any of those '[virtual]' ones? And how are you reserving the PCI devices?
Are you using xen-pciback.hide on the Linux command line?

Does 'xm dmesg' give you an logs? Do the logs have any warnings?
 
> 
> Let me know if I can forward any additional information or run any
> additional tests.
> 
> Have a good weekend.
> 
> Greg
> 
> qemu-dm log diff: ---------------------------------------------------------
> 1c1
> < domid: 3
> ---
> > domid: 1
> 3,5c3,5
> < Watching /local/domain/0/device-model/3/logdirty/cmd
> < Watching /local/domain/0/device-model/3/command
> < Watching /local/domain/3/cpu
> ---
> > Watching /local/domain/0/device-model/1/logdirty/cmd
> > Watching /local/domain/0/device-model/1/command
> > Watching /local/domain/1/cpu
> 9c9
> < Guest uuid = 7fcefb13-d1ef-105b-e38c-1e1454411e80
> ---
> > Guest uuid = eab6bbbb-4819-b970-a83c-03288a1541ad
> 14c14
> < xs_read(/local/domain/0/device-model/3/xen_extended_power_mgmt): read error
> ---
> > xs_read(/local/domain/0/device-model/1/xen_extended_power_mgmt): read error
> 18,20c18,20
> < xs_read(/local/domain/3/log-throttling): read error
> < qemu: ignoring not-understood drive `/local/domain/3/log-throttling'
> < medium change watch on `/local/domain/3/log-throttling' - unknown device, ignored
> ---
> > xs_read(/local/domain/1/log-throttling): read error
> > qemu: ignoring not-understood drive `/local/domain/1/log-throttling'
> > medium change watch on `/local/domain/1/log-throttling' - unknown device, ignored
> 69,106c69,77
> < pt_iomem_map: e_phys=f1020000 maddr=c1a00000 type=0 len=131072 index=2 first_map=1
> < pt_iomem_map: e_phys=f1060000 maddr=c1b22000 type=0 len=4096 index=0 first_map=1
> < pt_ioport_map: e_phys=c600 pio_base=3000 len=256 index=4 first_map=1
> < pt_ioport_map: e_phys=c600 pio_base=3000 len=256 index=4 first_map=0
> < ati_gfx_init: guest_pio_bar = 0xc600, host_pio_bar = 0x3000, pio_size=0x100 guest_mmio_bar1=0xe0000000, guest_mmio_bar2=0x0
> < platform_fixed_ioport: changed ro/rw state of ROM memory area. now is rw state.
> < platform_fixed_ioport: changed ro/rw state of ROM memory area. now is ro state.
> < pt_pci_read_config: [00:06:0] Error: Failed to read register with invalid access size alignment. [Offset:0eh][Length:4]
> < pt_pci_read_config: [00:06:0] Error: Failed to read register with invalid access size alignment. [Offset:0eh][Length:4]
> < pt_pci_read_config: [00:06:0] Error: Failed to read register with invalid access size alignment. [Offset:0eh][Length:4]
> < pt_pci_read_config: [00:06:0] Error: Failed to read register with invalid access size alignment. [Offset:0eh][Length:4]
> < pt_pci_read_config: [00:06:0] Error: Failed to read register with invalid access size alignment. [Offset:0eh][Length:4]
> < pt_pci_read_config: [00:06:0] Error: Failed to read register with invalid access size alignment. [Offset:0eh][Length:4]
> < pt_pci_read_config: [00:06:0] Error: Failed to read register with invalid access size alignment. [Offset:0eh][Length:4]
> < pt_iomem_map: e_phys=ffffffff maddr=b0000000 type=8 len=268435456 index=0 first_map=0
> < pt_iomem_map: e_phys=ffffffff maddr=c1a00000 type=0 len=131072 index=2 first_map=0
> < pt_iomem_map: e_phys=e0000000 maddr=b0000000 type=8 len=268435456 index=0 first_map=0
> < pt_iomem_map: e_phys=f1020000 maddr=c1a00000 type=0 len=131072 index=2 first_map=0
> < pt_ioport_map: e_phys=c600 pio_base=3000 len=256 index=4 first_map=0
> < pt_iomem_map: e_phys=ffffffff maddr=c1b22000 type=0 len=4096 index=0 first_map=0
> < pt_iomem_map: e_phys=f1060000 maddr=c1b22000 type=0 len=4096 index=0 first_map=0
> < pt_iomem_map: e_phys=ffffffff maddr=b0000000 type=8 len=268435456 index=0 first_map=0
> < pt_iomem_map: e_phys=ffffffff maddr=c1a00000 type=0 len=131072 index=2 first_map=0
> < pt_iomem_map: e_phys=e0000000 maddr=b0000000 type=8 len=268435456 index=0 first_map=0
> < pt_iomem_map: e_phys=f1020000 maddr=c1a00000 type=0 len=131072 index=2 first_map=0
> < pt_ioport_map: e_phys=c600 pio_base=3000 len=256 index=4 first_map=0
> < pt_ioport_map: e_phys=c600 pio_base=3000 len=256 index=4 first_map=0
> < pt_msgctrl_reg_write: guest enabling MSI, disable MSI-INTx translation
> < pci_intx: intx=1
> < pt_msi_disable: Unmap msi with pirq 37
> < pt_msgctrl_reg_write: setup msi for dev 30
> < pt_msi_setup: msi mapped with pirq 37
> < pt_msi_update: Update msi with pirq 37 gvec b0 gflags 0
> < pt_iomem_map: e_phys=ffffffff maddr=c1b22000 type=0 len=4096 index=0 first_map=0
> < pt_iomem_map: e_phys=f1060000 maddr=c1b22000 type=0 len=4096 index=0 first_map=0
> < pt_iomem_map: e_phys=ffffffff maddr=c1b22000 type=0 len=4096 index=0 first_map=0
> < shutdown requested in cpu_handle_ioreq
> < Issued domain 3 poweroff
> ---
> > pt_iomem_map: e_phys=f1000000 maddr=c1a00000 type=0 len=131072 index=2 first_map=1
> > pt_iomem_map: e_phys=f1040000 maddr=c1b22000 type=0 len=4096 index=0 first_map=1
> > pt_ioport_map: e_phys=c700 pio_base=3000 len=256 index=4 first_map=1
> > pt_ioport_map: e_phys=c700 pio_base=3000 len=256 index=4 first_map=0
> > ati_gfx_init: guest_pio_bar = 0xc700, host_pio_bar = 0x3000, pio_size=0x100 guest_mmio_bar1=0xe0000000, guest_mmio_bar2=0x0
> > ati_io_regs_read: Requested read of c74c/51020, mapped: 304c/12364
> > ati_hw_in: port I/O: 304c, base: 3000, size: 100
> > ati_hw_in: ioperm successful
> > ati_hw_in: Read: 0
> ---------------------------------------------------------------------------
> 
> }-- End of excerpt from "Dr. Greg Wettstein"
> 
> As always,
> Dr. G.W. Wettstein, Ph.D.   Enjellic Systems Development, LLC.
> 4206 N. 19th Ave.           Specializing in information infra-structure
> Fargo, ND  58102            development.
> PH: 701-281-1686
> FAX: 701-281-3949           EMAIL: greg@enjellic.com
> ------------------------------------------------------------------------------
> "Boy, it must not take much to make a phone work.  Looking at
>  everthing else here it must be the same way with the INTERNET."
>                                 -- Francis 'Fritz' Wettstein

Re: Q77 IGD instantly crashes on xen-pciback bind.

[Dr. Greg Wettstein]

On Nov 21,  2:57pm, "Dr. Greg Wettstein" wrote:
} Subject: [Xen-devel] Q77 IGD instantly crashes on xen-pciback bind.

> Hi, hope the week is ending well for everyone.

As I was walking out the door I remembered I had been delinquent with
information.  The dom0 kernel is 32-bit 3.14.22 straight from
kernel.org under a 64-bit hypervisor compiled from 4.4.1 sources.

The machine hang is reproducible in the absence of even starting a
guest so the problem appears to be specific to the process of
preparing the device for export through xen-pciback.

Greg

}-- End of excerpt from "Dr. Greg Wettstein"

As always,
Dr. G.W. Wettstein, Ph.D.   Enjellic Systems Development, LLC.
4206 N. 19th Ave.           Specializing in information infra-structure
Fargo, ND  58102            development.
PH: 701-281-1686
FAX: 701-281-3949           EMAIL: greg@enjellic.com
------------------------------------------------------------------------------
"Fools ignore complexity.  Pragmatists suffer it.  Some can avoid it.
Geniuses remove it.
                                -- Perliss' Programming Proverb #58
                                   SIGPLAN National, Sept. 1982

Re: Q77 IGD instantly crashes on xen-pciback bind.

[Pasi Kärkkäinen]

On Fri, Nov 21, 2014 at 03:02:33PM -0600, Dr. Greg Wettstein wrote:
> On Nov 21,  2:57pm, "Dr. Greg Wettstein" wrote:
> } Subject: [Xen-devel] Q77 IGD instantly crashes on xen-pciback bind.
> 
> > Hi, hope the week is ending well for everyone.
> 
> As I was walking out the door I remembered I had been delinquent with
> information.  The dom0 kernel is 32-bit 3.14.22 straight from
> kernel.org under a 64-bit hypervisor compiled from 4.4.1 sources.
>

Wow, quite an old thread :)
 
So you're still seeing the same problem with recent Xen/Linux versions.. 


> The machine hang is reproducible in the absence of even starting a
> guest so the problem appears to be specific to the process of
> preparing the device for export through xen-pciback.
> 

OK.. Maybe Konrad has some thoughts about this.


-- Pasi


> Greg
> 
> }-- End of excerpt from "Dr. Greg Wettstein"
> 
> As always,
> Dr. G.W. Wettstein, Ph.D.   Enjellic Systems Development, LLC.
> 4206 N. 19th Ave.           Specializing in information infra-structure
> Fargo, ND  58102            development.
> PH: 701-281-1686
> FAX: 701-281-3949           EMAIL: greg@enjellic.com
> ------------------------------------------------------------------------------
> "Fools ignore complexity.  Pragmatists suffer it.  Some can avoid it.
> Geniuses remove it.
>                                 -- Perliss' Programming Proverb #58
>                                    SIGPLAN National, Sept. 1982
> 

Re: Q77 IGD instantly crashes on xen-pciback bind.

[Pasi Kärkkäinen]

On Sun, Nov 23, 2014 at 04:05:48PM +0200, Pasi Kärkkäinen wrote:
> On Fri, Nov 21, 2014 at 03:02:33PM -0600, Dr. Greg Wettstein wrote:
> > On Nov 21,  2:57pm, "Dr. Greg Wettstein" wrote:
> > } Subject: [Xen-devel] Q77 IGD instantly crashes on xen-pciback bind.
> > 
> > > Hi, hope the week is ending well for everyone.
> > 
> > As I was walking out the door I remembered I had been delinquent with
> > information.  The dom0 kernel is 32-bit 3.14.22 straight from
> > kernel.org under a 64-bit hypervisor compiled from 4.4.1 sources.
> >
> 
> Wow, quite an old thread :)
> 


Oh, there's also the new thread about this, I guess it's better to continue there :)

http://lists.xenproject.org/archives/html/xen-devel/2014-11/msg02123.html



-- Pasi

 
> So you're still seeing the same problem with recent Xen/Linux versions.. 
> 
> 
> > The machine hang is reproducible in the absence of even starting a
> > guest so the problem appears to be specific to the process of
> > preparing the device for export through xen-pciback.
> > 
> 
> OK.. Maybe Konrad has some thoughts about this.
> 
> 
> -- Pasi
> 
> 
> > Greg
> > 
> > }-- End of excerpt from "Dr. Greg Wettstein"
> > 
> > As always,
> > Dr. G.W. Wettstein, Ph.D.   Enjellic Systems Development, LLC.
> > 4206 N. 19th Ave.           Specializing in information infra-structure
> > Fargo, ND  58102            development.
> > PH: 701-281-1686
> > FAX: 701-281-3949           EMAIL: greg@enjellic.com
> > ------------------------------------------------------------------------------
> > "Fools ignore complexity.  Pragmatists suffer it.  Some can avoid it.
> > Geniuses remove it.
> >                                 -- Perliss' Programming Proverb #58
> >                                    SIGPLAN National, Sept. 1982
> > 
> 
> 
> _______________________________________________
> Xen-devel mailing list
> Xen-devel@lists.xen.org
> http://lists.xen.org/xen-devel

Q77 IGD instantly crashes on xen-pciback bind.

[Dr. Greg Wettstein]

Hi, hope the week is ending well for everyone.

As readers of the list may remember we've kept the ATI primary adapter
passthrough patches current for qemu-traditional on Xen for a number
of years.  Our 'run-passthrough' utility for binding/unbind devices
and running a Windows guest with passthrough have enjoyed a tidy
number of downloads through the years as well.

We are taking on a passthrough project and in the process upgrading
our infrastructure to 4.4.x.  We also need to take on the issue of
passing Intel IGD adapters through to a windows guest.  We are
currently working on an Intel Q77 (DQ77KB) board in preparation for
moving to Q87 boards.

The Intel display adapter is showing up as the standard 00:02.0 PCI
device and things go south pretty quickly.  We create a slot for the
device on the pciback driver and as soon as we bind the device the
machine goes out like a light, no logs or diagnostics, just instantly
stone dead.

This issue is invariant over pciback in vpci or passthrough modes.  It
also occurs instantly when the pciback assignment is made with 'xl
pci-assignable-add'.

We will throw a '#define DEBUG 1' in the top of all the xen-pciback
driver code and see if we can get more information out of it.  I just
wanted to get this in front of the list in case this is a well known
issue or we are headed into other problems we should know about.

I'm assuming that qemu-traditional will handle an IGD primary
passthrough.  If any of the Intel guys are reading this give me a
shout with some directions/advice if we need to roll up our sleeves
and modify the device emulator.

At least it is an "Outlaw Josey Whales bug', not hard to track, leaves
dead machines wherever it goes... :-)

Best wishes for a pleasant weekend to everyone.

Greg

As always,
Dr. G.W. Wettstein, Ph.D.   Enjellic Systems Development, LLC.
4206 N. 19th Ave.           Specializing in information infra-structure
Fargo, ND  58102            development.
PH: 701-281-1686
FAX: 701-281-3949           EMAIL: greg@enjellic.com
------------------------------------------------------------------------------
"The cynics are right nine times out of ten."
                                -- H. L. Mencken

Re: Q77 IGD instantly crashes on xen-pciback bind.

[Pasi Kärkkäinen]

On Fri, Nov 21, 2014 at 02:57:14PM -0600, Dr. Greg Wettstein wrote:
> Hi, hope the week is ending well for everyone.
> 
> As readers of the list may remember we've kept the ATI primary adapter
> passthrough patches current for qemu-traditional on Xen for a number
> of years.  Our 'run-passthrough' utility for binding/unbind devices
> and running a Windows guest with passthrough have enjoyed a tidy
> number of downloads through the years as well.
> 
> We are taking on a passthrough project and in the process upgrading
> our infrastructure to 4.4.x.  We also need to take on the issue of
> passing Intel IGD adapters through to a windows guest.  We are
> currently working on an Intel Q77 (DQ77KB) board in preparation for
> moving to Q87 boards.
> 
> The Intel display adapter is showing up as the standard 00:02.0 PCI
> device and things go south pretty quickly.  We create a slot for the
> device on the pciback driver and as soon as we bind the device the
> machine goes out like a light, no logs or diagnostics, just instantly
> stone dead.
>

This might be a stupid question, but here goes anyway:
Do you have serial console set up? And all the debug/verbose options specified for Xen and Linux? 


Thanks,

-- Pasi

 
> This issue is invariant over pciback in vpci or passthrough modes.  It
> also occurs instantly when the pciback assignment is made with 'xl
> pci-assignable-add'.
> 
> We will throw a '#define DEBUG 1' in the top of all the xen-pciback
> driver code and see if we can get more information out of it.  I just
> wanted to get this in front of the list in case this is a well known
> issue or we are headed into other problems we should know about.
> 
> I'm assuming that qemu-traditional will handle an IGD primary
> passthrough.  If any of the Intel guys are reading this give me a
> shout with some directions/advice if we need to roll up our sleeves
> and modify the device emulator.
> 
> At least it is an "Outlaw Josey Whales bug', not hard to track, leaves
> dead machines wherever it goes... :-)
> 
> Best wishes for a pleasant weekend to everyone.
> 
> Greg
> 
> As always,
> Dr. G.W. Wettstein, Ph.D.   Enjellic Systems Development, LLC.
> 4206 N. 19th Ave.           Specializing in information infra-structure
> Fargo, ND  58102            development.
> PH: 701-281-1686
> FAX: 701-281-3949           EMAIL: greg@enjellic.com
> ------------------------------------------------------------------------------
> "The cynics are right nine times out of ten."
>                                 -- H. L. Mencken
> 

Re: Q77 IGD instantly crashes on xen-pciback bind.

[Dr. Greg Wettstein]

On Nov 23,  4:26pm, Pasi =?iso-8859-1?Q?K=E4rkk=E4inen?= wrote:
} Subject: Re: [Xen-devel] Q77 IGD instantly crashes on xen-pciback bind.

Hi Pasi, hope your week is starting out well, hi to Konrad from Oracle
as well as I see you included him.

> On Fri, Nov 21, 2014 at 02:57:14PM -0600, Dr. Greg Wettstein wrote:
> > Hi, hope the week is ending well for everyone.
> > 
> > As readers of the list may remember we've kept the ATI primary adapter
> > passthrough patches current for qemu-traditional on Xen for a number
> > of years.  Our 'run-passthrough' utility for binding/unbind devices
> > and running a Windows guest with passthrough have enjoyed a tidy
> > number of downloads through the years as well.
> > 
> > We are taking on a passthrough project and in the process upgrading
> > our infrastructure to 4.4.x.  We also need to take on the issue of
> > passing Intel IGD adapters through to a windows guest.  We are
> > currently working on an Intel Q77 (DQ77KB) board in preparation for
> > moving to Q87 boards.
> > 
> > The Intel display adapter is showing up as the standard 00:02.0 PCI
> > device and things go south pretty quickly.  We create a slot for the
> > device on the pciback driver and as soon as we bind the device the
> > machine goes out like a light, no logs or diagnostics, just instantly
> > stone dead.

I'm consolidating your comment from your other response as well so we
keep this on the same thread.

>> As I was walking out the door I remembered I had been delinquent
>> with information.  The dom0 kernel is 32-bit 3.14.22 straight from
>> kernel.org under a 64-bit hypervisor compiled from 4.4.1 sources.

> Wow, quite an old thread :)
>  
> So you're still seeing the same problem with recent Xen/Linux
> versions.. 

Yes, the perils of platforming for 7 year field deployments... :-)

I can certainly build up a toolchain against the HEAD of XEN git and
the most recent release of the kernel if everyone feels that would be
beneficial.

> This might be a stupid question, but here goes anyway: Do you have
> serial console set up? And all the debug/verbose options specified
> for Xen and Linux?

The platform in question doesn't have any serial ports, at least not
surfaced.  We will need to do a bit of wiring if we need to go in that
direction.

Now that I have the machine in a harness in the lab I will stick a
'#define DEBUG 1' in the top of drivers/xen/xen-pciback/pci_stub.c
since that is where the action seems to be going on.

The platform is headed for a measured computing environment so I
thought there may be some type of conflict with tboot holding a
reference to the VGA driver but I verified the issue in a straight
hypervisor boot.

I see that Tiejun Chen from Intel is sorting out issues with respect
to the need to export the ISA bridge into the device emulator in order
to support passthrough on these IGD devices.  I bound the 00:1f.0 ISA
bridge device to pciback and that worked but it did not change the
behavior of the regression.  When the 00:02.0 device is bound to
pciback the display is cleared and the machine dies in its tracks.

I will turn up debugging in pci_stub and see if I can pinpoint where
things blow up, somewhere in pcistub_init_device() I would imagine.

> Thanks,
> 
> -- Pasi

Have a good day.

}-- End of excerpt from Pasi =?iso-8859-1?Q?K=E4rkk=E4inen?=

As always,
Dr. G.W. Wettstein, Ph.D.   Enjellic Systems Development, LLC.
4206 N. 19th Ave.           Specializing in information infra-structure
Fargo, ND  58102            development.
PH: 701-281-1686
FAX: 701-281-3949           EMAIL: greg@enjellic.com
------------------------------------------------------------------------------
"Snow removal teaches all the important elements of succesful corporate
 politics:  1.) Be the first one to work.  2.) Always signal your
 intentions before moving.  3.) Be damn sure you're driving something
 big enough to deal with anything that decides not to get out of your way."
                                -- Dr. G.W. Wettstein
                                   Guerrilla Tactics for Corporate Survival

Re: Q77 IGD instantly crashes on xen-pciback bind.

[Pasi Kärkkäinen]

On Mon, Nov 24, 2014 at 03:59:49AM -0600, Dr. Greg Wettstein wrote:
> On Nov 23,  4:26pm, Pasi =?iso-8859-1?Q?K=E4rkk=E4inen?= wrote:
> } Subject: Re: [Xen-devel] Q77 IGD instantly crashes on xen-pciback bind.
> 
> Hi Pasi, hope your week is starting out well, hi to Konrad from Oracle
> as well as I see you included him.
>

Hello,

 
> > On Fri, Nov 21, 2014 at 02:57:14PM -0600, Dr. Greg Wettstein wrote:
> > > Hi, hope the week is ending well for everyone.
> > > 
> > > As readers of the list may remember we've kept the ATI primary adapter
> > > passthrough patches current for qemu-traditional on Xen for a number
> > > of years.  Our 'run-passthrough' utility for binding/unbind devices
> > > and running a Windows guest with passthrough have enjoyed a tidy
> > > number of downloads through the years as well.
> > > 
> > > We are taking on a passthrough project and in the process upgrading
> > > our infrastructure to 4.4.x.  We also need to take on the issue of
> > > passing Intel IGD adapters through to a windows guest.  We are
> > > currently working on an Intel Q77 (DQ77KB) board in preparation for
> > > moving to Q87 boards.
> > > 
> > > The Intel display adapter is showing up as the standard 00:02.0 PCI
> > > device and things go south pretty quickly.  We create a slot for the
> > > device on the pciback driver and as soon as we bind the device the
> > > machine goes out like a light, no logs or diagnostics, just instantly
> > > stone dead.
> 
> I'm consolidating your comment from your other response as well so we
> keep this on the same thread.
> 

OK.


> >> As I was walking out the door I remembered I had been delinquent
> >> with information.  The dom0 kernel is 32-bit 3.14.22 straight from
> >> kernel.org under a 64-bit hypervisor compiled from 4.4.1 sources.
> 
> > Wow, quite an old thread :)
> >  
> > So you're still seeing the same problem with recent Xen/Linux
> > versions.. 
> 
> Yes, the perils of platforming for 7 year field deployments... :-)
> 
> I can certainly build up a toolchain against the HEAD of XEN git and
> the most recent release of the kernel if everyone feels that would be
> beneficial.
> 
> > This might be a stupid question, but here goes anyway: Do you have
> > serial console set up? And all the debug/verbose options specified
> > for Xen and Linux?
> 
> The platform in question doesn't have any serial ports, at least not
> surfaced.  We will need to do a bit of wiring if we need to go in that
> direction.
> 

You mentioned it's Intel Q77 chipset based motherboard.. 
which means it should have Intel AMT functionality, which provides SOL (Serial-over-LAN),
which you can use as a serial console for Xen.

There are tools (at least amtterm) that you can use on another box to connect to the AMT SOL remotely..



> Now that I have the machine in a harness in the lab I will stick a
> '#define DEBUG 1' in the top of drivers/xen/xen-pciback/pci_stub.c
> since that is where the action seems to be going on.
> 
> The platform is headed for a measured computing environment so I
> thought there may be some type of conflict with tboot holding a
> reference to the VGA driver but I verified the issue in a straight
> hypervisor boot.
> 
> I see that Tiejun Chen from Intel is sorting out issues with respect
> to the need to export the ISA bridge into the device emulator in order
> to support passthrough on these IGD devices.  I bound the 00:1f.0 ISA
> bridge device to pciback and that worked but it did not change the
> behavior of the regression.  When the 00:02.0 device is bound to
> pciback the display is cleared and the machine dies in its tracks.
> 

Yeah, Tiejun is working on upstreaming the IGD passthru patches to Qemu-upstream.

Qemu-dm-traditional already has (most of) the IGD passthru patches. 


Hope that helps,

-- Pasi

> I will turn up debugging in pci_stub and see if I can pinpoint where
> things blow up, somewhere in pcistub_init_device() I would imagine.
> 
> > Thanks,
> > 
> > -- Pasi
> 
> Have a good day.
> 
> }-- End of excerpt from Pasi =?iso-8859-1?Q?K=E4rkk=E4inen?=
> 
> As always,
> Dr. G.W. Wettstein, Ph.D.   Enjellic Systems Development, LLC.
> 4206 N. 19th Ave.           Specializing in information infra-structure
> Fargo, ND  58102            development.
> PH: 701-281-1686
> FAX: 701-281-3949           EMAIL: greg@enjellic.com
> ------------------------------------------------------------------------------
> "Snow removal teaches all the important elements of succesful corporate
>  politics:  1.) Be the first one to work.  2.) Always signal your
>  intentions before moving.  3.) Be damn sure you're driving something
>  big enough to deal with anything that decides not to get out of your way."
>                                 -- Dr. G.W. Wettstein
>                                    Guerrilla Tactics for Corporate Survival

Re: Q77 IGD instantly crashes on xen-pciback bind.

[Dr. Greg Wettstein]

On Nov 24,  1:28pm, Pasi =?iso-8859-1?Q?K=E4rkk=E4inen?= wrote:

> Hello,

Hi, hope the week is going well for everyone.

> > >> As I was walking out the door I remembered I had been delinquent
> > >> with information.  The dom0 kernel is 32-bit 3.14.22 straight from
> > >> kernel.org under a 64-bit hypervisor compiled from 4.4.1 sources.
> > 
> > > Wow, quite an old thread :)
> > >  
> > > So you're still seeing the same problem with recent Xen/Linux
> > > versions.. 
> > 
> > Yes, the perils of platforming for 7 year field deployments... :-)
> > 
> > I can certainly build up a toolchain against the HEAD of XEN git and
> > the most recent release of the kernel if everyone feels that would be
> > beneficial.
> > 
> > > This might be a stupid question, but here goes anyway: Do you have
> > > serial console set up? And all the debug/verbose options specified
> > > for Xen and Linux?
> > 
> > The platform in question doesn't have any serial ports, at least not
> > surfaced.  We will need to do a bit of wiring if we need to go in that
> > direction.

> You mentioned it's Intel Q77 chipset based motherboard..  which
> means it should have Intel AMT functionality, which provides SOL
> (Serial-over-LAN), which you can use as a serial console for Xen.
>
> There are tools (at least amtterm) that you can use on another box
> to connect to the AMT SOL remotely..

So we wired up serial console connectivity to the test box and
repeated the VGA device binding with loglvl=all.  We lost the box
immediately without anything being written to the logs.

So we went hunting.

Interestingly the problem appears to be secondary to a BIOS
configuration option.  This may be specific to this platform but we
wanted to get it documented in the thread in case anyone else runs
into this.

The DQ77KB BIOS we are using has an option for 'IGD flat panel
display'.  The default option is LVDS, setting this to 'disabled'
clears the problem.

I haven't run down where things go wrong in pci_stub but I assume it
does something to the hardware which causes a problem when the video
controller is reset and then shutdown.

> > Now that I have the machine in a harness in the lab I will stick a
> > '#define DEBUG 1' in the top of drivers/xen/xen-pciback/pci_stub.c
> > since that is where the action seems to be going on.
> > 
> > The platform is headed for a measured computing environment so I
> > thought there may be some type of conflict with tboot holding a
> > reference to the VGA driver but I verified the issue in a straight
> > hypervisor boot.
> > 
> > I see that Tiejun Chen from Intel is sorting out issues with respect
> > to the need to export the ISA bridge into the device emulator in order
> > to support passthrough on these IGD devices.  I bound the 00:1f.0 ISA
> > bridge device to pciback and that worked but it did not change the
> > behavior of the regression.  When the 00:02.0 device is bound to
> > pciback the display is cleared and the machine dies in its tracks.

> Yeah, Tiejun is working on upstreaming the IGD passthru patches to
> Qemu-upstream.
>
> Qemu-dm-traditional already has (most of) the IGD passthru patches. 
> 
> Hope that helps,

So we are obviously working with qemu-dm-traditional and with the
IGD/LVDS BIOS configuration issue fixed the adapater passthrough is
working and Windows7 is coming up and detecting the IGD as a standard
VGA display adapter.  Additional invocations of the VM after the first
one result in failed passthrough with a garbled display.

I spent an afternoon wandering through the mailing lists and found
what I think are the two patches which are needed to map the 00:1f.0
ISA bridge device into the guest.  From the discussions surrounding
those patches it appears as if the Windows HD driver needs addresses
managed by that bridge to recognize the IGD device.

I will get those patches wired into qemu-dm-traditional and tested in
between whisky, wine, turkey and napping today.... :-)

I'm hoping that this positively impacts the ability to execute
multiple sessions.  I will report back the results so we have all of
this in the mailing list record.

> -- Pasi

Thanks for offering the pointers, have a good day.

Greg

}-- End of excerpt from Pasi =?iso-8859-1?Q?K=E4rkk=E4inen?=

As always,
Dr. G.W. Wettstein, Ph.D.   Enjellic Systems Development, LLC.
4206 N. 19th Ave.           Specializing in information infra-structure
Fargo, ND  58102            development.
PH: 701-281-1686
FAX: 701-281-3949           EMAIL: greg@enjellic.com
------------------------------------------------------------------------------
"Immortality is an adequate definition of high availability for me."
                                -- Gregory F. Pfister

Re: Q77 IGD instantly crashes on xen-pciback bind.

[Sander Eikelenboom]

[-- Attachment #1: Type: text/plain, Size: 5493 bytes --]


Thursday, November 27, 2014, 11:23:24 AM, you wrote:

> On Nov 24,  1:28pm, Pasi =?iso-8859-1?Q?K=E4rkk=E4inen?= wrote:

>> Hello,

> Hi, hope the week is going well for everyone.

>> > >> As I was walking out the door I remembered I had been delinquent
>> > >> with information.  The dom0 kernel is 32-bit 3.14.22 straight from
>> > >> kernel.org under a 64-bit hypervisor compiled from 4.4.1 sources.
>> > 
>> > > Wow, quite an old thread :)
>> > >  
>> > > So you're still seeing the same problem with recent Xen/Linux
>> > > versions.. 
>> > 
>> > Yes, the perils of platforming for 7 year field deployments... :-)
>> > 
>> > I can certainly build up a toolchain against the HEAD of XEN git and
>> > the most recent release of the kernel if everyone feels that would be
>> > beneficial.
>> > 
>> > > This might be a stupid question, but here goes anyway: Do you have
>> > > serial console set up? And all the debug/verbose options specified
>> > > for Xen and Linux?
>> > 
>> > The platform in question doesn't have any serial ports, at least not
>> > surfaced.  We will need to do a bit of wiring if we need to go in that
>> > direction.

>> You mentioned it's Intel Q77 chipset based motherboard..  which
>> means it should have Intel AMT functionality, which provides SOL
>> (Serial-over-LAN), which you can use as a serial console for Xen.
>>
>> There are tools (at least amtterm) that you can use on another box
>> to connect to the AMT SOL remotely..

> So we wired up serial console connectivity to the test box and
> repeated the VGA device binding with loglvl=all.  We lost the box
> immediately without anything being written to the logs.

> So we went hunting.

> Interestingly the problem appears to be secondary to a BIOS
> configuration option.  This may be specific to this platform but we
> wanted to get it documented in the thread in case anyone else runs
> into this.

> The DQ77KB BIOS we are using has an option for 'IGD flat panel
> display'.  The default option is LVDS, setting this to 'disabled'
> clears the problem.

> I haven't run down where things go wrong in pci_stub but I assume it
> does something to the hardware which causes a problem when the video
> controller is reset and then shutdown.

>> > Now that I have the machine in a harness in the lab I will stick a
>> > '#define DEBUG 1' in the top of drivers/xen/xen-pciback/pci_stub.c
>> > since that is where the action seems to be going on.
>> > 
>> > The platform is headed for a measured computing environment so I
>> > thought there may be some type of conflict with tboot holding a
>> > reference to the VGA driver but I verified the issue in a straight
>> > hypervisor boot.
>> > 
>> > I see that Tiejun Chen from Intel is sorting out issues with respect
>> > to the need to export the ISA bridge into the device emulator in order
>> > to support passthrough on these IGD devices.  I bound the 00:1f.0 ISA
>> > bridge device to pciback and that worked but it did not change the
>> > behavior of the regression.  When the 00:02.0 device is bound to
>> > pciback the display is cleared and the machine dies in its tracks.

>> Yeah, Tiejun is working on upstreaming the IGD passthru patches to
>> Qemu-upstream.
>>
>> Qemu-dm-traditional already has (most of) the IGD passthru patches. 
>> 
>> Hope that helps,

> So we are obviously working with qemu-dm-traditional and with the
> IGD/LVDS BIOS configuration issue fixed the adapater passthrough is
> working and Windows7 is coming up and detecting the IGD as a standard
> VGA display adapter.  Additional invocations of the VM after the first
> one result in failed passthrough with a garbled display.

This is probably due to the current lack of slot/bus reset in xen-pciback,
Konrad has a preliminary kernel patch for xen-pciback that does this.
I have attached the patch, though it has some rough edges in the design :-)
 
I'm currently running with his 3.19 xen-pciback patches series + the preliminary
patch for slot/bus reset and rebooting a guest with vga/pci passthrough now
works. (i'm running with a radeon card, passed through as a secondary card
to the emulated qemu one, in a linux guest using qemu-xen, so i can't help
you with your other questions and problems).

--
Sander

> I spent an afternoon wandering through the mailing lists and found
> what I think are the two patches which are needed to map the 00:1f.0
> ISA bridge device into the guest.  From the discussions surrounding
> those patches it appears as if the Windows HD driver needs addresses
> managed by that bridge to recognize the IGD device.

> I will get those patches wired into qemu-dm-traditional and tested in
> between whisky, wine, turkey and napping today.... :-)

> I'm hoping that this positively impacts the ability to execute
> multiple sessions.  I will report back the results so we have all of
> this in the mailing list record.

>> -- Pasi

> Thanks for offering the pointers, have a good day.

> Greg

> }-- End of excerpt from Pasi =?iso-8859-1?Q?K=E4rkk=E4inen?=

> As always,
> Dr. G.W. Wettstein, Ph.D.   Enjellic Systems Development, LLC.
> 4206 N. 19th Ave.           Specializing in information infra-structure
> Fargo, ND  58102            development.
> PH: 701-281-1686
> FAX: 701-281-3949           EMAIL: greg@enjellic.com
> ------------------------------------------------------------------------------
> "Immortality is an adequate definition of high availability for me."
>                                 -- Gregory F. Pfister



[-- Attachment #2: 0001-xen-pciback-Implement-PCI-reset-slot-or-bus-with-do_.patch --]
[-- Type: application/octet-stream, Size: 10270 bytes --]

From fa262362d924da9f63b0739c470bd1a67c36bfae Mon Sep 17 00:00:00 2001
From: Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>
Date: Tue, 8 Jul 2014 11:22:45 -0400
Subject: [PATCH] xen/pciback: Implement PCI reset slot or bus with 'do_flr'
 SysFS attribute

The life-cycle of a PCI device in Xen pciback is complex
and is constrained by the PCI generic locking mechanism.

It starts with the device being binded to us - for which
we do a device function reset (and done via SysFS
so the PCI lock is held)

If the device is unbinded from us - we also do a function
reset (also done via SysFS so the PCI lock is held).

If the device is un-assigned from a guest - we do a function
reset (no PCI lock).

All on the individual PCI function level (so bus:device:function).

Unfortunatly a function reset is not adequate for certain
PCIe devices. The reset for an individual PCI function "means
device must support FLR (PCIe or AF), PM reset on D3hot->D0
device specific reset, or be a singleton device on a bus
a secondary bus reset.  FLR does not have widespread support,
reset is not very reliable, and bus topology is dictated by the
and device design.  We need to provide a means for a user to
a bus reset in cases where the existing mechanisms are not
 or not reliable. " (Adam Williamson, 'vfio-pci: PCI hot reset
interface' commit 8b27ee60bfd6bbb84d2df28fa706c5c5081066ca).

As such to do a slot or a bus reset is we need another mechanism.
This is not exposed SysFS as there is no good way of exposing
a bus topology there.

This is due to the complexity - we MUST know that the different
functions off a PCIe device are not in use by other drivers, or
if they are in use (say one of them is assigned to a guest
and the other is idle) - it is still OK to reset the slot
(assuming both of them are owned by Xen pciback).

This patch does that by doing an slot or bus reset (if
slot not supported) if all of the functions of a PCIe
device belong to Xen PCIback. We do not care if the device is
in-use as we depend on the toolstack to be aware of this -
however if it is we will WARN the user.

Due to the complexity with the PCI lock we cannot do
the reset when a device is binded ('echo $BDF > bind')
or when unbinded ('echo $BDF > unbind') as the pci_[slot|bus]_reset
also take the same lock resulting in a dead-lock.

Putting the reset function in a workqueue or thread
won't work either - as we have to do the reset function
outside the 'unbind' context (it holds the PCI lock).
But once you 'unbind' a device the device is no longer
under the ownership of Xen pciback and the pci_set_drvdata
has been reset so we cannot use a thread for this.

Instead of doing all this complex dance, we depend on the toolstack
doing the right thing. As such implement the 'do_flr' SysFS attribute
which 'xl' uses when a device is detached or attached from/to a guest.
It bypasses the need to worry about the PCI lock.

To not inadvertly do a bus reset that would affect devices that
are in use by other drivers (other than Xen pciback) prior
to the reset we check that all of the devices under the bridge
are owned by Xen pciback. If they are not we do not do
the bus (or slot) reset.

We also warn the user if the device is in use - but still
continue with the reset. This should not happen as the toolstack
also does the check.

Signed-off-by: Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>
---
 Documentation/ABI/testing/sysfs-driver-pciback |  12 +++
 drivers/xen/xen-pciback/pci_stub.c             | 139 ++++++++++++++++++++-----
 2 files changed, 124 insertions(+), 27 deletions(-)

diff --git a/Documentation/ABI/testing/sysfs-driver-pciback b/Documentation/ABI/testing/sysfs-driver-pciback
index 6a733bf..89dcf71 100644
--- a/Documentation/ABI/testing/sysfs-driver-pciback
+++ b/Documentation/ABI/testing/sysfs-driver-pciback
@@ -11,3 +11,15 @@ Description:
                 #echo 00:19.0-E0:2:FF > /sys/bus/pci/drivers/pciback/quirks
                 will allow the guest to read and write to the configuration
                 register 0x0E.
+
+
+What:           /sys/bus/pci/drivers/pciback/do_flr
+Date:          	December 2014
+KernelVersion:  3.19
+Contact:        xen-devel@lists.xenproject.org
+Description:
+                An option to slot or bus reset an PCI device owned by
+                Xen PCI backend. Writing a string of DDDD:BB:DD.F will cause
+                the driver to perform an slot or bus reset if the device
+                supports. It also checks to make sure that all of the devices
+                under the bridge are owned by Xen PCI backend.
diff --git a/drivers/xen/xen-pciback/pci_stub.c b/drivers/xen/xen-pciback/pci_stub.c
index ff27efa..032f17a8 100644
--- a/drivers/xen/xen-pciback/pci_stub.c
+++ b/drivers/xen/xen-pciback/pci_stub.c
@@ -100,14 +100,9 @@ static void pcistub_device_release(struct kref *kref)
 
 	xen_unregister_device_domain_owner(dev);
 
-	/* Call the reset function which does not take lock as this
-	 * is called from "unbind" which takes a device_lock mutex.
-	 */
-	__pci_reset_function_locked(dev);
+	/* Reset is done by the toolstack by using 'do_flr' on the SysFS. */
 	if (pci_load_and_free_saved_state(dev, &dev_data->pci_saved_state))
 		dev_info(&dev->dev, "Could not reload PCI state\n");
-	else
-		pci_restore_state(dev);
 
 	if (dev->msix_cap) {
 		struct physdev_pci_device ppdev = {
@@ -123,9 +118,6 @@ static void pcistub_device_release(struct kref *kref)
 				 err);
 	}
 
-	/* Disable the device */
-	xen_pcibk_reset_device(dev);
-
 	kfree(dev_data);
 	pci_set_drvdata(dev, NULL);
 
@@ -242,6 +234,86 @@ struct pci_dev *pcistub_get_pci_dev(struct xen_pcibk_device *pdev,
 	return found_dev;
 }
 
+struct wrapper_args {
+	struct pci_dev *dev;
+	int in_use;
+};
+
+static int pcistub_pci_walk_wrapper(struct pci_dev *dev, void *data)
+{
+	struct pcistub_device *psdev, *found_psdev = NULL;
+	struct wrapper_args *arg = data;
+	unsigned long flags;
+
+	spin_lock_irqsave(&pcistub_devices_lock, flags);
+	list_for_each_entry(psdev, &pcistub_devices, dev_list) {
+		if (psdev->dev == dev) {
+			found_psdev = psdev;
+			if (psdev->pdev)
+				arg->in_use++;
+			break;
+		}
+	}
+	spin_unlock_irqrestore(&pcistub_devices_lock, flags);
+	dev_dbg(&dev->dev, "%s\n", found_psdev ? "OK" : "not owned by us!");
+
+	if (!found_psdev)
+		arg->dev = dev;
+	return found_psdev ? 0 : 1;
+}
+
+static int pcistub_reset_pci_dev(struct pci_dev *dev)
+{
+	struct wrapper_args arg = { .dev = NULL, .in_use = 0 };
+	bool slot = false, bus = false;
+	int rc;
+
+	if (!dev)
+		return -EINVAL;
+
+	if (!pci_probe_reset_slot(dev->slot))
+		slot = true;
+	else if (!pci_probe_reset_bus(dev->bus)) {
+		/* We won't attempt to reset a root bridge. */
+		if (!pci_is_root_bus(dev->bus))
+			bus = true;
+	}
+	dev_dbg(&dev->dev, "resetting (FLR, D3, %s %s) the device\n",
+		slot ? "slot" : "", bus ? "bus" : "");
+
+	pci_walk_bus(dev->bus, pcistub_pci_walk_wrapper, &arg);
+
+	if (arg.in_use)
+		dev_err(&dev->dev, "is in use!\n");
+
+	/*
+	 * Takes the PCI lock. OK to do it as we are never called
+	 * from 'unbind' state and don't deadlock.
+	 */
+	if (!pci_load_saved_state(dev, dev_data->pci_saved_state))
+		pci_restore_state(dev);
+
+	pci_reset_function(dev);
+
+	/* This disables the device. */
+	xen_pcibk_reset_device(dev);
+
+	/* And cleanup up our emulated fields. */
+	xen_pcibk_config_reset_dev(dev);
+
+	if (!bus && !slot)
+		return 0;
+
+	/* All slots or devices under the bus should be part of pcistub! */
+	if (arg.dev) {
+		dev_err(&dev->dev, "depends on: %s!\n", pci_name(arg.dev));
+		return -EBUSY;
+	}
+	rc = slot ? pci_try_reset_slot(dev->slot): pci_try_reset_bus(dev->bus);
+
+	return rc;
+}
+
 /*
  * Called when:
  *  - XenBus state has been reconfigure (pci unplug). See xen_pcibk_remove_device
@@ -277,25 +349,10 @@ void pcistub_put_pci_dev(struct pci_dev *dev)
 	* pcistub and xen_pcibk when AER is in processing
 	*/
 	down_write(&pcistub_sem);
-	/* Cleanup our device
-	 * (so it's ready for the next domain)
+	/* Cleanup our device (so it's ready for the next domain)
+	 * That is the job of the toolstack which has to call 'do_flr' before
+	 * providing the PCI device to a guest (see pcistub_do_flr).
 	 */
-	device_lock_assert(&dev->dev);
-	dev_data = pci_get_drvdata(dev);
-	ret = pci_load_saved_state(dev, dev_data->pci_saved_state);
-	if (ret < 0)
-		dev_warn(&dev->dev, "Could not reload PCI state\n");
-	else {
-		__pci_reset_function_locked(dev);
-		/*
-		 * The usual sequence is pci_save_state & pci_restore_state
-		 * but the guest might have messed the configuration space up.
-		 * Use the initial version (when device was bound to us).
-		 */
-		pci_restore_state(dev);
-	}
-	/* This disables the device. */
-	xen_pcibk_reset_device(dev);
 
 	/* And cleanup up our emulated fields. */
 	xen_pcibk_config_reset_dev(dev);
@@ -1389,6 +1446,29 @@ static ssize_t permissive_show(struct device_driver *drv, char *buf)
 static DRIVER_ATTR(permissive, S_IRUSR | S_IWUSR, permissive_show,
 		   permissive_add);
 
+static ssize_t pcistub_do_flr(struct device_driver *drv, const char *buf,
+				size_t count)
+{
+	int domain, bus, slot, func;
+	int err;
+	struct pcistub_device *psdev;
+
+	err = str_to_slot(buf, &domain, &bus, &slot, &func);
+	if (err)
+		goto out;
+
+	psdev = pcistub_device_find(domain, bus, slot, func);
+	if (psdev) {
+		err = pcistub_reset_pci_dev(psdev->dev);
+		pcistub_device_put(psdev);
+	} else
+		err = -ENODEV;
+out:
+	if (!err)
+		err = count;
+	return err;
+}
+static DRIVER_ATTR(do_flr, S_IWUSR, NULL, pcistub_do_flr);
 static void pcistub_exit(void)
 {
 	driver_remove_file(&xen_pcibk_pci_driver.driver, &driver_attr_new_slot);
@@ -1402,6 +1482,8 @@ static void pcistub_exit(void)
 			   &driver_attr_irq_handlers);
 	driver_remove_file(&xen_pcibk_pci_driver.driver,
 			   &driver_attr_irq_handler_state);
+	driver_remove_file(&xen_pcibk_pci_driver.driver,
+			   &driver_attr_do_flr);
 	pci_unregister_driver(&xen_pcibk_pci_driver);
 }
 
@@ -1495,6 +1577,9 @@ static int __init pcistub_init(void)
 	if (!err)
 		err = driver_create_file(&xen_pcibk_pci_driver.driver,
 					&driver_attr_irq_handler_state);
+	if (!err)
+		err = driver_create_file(&xen_pcibk_pci_driver.driver,
+					&driver_attr_do_flr);
 	if (err)
 		pcistub_exit();
 
-- 
1.9.3


[-- Attachment #3: Type: text/plain, Size: 126 bytes --]

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: Q77 IGD instantly crashes on xen-pciback bind.

[Dr. Greg Wettstein]

On Nov 27, 12:11pm, Sander Eikelenboom wrote:
} Subject: Re: [Xen-devel] Q77 IGD instantly crashes on xen-pciback bind.

Hi, hope the week has gone well for everyone.

> > So we are obviously working with qemu-dm-traditional and with the
> > IGD/LVDS BIOS configuration issue fixed the adapater passthrough is
> > working and Windows7 is coming up and detecting the IGD as a standard
> > VGA display adapter.  Additional invocations of the VM after the first
> > one result in failed passthrough with a garbled display.

> This is probably due to the current lack of slot/bus reset in
> xen-pciback, Konrad has a preliminary kernel patch for xen-pciback
> that does this.  I have attached the patch, though it has some rough
> edges in the design :-)
>
> I'm currently running with his 3.19 xen-pciback patches series + the
> preliminary patch for slot/bus reset and rebooting a guest with
> vga/pci passthrough now works. (i'm running with a radeon card,
> passed through as a secondary card to the emulated qemu one, in a
> linux guest using qemu-xen, so i can't help you with your other
> questions and problems).

Thanks for taking the time for respond and forward along the patch.

I back ported the do_flr patch into the 3.14.x kernel and spent some
time working with it.  I thought it might be useful to others to
document what we ran into.

First of all the issue with the unsuccessful boot of Windows after the
first invocation doesn't appear to have anything to do with resetting
the card.  This was fixed by installing the most recent version of the
Intel HD drivers in the Windows guest.

If IGD passthrough is done without the HD drivers Windows 7 appears to
use its standard VGA driver which seems to be able to initialize and
run the IGD device but does not appear to shutdown the device in a
manner in which it can be re-started.  After the first invocation of
the guest is shutdown the screen goes to a solid color.  Subsequent
invocations result in the flashing multi-color screens which others
have documented.

With the HD drivers installed IGD passthrough works fine through
multiple invocations of a guest with the stock xen-pciback in 3.14.x.
We ran 40-50 repetitive Windows guest invocations and every one was
completely deterministic.

That being said we ran into an issue which we wanted to bounce off the
list in the context of this thread.

One of the things we were not able to do was to successfuly re-start
the IGD display for dom0.  After spending a lot of time going through
Konrad's reset code it appears the IGD devices in the Q77 and Q87
chipset implementations we are looking at are not advertising reset
functions which can be used.

It appears that the IGD devices are advertising that they need a
Device Specific Initialization (DSI) sequence in order to be enabled
or powered up, if we interpret the output of lscpi properly, ie:

---------------------------------------------------------------------------
00:02.0 VGA compatible controller: Intel Corporation Device 0152 (rev 09) (prog-if 00 [VGA controller])
        Subsystem: Intel Corporation Device 2036
        Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx-
        Status: Cap+ 66MHz- UDF- FastB2B+ ParErr- DEVSEL=fast >TAbort- <TAbort-
<MAbort- >SERR- <PERR- INTx-
        Latency: 0
        Interrupt: pin A routed to IRQ 11
        Region 0: Memory at f7800000 (64-bit, non-prefetchable) [size=4M]
        Region 2: Memory at e0000000 (64-bit, prefetchable) [size=256M]
        Region 4: I/O ports at f000 [size=64]
        Expansion ROM at <unassigned> [disabled]
        Capabilities: [90] MSI: Mask- 64bit- Count=1/1 Enable-
                Address: 00000000  Data: 0000
        Capabilities: [d0] Power Management version 2
                Flags: PMEClk- DSI+ D1- D2- AuxCurrent=0mA PME(D0-,D1-,D2-,D3hot-,D3cold-)
                Status: D0 PME-Enable- DSel=0 DScale=0 PME-
        Capabilities: [a4] PCIe advanced features <?>
---------------------------------------------------------------------------

The ATI adapters which we have a lot of passthrough experience with
are FLR capable and can be re-enabled after passthrough by writing a
'1' to the enable pseudo-file in the /sys/bus/pci/devices/BDF
directory.  The IGD adapters seem to fail to respond to a request to
be re-enabled.

We would certainly be interested in any suggestions the collective may
have with respect to how to get the adapter turned back on and/or
implement a reset.

> Sander

Have a good weekend.

Greg

}-- End of excerpt from Sander Eikelenboom

As always,
Dr. G.W. Wettstein, Ph.D.   Enjellic Systems Development, LLC.
4206 N. 19th Ave.           Specializing in information infra-structure
Fargo, ND  58102            development.
PH: 701-281-1686
FAX: 701-281-3949           EMAIL: greg@enjellic.com
------------------------------------------------------------------------------
"Laugh now but you won't be laughing when we find you laying on the
 side of the road dead."
                                -- Betty Wettstein
                                   At the Lake

Re: Q77 IGD instantly crashes on xen-pciback bind.

[Pasi Kärkkäinen]

On Fri, Nov 28, 2014 at 08:00:40PM -0600, Dr. Greg Wettstein wrote:
> On Nov 27, 12:11pm, Sander Eikelenboom wrote:
> } Subject: Re: [Xen-devel] Q77 IGD instantly crashes on xen-pciback bind.
> 
> Hi, hope the week has gone well for everyone.
>

Hello, hopefully your weekend is going well.

 
> > > So we are obviously working with qemu-dm-traditional and with the
> > > IGD/LVDS BIOS configuration issue fixed the adapater passthrough is
> > > working and Windows7 is coming up and detecting the IGD as a standard
> > > VGA display adapter.  Additional invocations of the VM after the first
> > > one result in failed passthrough with a garbled display.
> 
> > This is probably due to the current lack of slot/bus reset in
> > xen-pciback, Konrad has a preliminary kernel patch for xen-pciback
> > that does this.  I have attached the patch, though it has some rough
> > edges in the design :-)
> >
> > I'm currently running with his 3.19 xen-pciback patches series + the
> > preliminary patch for slot/bus reset and rebooting a guest with
> > vga/pci passthrough now works. (i'm running with a radeon card,
> > passed through as a secondary card to the emulated qemu one, in a
> > linux guest using qemu-xen, so i can't help you with your other
> > questions and problems).
> 
> Thanks for taking the time for respond and forward along the patch.
> 
> I back ported the do_flr patch into the 3.14.x kernel and spent some
> time working with it.  I thought it might be useful to others to
> document what we ran into.
> 
> First of all the issue with the unsuccessful boot of Windows after the
> first invocation doesn't appear to have anything to do with resetting
> the card.  This was fixed by installing the most recent version of the
> Intel HD drivers in the Windows guest.
> 
> If IGD passthrough is done without the HD drivers Windows 7 appears to
> use its standard VGA driver which seems to be able to initialize and
> run the IGD device but does not appear to shutdown the device in a
> manner in which it can be re-started.  After the first invocation of
> the guest is shutdown the screen goes to a solid color.  Subsequent
> invocations result in the flashing multi-color screens which others
> have documented.
> 
> With the HD drivers installed IGD passthrough works fine through
> multiple invocations of a guest with the stock xen-pciback in 3.14.x.
> We ran 40-50 repetitive Windows guest invocations and every one was
> completely deterministic.
>

It would be nice if you could let us know the exact Intel IGD Windows driver version
that worked well for you? It might be a good reference for others aswell.


Thanks,
 
-- Pasi

Re: Q77 IGD instantly crashes on xen-pciback bind.

[Konrad Rzeszutek Wilk]

On Fri, Nov 28, 2014 at 08:00:40PM -0600, Dr. Greg Wettstein wrote:
> On Nov 27, 12:11pm, Sander Eikelenboom wrote:
> } Subject: Re: [Xen-devel] Q77 IGD instantly crashes on xen-pciback bind.
> 
> Hi, hope the week has gone well for everyone.
> 
> > > So we are obviously working with qemu-dm-traditional and with the
> > > IGD/LVDS BIOS configuration issue fixed the adapater passthrough is
> > > working and Windows7 is coming up and detecting the IGD as a standard
> > > VGA display adapter.  Additional invocations of the VM after the first
> > > one result in failed passthrough with a garbled display.
> 
> > This is probably due to the current lack of slot/bus reset in
> > xen-pciback, Konrad has a preliminary kernel patch for xen-pciback
> > that does this.  I have attached the patch, though it has some rough
> > edges in the design :-)
> >
> > I'm currently running with his 3.19 xen-pciback patches series + the
> > preliminary patch for slot/bus reset and rebooting a guest with
> > vga/pci passthrough now works. (i'm running with a radeon card,
> > passed through as a secondary card to the emulated qemu one, in a
> > linux guest using qemu-xen, so i can't help you with your other
> > questions and problems).
> 
> Thanks for taking the time for respond and forward along the patch.
> 
> I back ported the do_flr patch into the 3.14.x kernel and spent some
> time working with it.  I thought it might be useful to others to
> document what we ran into.
> 
> First of all the issue with the unsuccessful boot of Windows after the
> first invocation doesn't appear to have anything to do with resetting
> the card.  This was fixed by installing the most recent version of the
> Intel HD drivers in the Windows guest.
> 
> If IGD passthrough is done without the HD drivers Windows 7 appears to
> use its standard VGA driver which seems to be able to initialize and
> run the IGD device but does not appear to shutdown the device in a
> manner in which it can be re-started.  After the first invocation of
> the guest is shutdown the screen goes to a solid color.  Subsequent
> invocations result in the flashing multi-color screens which others
> have documented.
> 
> With the HD drivers installed IGD passthrough works fine through
> multiple invocations of a guest with the stock xen-pciback in 3.14.x.
> We ran 40-50 repetitive Windows guest invocations and every one was
> completely deterministic.
> 
> That being said we ran into an issue which we wanted to bounce off the
> list in the context of this thread.
> 
> One of the things we were not able to do was to successfuly re-start
> the IGD display for dom0.  After spending a lot of time going through
> Konrad's reset code it appears the IGD devices in the Q77 and Q87
> chipset implementations we are looking at are not advertising reset
> functions which can be used.
> 
> It appears that the IGD devices are advertising that they need a
> Device Specific Initialization (DSI) sequence in order to be enabled
> or powered up, if we interpret the output of lscpi properly, ie:
> 
> ---------------------------------------------------------------------------
> 00:02.0 VGA compatible controller: Intel Corporation Device 0152 (rev 09) (prog-if 00 [VGA controller])
>         Subsystem: Intel Corporation Device 2036
>         Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx-
>         Status: Cap+ 66MHz- UDF- FastB2B+ ParErr- DEVSEL=fast >TAbort- <TAbort-
> <MAbort- >SERR- <PERR- INTx-
>         Latency: 0
>         Interrupt: pin A routed to IRQ 11
>         Region 0: Memory at f7800000 (64-bit, non-prefetchable) [size=4M]
>         Region 2: Memory at e0000000 (64-bit, prefetchable) [size=256M]
>         Region 4: I/O ports at f000 [size=64]
>         Expansion ROM at <unassigned> [disabled]
>         Capabilities: [90] MSI: Mask- 64bit- Count=1/1 Enable-
>                 Address: 00000000  Data: 0000
>         Capabilities: [d0] Power Management version 2
>                 Flags: PMEClk- DSI+ D1- D2- AuxCurrent=0mA PME(D0-,D1-,D2-,D3hot-,D3cold-)
>                 Status: D0 PME-Enable- DSel=0 DScale=0 PME-
>         Capabilities: [a4] PCIe advanced features <?>
> ---------------------------------------------------------------------------
> 
> The ATI adapters which we have a lot of passthrough experience with
> are FLR capable and can be re-enabled after passthrough by writing a
> '1' to the enable pseudo-file in the /sys/bus/pci/devices/BDF
> directory.  The IGD adapters seem to fail to respond to a request to
> be re-enabled.
> 
> We would certainly be interested in any suggestions the collective may
> have with respect to how to get the adapter turned back on and/or
> implement a reset.

CC-ing Chien who has been looking at adding IGD passthrough support
in the qemu-upstream. Perhaps he can provide some ideas.

Thanks.
> 
> > Sander
> 
> Have a good weekend.
> 
> Greg
> 
> }-- End of excerpt from Sander Eikelenboom
> 
> As always,
> Dr. G.W. Wettstein, Ph.D.   Enjellic Systems Development, LLC.
> 4206 N. 19th Ave.           Specializing in information infra-structure
> Fargo, ND  58102            development.
> PH: 701-281-1686
> FAX: 701-281-3949           EMAIL: greg@enjellic.com
> ------------------------------------------------------------------------------
> "Laugh now but you won't be laughing when we find you laying on the
>  side of the road dead."
>                                 -- Betty Wettstein
>                                    At the Lake

Re: Q77 IGD instantly crashes on xen-pciback bind.

[Dr. Greg Wettstein]

On Nov 29,  1:09pm, Pasi =?iso-8859-1?Q?K=E4rkk=E4inen?= wrote:
} Subject: Re: [Xen-devel] Q77 IGD instantly crashes on xen-pciback bind.

Good morning Pasi, thanks for taking the time to follow up.

> > > > So we are obviously working with qemu-dm-traditional and with the
> > > > IGD/LVDS BIOS configuration issue fixed the adapater passthrough is
> > > > working and Windows7 is coming up and detecting the IGD as a standard
> > > > VGA display adapter.  Additional invocations of the VM after the first
> > > > one result in failed passthrough with a garbled display.
> > 
> > > This is probably due to the current lack of slot/bus reset in
> > > xen-pciback, Konrad has a preliminary kernel patch for xen-pciback
> > > that does this.  I have attached the patch, though it has some rough
> > > edges in the design :-)
> > >
> > > I'm currently running with his 3.19 xen-pciback patches series + the
> > > preliminary patch for slot/bus reset and rebooting a guest with
> > > vga/pci passthrough now works. (i'm running with a radeon card,
> > > passed through as a secondary card to the emulated qemu one, in a
> > > linux guest using qemu-xen, so i can't help you with your other
> > > questions and problems).
> > 
> > Thanks for taking the time for respond and forward along the patch.
> > 
> > I back ported the do_flr patch into the 3.14.x kernel and spent some
> > time working with it.  I thought it might be useful to others to
> > document what we ran into.
> > 
> > First of all the issue with the unsuccessful boot of Windows after the
> > first invocation doesn't appear to have anything to do with resetting
> > the card.  This was fixed by installing the most recent version of the
> > Intel HD drivers in the Windows guest.
> > 
> > If IGD passthrough is done without the HD drivers Windows 7 appears to
> > use its standard VGA driver which seems to be able to initialize and
> > run the IGD device but does not appear to shutdown the device in a
> > manner in which it can be re-started.  After the first invocation of
> > the guest is shutdown the screen goes to a solid color.  Subsequent
> > invocations result in the flashing multi-color screens which others
> > have documented.
> > 
> > With the HD drivers installed IGD passthrough works fine through
> > multiple invocations of a guest with the stock xen-pciback in 3.14.x.
> > We ran 40-50 repetitive Windows guest invocations and every one was
> > completely deterministic.

> It would be nice if you could let us know the exact Intel IGD
> Windows driver version that worked well for you? It might be a good
> reference for others aswell.

I just fired up a pass-through session to check on this.  The driver
version and date are as follows:

	Driver Date:		09/26/2014
	Driver Version:		10.18.10.29458

They were the most current version of the driver available on Intel's
website as of a week ago.

As soon as we sort out the details on whether or not we can get the
IGD device re-started we will put a patch up on our FTP server with
the changes needed to implement both ATI and IGD passthrough on 4.4.1
as a resource to the community.

> Thanks,
>  
> -- Pasi

Have a good day.

Greg

}-- End of excerpt from Pasi =?iso-8859-1?Q?K=E4rkk=E4inen?=

As always,
Dr. G.W. Wettstein, Ph.D.   Enjellic Systems Development, LLC.
4206 N. 19th Ave.           Specializing in information infra-structure
Fargo, ND  58102            development.
PH: 701-281-1686
FAX: 701-281-3949           EMAIL: greg@enjellic.com
------------------------------------------------------------------------------
"In the future, company names will be a 32-character hex string."
                                -- Bruce Schneier