Re: qemu-xen-dir + PCI passthrough = BOOM

[Anthony PERARD <anthony.perard@citrix.com>]

On Thu, Jan 09, 2014 at 10:28:47PM -0500, Konrad Rzeszutek Wilk wrote:
> On Thu, Jan 09, 2014 at 02:56:24PM +0000, Anthony PERARD wrote:
> > On Wed, Jan 08, 2014 at 02:44:51PM -0500, Konrad Rzeszutek Wilk wrote:
> > > On Wed, Dec 18, 2013 at 02:48:24PM +0000, Anthony PERARD wrote:
> > > > On Mon, Dec 16, 2013 at 10:08:16AM -0500, Konrad Rzeszutek Wilk wrote:
> > > > > On Fri, Dec 06, 2013 at 04:03:10PM +0000, Wei Liu wrote:
> > > > > > On Fri, Dec 06, 2013 at 04:00:18PM +0000, Wei Liu wrote:
> > > > > > [...]
> > > > > > > > Those Xen report something like:
> > > > > > > > (XEN) page_alloc.c:1460:d0 Over-allocation for domain 46: 131329 >
> > > > > > > > 131328
> > > > > > > > (XEN) memory.c:132:d0 Could not allocate order=0 extent: id=46
> > > > > > > > memflags=0 (62 of 64)
> > > > > > > > 
> > > > > > > > ?
> > > > > > > > 
> > > > > > > > (I tryied to reproduce the issue by simply add many emulated e1000 in
> > > > > > > > QEMU :) )
> > > > > > > > 
> > > > 
> > > > > -bash-4.1# lspci -s 01:00.0 -v 
> > > > > 01:00.0 Ethernet controller: Intel Corporation 82576 Gigabit Network Connection (rev 01)
> > > > >         Subsystem: Intel Corporation Gigabit ET Dual Port Server Adapter
> > > > >         Flags: fast devsel, IRQ 16
> > > > >         Memory at fbc20000 (32-bit, non-prefetchable) [disabled] [size=128K]
> > > > >         Memory at fb800000 (32-bit, non-prefetchable) [disabled] [size=4M]
> > > > >         I/O ports at e020 [disabled] [size=32]
> > > > >         Memory at fbc44000 (32-bit, non-prefetchable) [disabled] [size=16K]
> > > > >         Expansion ROM at fb400000 [disabled] [size=4M]
> > > > 
> > > > BTW, I think this is the issue, the Expansion ROM. qemu-xen will
> > > > allocate memory for it. Will have maybe have to find another way.
> > > > qemu-trad those not seems to allocate memory, but I haven't been very
> > > > far in trying to check that.
> > > 
> > > And indeed that is the case. The "Fix" below fixes it.
> > > 
> > > 
> > > Based on that and this guest config:
> > > disk = [ 'file:/mnt/lab/latest/root_image.iso,hdc:cdrom,r']
> > > memory = 2048
> > > boot="d"
> > > maxvcpus=32
> > > vcpus=1
> > > serial='pty'
> > > vnclisten="0.0.0.0"
> > > name="latest"
> > > vif = [ 'mac=00:0F:4B:00:00:68, bridge=switch' ]
> > > pci = ["01:00.0"]
> > > 
> > > I can boot the guest.
> > 
> > And can you access the ROM from the guest ?
> 
> I hadn't tried it. This is with a NIC and I just wanted to see if it
> could do PCI passthrough without using the Option ROM.
> > 
> > 
> > Also, I have another patch, it will initialize the PCI ROM BAR like any
> > other BAR. In this case, if qemu is envolved in the access to ROM, it
> > will print an error, like it the case for other BAR. 
> > 
> > I tried to test it, but it was with an embedded VGA card. When I dump
> > the ROM, I got the same one as the emulated card instead of the ROM from
> > the device.
> 
> Oddly enough for me with your patch the NIC's BIOS was invoked and
> it tried to PXE boot:
> 
> (d1) [2014-01-10 03:20:29] Running option rom at ca00:0003
> 
> (d1) [2014-01-10 03:20:47] Booting from DVD/CD...
> (d1) [2014-01-10 03:20:47] Booting from 0000:7c00
> ..
> and I did see the PXE boot menu in my guest - so even
> better!

Perfect, look like it is the fix for PCI passthrough.

> I have not yet done the GPU - this issue was preventing me from using
> qemu-xen as it would always blow up before SeaBIOS was in the picture.
> 
> If you would like to put 'Reported-and-Tested-by: Konrad Rzeszutek Wilk
> <konrad.wilk@oracle.com>' please do.

Will do.

> Thank you!
> > 
> > 
> > diff --git a/hw/xen/xen_pt.c b/hw/xen/xen_pt.c
> > index 6dd7a68..2bbdb6d 100644
> > --- a/hw/xen/xen_pt.c
> > +++ b/hw/xen/xen_pt.c
> > @@ -440,8 +440,8 @@ static int xen_pt_register_regions(XenPCIPassthroughState *s)
> >  
> >          s->bases[PCI_ROM_SLOT].access.maddr = d->rom.base_addr;
> >  
> > -        memory_region_init_rom_device(&s->rom, OBJECT(s), NULL, NULL,
> > -                                      "xen-pci-pt-rom", d->rom.size);
> > +        memory_region_init_io(&s->rom, OBJECT(s), &ops, &s->dev,
> > +                              "xen-pci-pt-rom", d->rom.size);
> >          pci_register_bar(&s->dev, PCI_ROM_SLOT, PCI_BASE_ADDRESS_MEM_PREFETCH,
> >                           &s->rom);
> >  
> > 
> > -- 
> > Anthony PERARD

-- 
Anthony PERARD