Re: PCI hotplug problem [was: PV driver domains and S3 sleep]

[Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>]

On Fri, Sep 24, 2010 at 04:24:58PM +0200, Rafal Wojtczuk wrote:
> On Thu, Sep 16, 2010 at 12:52:02PM +0100, Keir Fraser wrote:
> > > The topic is self-explanatory: how to ensure that a PV driver domain correctly
> > > prepares its PCI devices for S3 sleep?
> [cut]
> > > Currently, how are these issues taken care of in the mainstream Xen?
> 
> > I don't think it currently is handled. HVM driver domains (using VT-d or
> > equivalent) can be put into virtual S3. We would need an equivalent concept
> > for PV driver domains. Or for devices to be hot-unplugged from the driver
> > domain, and re-plugged on resume?
> 
> The idea of using PCI hotplug is nice, however, PCI hotplug does not seem to
> work with the used setup (xen-3.4.3, all 64bit). Hot-unplug works, however the 
> following hotplug makes the driver domain kernel spit out the following:
> 
> Sep 24 09:46:01 localhost kernel: [  113.045927] pcifront pci-0: Rescanning
> PCI Frontend Bus 0000:00
> Sep 24 09:46:15 localhost kernel: [  126.843990] pcifront pci-0: Rescanning
> PCI Frontend Bus 0000:00
> Sep 24 09:46:15 localhost kernel: [  126.846217] pcifront pci-0: New device
> on 0000:00:01.00 found.
> Sep 24 09:46:15 localhost kernel: [  126.846523] iwlagn 0000:00:01.0: device
> not available (can't reserve [mem 0xf8000000-0xf8001fff 64bit])
> 
> ^C
> [root@localhost ~]# cat /proc/iomem 
> f6000000-f600ffff : 0000:00:00.0
>   f6000000-f600ffff : tg3
> [root@localhost ~]# lspci
> 00:00.0 Ethernet controller: Broadcom Corporation NetLink BCM5787M Gigabit
> Ethernet PCI Express (rev 02)
> 00:01.0 Network controller: Intel Corporation PRO/Wireless 4965 AG or AGN
> [Kedron] Network Connection (rev 61)
> 
> Nothing suspicious in xend, Xen and dom0 logs.
> 
> The domU and dom0 kernels are the same, 2.6.34.1-10.xenlinux (SUSE patches
> for 2.6.34.1).
> 
> With old pvops (2.6.31.9-1.pvops0) in domU, the message on the hot-plug is similar:
> Sep 24 09:50:40 localhost kernel: pcifront pci-0: Rescanning PCI Frontend
> Bus 0000:00
> Sep 24 09:50:51 localhost kernel: pcifront pci-0: Rescanning PCI Frontend
> Bus 0000:00
> Sep 24 09:50:51 localhost kernel: pcifront pci-0: New device on
> 0000:00:01.00 found.
> Sep 24 09:50:51 localhost kernel: iwlagn 0000:00:01.0: device not available
> because of BAR 0 [0xf8000000-0xf8001fff] collisions
> 
> Others seem to experience similar problems (e.g.
> http://permalink.gmane.org/gmane.comp.emulators.xen.devel/80766). Does
> anyone know the solution ?

I had an off-mailing list conversation with that fellow and I spun out
a bunch of patches to fix his issue.

You need these patches:
Konrad Rzeszutek Wilk (3):
      xen-pcifront: Enforce scanning of device functions on initial execution.
      xen-pcifront: Claim PCI resources before going live.
      xen-pcifront: Don't race with udev when discovering new devices.

I think they are in Jeremy's upstream tree.. ah, right you guys aren't using
Jeremy's tree.

Get them from: git://git.kernel.org/pub/scm/linux/kernel/git/konrad/xen.git

pv/pcifront-2.6.34

you also might want to update your pciback driver too (pv/pciback-2.6.32)
> 
> Regards,
> Rafal Wojtczuk
> 
> _______________________________________________
> Xen-devel mailing list
> Xen-devel@lists.xensource.com
> http://lists.xensource.com/xen-devel