Re: [Qemu-devel] [Question] Why doesn't PCIe hotplug work for Q35 machine?

["Michael S. Tsirkin" <mst@redhat.com>]

On Tue, Aug 19, 2014 at 06:25:56AM +0000, Gonglei (Arei) wrote:
> > >> Subject: Re: [Question] Why doesn't PCIe hotplug work for Q35 machine?
> > >>
> > >> On Sun, 2014-08-17 at 13:00 +0200, Michael S. Tsirkin wrote:
> > >>> On Fri, Aug 15, 2014 at 07:33:29AM +0000, Gonglei (Arei) wrote:
> > >>>> Hi,
> > >>>>
> > >>>> I noticed that the qemu-2.1 release change log says
> > >>>> " PCIe: Basic hot-plug/hot-unplug support for Q35 machine."
> > >>>> And then I made a testing for the hotplugging function of Q35.
> > >>>> But I'm failed, and I got the dmesg log in guest os as below:
> > >>>>
> > >>>> [ 159.035250] Pciehp 0000:05:00.0:pcie24: Button pressed on Slot (0 - 4)
> > >>>> [ 159.035274] Pciehp 0000:05:00.0:pcie24: Card present on Slot (0 - 4)
> > >>>> [ 159.036517] Pciehp 0000:05:00.0:pcie24: PCI slot #0 - 4 - powering on
> > due
> > >> to button press.
> > >>>> [ 159.188049] Pciehp 0000:05:00.0:pcie24: Failed to check link status
> > >>>> [ 159.201968] Pciehp 0000:05:00.0:pcie24: Card not present on Slot (0 - 4)
> > >>>> [ 159.202529] Pciehp 0000:05:00.0:pcie24: Already disabled on Slot (0 - 4)
> > >>>>
> > >>>> Steps of testing:
> > >>>>
> > >>>> #1. QEMU version:
> > >>>>
> > >>>> 	The lateset master tree source.
> > >>>>
> > >>>> #2. Command line:
> > >>>>
> > >>>> ./qemu-system-x86_64 -enable-kvm -m 2048 -machine q35 -device
> > >> ide-drive,bus=ide.2,drive=MacHDD \
> > >>>>  -drive
> > id=MacHDD,if=none,file=/mnt/sdb/gonglei/image/redhat_q35.img
> > >> -monitor stdio -vnc :10 -readconfig ../docs/q35-chipset.cfg
> > >>>> QEMU 2.0.93 monitor - type 'help' for more information
> > >>>> (qemu) device_add
> > >> virtio-net-pci,id=nic2,bus=pcie-switch-downstream-port-1-1,addr=1.0
> > >>>
> > >>> I don't think you can use any slot except slot 0 for pci express.
> > >
> > > OK. Does the PCIe specification say that?
> > > I appreciate very much that you explain more.
> > 
> > The closest I could find is in "7.3. Configuration Transaction
> > Rules"/"7.3.1. Device Number":
> > 
> > With non-ARI Devices, PCI Express components are restricted to
> > implementing a single Device Number on their primary interface (Upstream
> > Port) [...] Downstream Ports that do not have ARI Forwarding enabled
> > must associate only Device 0 with the device attached to the Logical Bus
> > representing the Link from the Port. Configuration Requests
> > targeting the Bus Number associated with a Link specifying Device Number
> > 0 are delivered to the device attached to the Link; Configuration
> > Requests specifying all other Device Numbers (1-31)
> > must be terminated by the Switch Downstream Port or the Root Port with
> > an Unsupported Request Completion Status (equivalent to Master Abort in
> > PCI).
> > 
> Thanks a lot, Paolo.
> 
> And I found another issue when cold-plugging don't using slot 0, the PCIe device also can't
> be searched in guest os. 
> 
> So, I have some questions and ideas:
> 
> 1. Does qemu support ARI Forwarding for PCIe at present? If yes, how to enable it ?

What do you mean by forwarding?
What would you like to do?
We do have code to emulate ARI, I don't think many people
tested it so it's likely incomplete.

> 2. If not, we should add some check for PCIe root ports and downstream ports,
>  meanwhile add explaining document.

You want an attempt to add a device at slot !=0 to report an error?
We can do this if device at slot 0 does not have ARI support.
Seems like a low priority issue I think.

> 3. Those check should add in general code level, both hotplug and coldplug.

Generally it's not clear how we want to support hotplug for
multifunction devices. One way it to add functions != 0 then
add function 0 and notify guest.
If so, then of course we can't check things ...

> Am I right?  Thanks.
> 
> Best regards,
> -Gonglei