Re: [Qemu-devel] Help: Does Qemu support virtio-pci for net-device and disk device?

[Marcel Apfelbaum <marcel@redhat.com>]

On 08/17/2016 08:00 PM, Laine Stump wrote:
> On 08/17/2016 12:13 PM, Andrew Jones wrote:
>> On Wed, Aug 17, 2016 at 08:08:11PM +0800, Kevin Zhao wrote:
>>> Hi all,
[...]

Hi,

>>
>> 1) If we want to support both PCIe devices and PCI, then things are messy.
>>     Currently we propose dropping PCI support. mach-virt pretty much
>>     exclusively uses virtio, which can be set to PCIe mode (virtio-1.0)
>
> I have a libvirt patch just about ACKed for pushing upstream that will automatically assign virtio-pci devices to a PCIe slot (if the qemu binary supports virtio-1.0):
>
> https://www.redhat.com/archives/libvir-list/2016-August/msg00852.html
>
> Separate patches do the same for the e1000e emulated network device (which you probably don't care about) and the nec-usb-xhci (USB3) controller (more useful):
>
> https://www.redhat.com/archives/libvir-list/2016-August/msg00732.html
>
> Once these are in place, the only type of device of any consequence that I can see still having no PCIe alternative is audio (even though only the virgl video device is PCIe, libvirt has always
> assigned the primary video to slot 1 on pcie-root anyway (although you shouldn't put a legacy PCI device on a pcie-root-port or pcie-switch-downstream-port, it is acceptable to plug it directly into
> pcie-root (as long as you know you won't need to hotplug it).
>

I agree, please don't allow plugging PCI devices into PCIe ports (root ports/downstream ports). Use pcie.0 bus as you mentioned or start a pci "hierarchy" with a dmi-pci and pci-pci bridge.

>> 2) root complex ports, switches (upstream/downstream ports) are currently
>>     based on Intel parts. Marcel is thinking about creating generic models.
>
> I say this every time it comes up, so just to be consistent: +1 :-)
>

I plan do that for my "own selfish" reasons, one of them is to be able to specify
the IO/MEM range the guest firmware should assign to a PCIe port. But, of course,
I'll keep them neutral enough; I might need help with testing the mach-virt machines.

>> 3) libvirt needs to learn how to plug everything together, in proper PCIe
>>     fashion, leaving holes for hotplug.
>
> See above about virtio, although that doesn't cover the whole story. The other part (which I'm working on right now) is that libvirt needs to automatically add pcie-root-port,
> pcie-switch-upstream-port, and pcie-switch-downstream-port devices as necessary. With the patches I mentioned above, you still have to manually add enough pcie-*-port controllers to the config, and
> then libvirt will plug the PCIe devices into the right place. This is simple enough to do, but it does require intervention.
>
> As far as leaving holes for hotplug, there's actually still a bit of an open question there - with machinetypes that use only legacy-PCI, *all* slots are hotpluggable, and they're added 31 at a time,
> so there was never any question about which slots were hotpluggable, and it would be very rare to end up with a configuration that had 0 free slots available for hotplug (actually libvirt would always
> make sure there was at least one, but in practice there would be many more open slots). With PCIe-capable machinetypes that is changed, since the root complex (pcie-root) doesn't support hotplug, and
> new slots are added 1 at a time (pcie-*-port) rather than 31 at a time. This means you have to really go out of your way if you want open slots for hotplug (and even if you want devices in the machine
> at boot time to be hot-unpluggable).
>

I agree you need to take into account hotplug for Q35, by leaving some PCIe root ports available, but it shouldn't be too much trouble.

> I'm still not sure just how far we need to go in this regard.  We've already decided that, unless manually set to an address on pcie-root by the user/management application, all PCI devices will be
> auto-assigned to a slot that supports hotplug.

Good idea.

  What I'm not sure about is whether we should always auto-add an extra pcie-*-root to be sure a device can be hotplugged, or if we should admit that 1
> available slot isn't good enough for all situations, so we should instead just leave it up to the user/management to manually add extra ports if they think they'll want to hotplug something later.

Why not? Leaving 1 or 2 PCIe ports should be enough. On each port you can hotplug a switch with several downstream ports. You can continue nesting the switches up to depth of 6-7 I think.

Thanks,
Marcel

>
[...]
>>
>> Thanks,
>> drew
>
>