Re: [RFC 0/1] virtio-net: add support for SR-IOV emulation

[Yui Washizu <yui.washidu@gmail.com>]

On 2023/07/24 15:58, Jason Wang wrote:
> On Mon, Jul 24, 2023 at 10:32 AM Yui Washizu <yui.washidu@gmail.com> wrote:
>>
>> On 2023/07/20 11:20, Jason Wang wrote:
>>> On Wed, Jul 19, 2023 at 9:59 AM Yui Washizu <yui.washidu@gmail.com> wrote:
>>>> This patch series is the first step towards enabling
>>>> hardware offloading of the L2 packet switching feature on virtio-net device to host machine.
>>>> We are considering that this hardware offloading enables
>>>> the use of high-performance networks in virtual infrastructures,
>>>> such as container infrastructures on VMs.
>>>>
>>>> To enable L2 packet switching by SR-IOV VFs, we are considering the following:
>>>> - making the guest recognize virtio-net devices as SR-IOV PF devices
>>>>     (archived with this patch series)
>>>> - allowing virtio-net devices to connect SR-IOV VFs to the backend networks,
>>>>     leaving the L2 packet switching feature to the management layer like libvirt
>>> Could you please show the qemu command line you want to propose here?
>>
>> I am considering how to specify the properties of VFs to connect SR-IOV
>> VFs to the backend networks.
>>
>>
>> For example:
>>
>>
>> qemu-system-x86_64 -device
>> pcie-root-port,port=8,chassis=8,id=pci.8,bus=pcie.0,multifunction=on
>>                      -netdev tap,id=hostnet0,vhost=on
>>                      -netdev tap,id=vfnet1,vhost=on # backend network for
>> SR-IOV VF 1
>>                      -netdev tap,id=vfnet2,vhost=on # backend network for
>> SR-IOV VF 2
>>                      -device
>> virtio-net-pci,netdev=hostnet0,sriov_max_vfs=2,sriov_netdev=vfnet1:vfnet2,...
>>
>>
>> In this example, we can specify multiple backend networks to the VFs
>> by adding "sriov_netdev" and separating them with ":".
> This seems what is in my mind as well, more below
>
>> Additionally, when passing properties like "rx_queue_size" to VFs, we
>> can utilize new properties,
>> such as "sriov_rx_queue_size_per_vfs," to ensure that the same value is
>> passed to all VFs.
> Or we can introduce new device like:
>
> -netdev tap,id=hn0 \
> -device virtio-net-pci,netdev=hn0,id=vnet_pf \
> -netdev tap,netdev=hn1 \
> -device virtio-net-pci-vf,netdev=hn1,id=vf0,pf=vnet_pf,rx_queue_size=XYZ ... \
>
> This allows us to reuse the codes for configuring vf parameters. But
> note that rx_queue_size doesn't make too much sense to vhost-vDPA, as
> qemu can perform nothing more than a simple sanity test.
>
> Thanks


Thanks for proposing this new way.

I have considered how to implement this.


As virtio-net-pci-vf device should show up

on the guest only when the guest OS creates a VF,

the guest must not be able to see the VF device on PCI bus when qemu starts.

However, it's hard to realize this without overcomplicating

relevant code due to current qemu implementation.

It's because qdev_device_add_from_qdict,

a function which is called when devices are specified

with "-device" option of qemu startup command,

always create devices by qdev_new and qdev_realize.

It might be possible that we fix it

so that qdev_new/qdev_realize aren't triggered for virtio-net-pci-vf 
devices,

but It seems that we need to special case the device in very generic code

like qdev_device_add_from_qdict(), qdev_device_add(),

device_init_func() or their caller function.


Given my current ideas,

it seems like this PATCH could become complex.

Woule you have any suggestions

for achieving this in more simple way possible ?



>> I'm still considering about how to specify it, so please give me any
>> comments if you have any.
>>
>>
>>>>     - This makes hardware offloading of L2 packet switching possible.
>>>>       For example, when using vDPA devices, it allows the guest
>>>>       to utilize SR-IOV NIC embedded switch of hosts.
>>> This would be interesting.
>>>
>>> Thanks
>>>
>>>> This patch series aims to enable SR-IOV emulation on virtio-net devices.
>>>> With this series, the guest can identify the virtio-net device as an SR-IOV PF device.
>>>> The newly added property 'sriov_max_vfs' allows us to enable the SR-IOV feature
>>>> on the virtio-net device.
>>>> Currently, we are unable to specify the properties of a VF created from the guest.
>>>> The properties are set to their default values.
>>>> In the future, we plan to allow users to set the properties.
>>>>
>>>> qemu-system-x86_64 --device virtio-net,sriov_max_vfs=<num>
>>>> # when 'sriov_max_vfs' is present, the SR-IOV feature will be automatically enabled
>>>> # <num> means the max number of VF on guest
>>>>
>>>> Example commands to create VFs in virtio-net device from the guest:
>>>>
>>>> guest% readlink -f /sys/class/net/eth1/device
>>>>    /sys/devices/pci0000:00/0000:00:02.0/0000:01:00.0/virtio1
>>>> guest% echo "2" > /sys/devices/pci0000:00/0000:00:02.0/0000:01:00.0/sriov_numvfs
>>>> guest% ip link show
>>>>    eth0: ....
>>>>    eth1: ....
>>>>    eth2: .... #virtual VF created
>>>>    eth3: .... #virtual VF created
>>>>
>>>> Please note that communication between VF and PF/VF is not possible by this patch series itself.
>>>>
>>>> Yui Washizu (1):
>>>>     virtio-pci: add SR-IOV capability
>>>>
>>>>    hw/pci/msix.c                  |  8 +++--
>>>>    hw/pci/pci.c                   |  4 +++
>>>>    hw/virtio/virtio-pci.c         | 62 ++++++++++++++++++++++++++++++----
>>>>    include/hw/virtio/virtio-pci.h |  1 +
>>>>    4 files changed, 66 insertions(+), 9 deletions(-)
>>>>
>>>> --
>>>> 2.39.3
>>>>