Re: [PATCH 00/21] Control VQ support in vDPA

From: Jason Wang <jasowang@redhat.com>
To: "Michael S. Tsirkin" <mst@redhat.com>
Cc: kvm@vger.kernel.org, lulu@redhat.com, netdev@vger.kernel.org,
	linux-kernel@vger.kernel.org,
	virtualization@lists.linux-foundation.org, eperezma@redhat.com,
	stefanha@redhat.com, eli@mellanox.com, lingshan.zhu@intel.com,
	rob.miller@broadcom.com
Subject: Re: [PATCH 00/21] Control VQ support in vDPA
Date: Fri, 18 Dec 2020 10:56:34 +0800	[thread overview]
Message-ID: <c120d0ce-7d42-ab6e-1040-ab85985f7cbe@redhat.com> (raw)
In-Reply-To: <20201217163513-mutt-send-email-mst@kernel.org>

On 2020/12/18 上午6:28, Michael S. Tsirkin wrote:
> On Thu, Dec 17, 2020 at 05:02:49PM +0800, Jason Wang wrote:
>> On 2020/12/17 下午3:58, Michael S. Tsirkin wrote:
>>> On Thu, Dec 17, 2020 at 11:30:18AM +0800, Jason Wang wrote:
>>>> On 2020/12/16 下午5:47, Michael S. Tsirkin wrote:
>>>>> On Wed, Dec 16, 2020 at 02:47:57PM +0800, Jason Wang wrote:
>>>>>> Hi All:
>>>>>>
>>>>>> This series tries to add the support for control virtqueue in vDPA.
>>>>>>
>>>>>> Control virtqueue is used by networking device for accepting various
>>>>>> commands from the driver. It's a must to support multiqueue and other
>>>>>> configurations.
>>>>>>
>>>>>> When used by vhost-vDPA bus driver for VM, the control virtqueue
>>>>>> should be shadowed via userspace VMM (Qemu) instead of being assigned
>>>>>> directly to Guest. This is because Qemu needs to know the device state
>>>>>> in order to start and stop device correctly (e.g for Live Migration).
>>>>>>
>>>>>> This requies to isolate the memory mapping for control virtqueue
>>>>>> presented by vhost-vDPA to prevent guest from accesing it directly.
>>>>>> To achieve this, vDPA introduce two new abstractions:
>>>>>>
>>>>>> - address space: identified through address space id (ASID) and a set
>>>>>>                     of memory mapping in maintained
>>>>>> - virtqueue group: the minimal set of virtqueues that must share an
>>>>>>                     address space
>>>>> How will this support the pretty common case where control vq
>>>>> is programmed by the kernel through the PF, and others by the VFs?
>>>> In this case, the VF parent need to provide a software control vq and decode
>>>> the command then send them to VF.
>>> But how does that tie to the address space infrastructure?
>> In this case, address space is not a must.
> That's ok, problem is I don't see how address space is going
> to work in this case at all.
>
> There's no address space there that userspace/guest can control.
>

The virtqueue group is mandated by parent but the association between 
virtqueue group and address space is under the control of userspace (Qemu).

A simple but common case is that:

1) Device advertise two virtqueue groups: group 0 contains RX and TX, 
group 1 contains CVQ.
2) Device advertise two address spaces

Then, for vhost-vDPA using by VM:

1) associate group 0 with as 0, group 1 with as 1 (via vhost-vDPA 
VHOST_VDPA_SET_GROUP_ASID)
2) Publish guest memory mapping via IOTLB asid 0
3) Publish control virtqueue mapping via IOTLB asid 1

Then the DMA is totally isolated in this case.

For vhost-vDPA using by DPDK or virtio-vDPA

1) associate group 0 and group 1 with as 0

since we don't need DMA isolation in this case.

In order to let it be controlled by Guest, we need extend virtio spec to 
support those concepts.

Thanks

_______________________________________________
Virtualization mailing list
Virtualization@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/virtualization