Re: [Qemu-devel] [RFC] vhost-user: introduce F_NEED_ALL_IOTLB protocol feature

[Jason Wang <jasowang@redhat.com>]

On 2018年04月12日 16:10, Tiwei Bie wrote:
> On Thu, Apr 12, 2018 at 03:38:50PM +0800, Jason Wang wrote:
>> On 2018年04月12日 09:44, Tiwei Bie wrote:
>>> On Wed, Apr 11, 2018 at 08:37:17PM +0300, Michael S. Tsirkin wrote:
>>>> On Wed, Apr 11, 2018 at 04:38:53PM +0800, Tiwei Bie wrote:
>>>>> On Wed, Apr 11, 2018 at 04:01:19PM +0800, Jason Wang wrote:
>>>>>> On 2018年04月11日 15:20, Tiwei Bie wrote:
>>>>>>> This patch introduces VHOST_USER_PROTOCOL_F_NEED_ALL_IOTLB
>>>>>>> feature for vhost-user. By default, vhost-user backend needs
>>>>>>> to query the IOTLBs from QEMU after meeting unknown IOVAs.
>>>>>>> With this protocol feature negotiated, QEMU will provide all
>>>>>>> the IOTLBs to vhost-user backend without waiting for the
>>>>>>> queries from backend. This is helpful when using a hardware
>>>>>>> accelerator which is not able to handle unknown IOVAs at the
>>>>>>> vhost-user backend.
>>>>>>>
>>>>>>> Signed-off-by: Tiwei Bie<tiwei.bie@intel.com>
>>>>>>> ---
>>>>>>> The idea of this patch is to let QEMU push all the IOTLBs
>>>>>>> to vhost-user backend without waiting for the queries from
>>>>>>> the backend. Because hardware accelerator at the vhost-user
>>>>>>> backend may not be able to handle unknown IOVAs.
>>>>>>>
>>>>>>> This is just a RFC for now. It seems that, it doesn't work
>>>>>>> as expected when guest is using kernel driver (To handle
>>>>>>> this case, it seems that some RAM regions' events also need
>>>>>>> to be listened). Any comments would be appreciated! Thanks!
>>>>>> Interesting, a quick question is why this is needed? Can we just use exist
>>>>>> IOTLB update message?
>>>>> Yeah, we are still using the existing IOTLB update messages
>>>>> to send the IOTLB messages to backend. The only difference
>>>>> is that, QEMU won't wait for the queries before sending the
>>>>> IOTLB update messages.
>>>> So I have a concern with that, in that without any flow
>>>> control the socket buffer used by vhost-user might become
>>>> full.
>>> Each IOTLB update message needs a reply. So I think it
>>> won't happen.
>> Is this what we've already done now? I don't find any statement on this in
>> vhost-user.txt?
>>
>> """
>>   * VHOST_USER_IOTLB_MSG
>>
>>        Id: 22
>>        Equivalent ioctl: N/A (equivalent to VHOST_IOTLB_MSG message type)
>>        Master payload: struct vhost_iotlb_msg
>>        Slave payload: u64
>>
>>        Send IOTLB messages with struct vhost_iotlb_msg as payload.
>>        Master sends such requests to update and invalidate entries in the
>> device
>>        IOTLB. The slave has to acknowledge the request with sending zero as
>> u64
>>        payload for success, non-zero otherwise.
>>        This request should be send only when VIRTIO_F_IOMMU_PLATFORM feature
>>        has been successfully negotiated.
>> """
> Yeah, it's what we've already done now. It's in the above
> statement you quoted:
>
> "The slave has to acknowledge the request with sending zero as
> u64 payload for success, non-zero otherwise."

My bad.

Actually, there's a minor optimization here. When QI is enabled, we only 
need replay ack for wait descriptor, this allows some kinds of batching. 
Another interesting idea is to send multiqueue request through a single 
message.

Thanks

>
>> And you probably need to modify the following statement:
>>
>> """
>> The master isn't expected to take the initiative to send IOTLB update
>> messages,
>> as the slave sends IOTLB miss messages for the guest virtual memory areas it
>> needs to access.
>> """
> Yeah, you're right. Thanks! This statement needs some minor updates.
>
> Best regards,
> Tiwei Bie
>
>> Thanks
>>
>>
>>> Best regards,
>>> Tiwei Bie
>>>