Re: vhost-user technical isssues

[Linhaifeng <haifeng.lin-hv44wF8Li93QT0dZR+AlfA@public.gmane.org>]

On 2014/11/14 9:28, Xie, Huawei wrote:
> 
> 
>> -----Original Message-----
>> From: Linhaifeng [mailto:haifeng.lin-hv44wF8Li93QT0dZR+AlfA@public.gmane.org]
>> Sent: Wednesday, November 12, 2014 11:28 PM
>> To: Xie, Huawei; 'Tetsuya Mukawa'; dev-VfR2kkLFssw@public.gmane.org
>> Subject: Re: [dpdk-dev] vhost-user technical isssues
>>
>>
>>
>> On 2014/11/12 5:37, Xie, Huawei wrote:
>>> Hi Tetsuya:
>>> There are two major technical issues in my mind for vhost-user
>> implementation.
>>>
>>> 1) memory region map
>>> Vhost-user passes us file fd and offset for each memory region. Unfortunately
>> the mmap offset is "very" wrong. I discovered this issue long time ago, and also
>> found
>>> that I couldn't mmap the huge page file even with correct offset(need double
>> check).
>>> Just now I find that people reported this issue on Nov 3.
>>> [Qemu-devel] [PULL 27/29] vhost-user: fix mmap offset calculation
>>> Anyway, I turned to the same idea used in our DPDK vhost-cuse: only use the fd
>> for region(0) to map the  whole file.
>>> I think we should use this way temporarily to support qemu-2.1 as it has that
>> bug.
>>>
>>
>> the size of region 0 is not same as the file size. may be you should mmap the
>> other region.
> Haifeng:
> 
> Will calculate the maximum memory size, and use any file fd to mmap it.
> Here we assume the fds for different regions actually point to the same file.

actually there may be two hugepage files created by qemu.
one day i create a 4G VM found qemu create 2 hugepage file and send them to vhost-user.
you can try to test it.

> 
> In theory we should use the fd for each region to map each memory region.
> In fact we could map once. This will also save address space for 1GB huge page
> due to mmap alignment requirement.
>>
>> region 0:
>>         gpa = 0x0
>>         size = 655360
>>         ua = 0x2aaaaac00000
>>         offset = 0
>>
>> region 1:// use this region to mmap.BTW how to avoid mmap twice when there
>> are two devices?
>>         gpa = 0xC0000
>>         size = 2146697216
>>         ua = 0x2aaaaacc0000
>>         offset = 786432
> 
> What do you mean by two devices?
>>
>>

e.g there are two vhost-user backends in a VM, we will receive two SET_MEM_TABLE messages, actually we only need mmap once in one message.

I think qemu should add a new message to send all hugepage fd and size once.
as this we not need to mmap and calculate memory in set_mem_table message.

>>
>>> 2) what message is the indicator for vhost start/release?
>>> Previously  for vhost-cuse, it has SET_BACKEND message.
>>> What we should do for vhost-user?
>>> SET_VRING_KICK for start?
>>> What about for release?
>>> Unlike the kernel virtio, the DPDK virtio in guest could be restarted.
>>>
>>> Thoughts?
>>>
>>> -huawei
>>>
>>>
>>
>> --
>> Regards,
>> Haifeng
> 
> 
> .
> 

-- 
Regards,
Haifeng