Re: [PATCH v1 0/2] Virtio-net PMD Extension to work on host

[Tetsuya Mukawa <mukawa@igel.co.jp>]

On 2016/01/06 14:42, Tan, Jianfeng wrote:
>
>
> On 1/6/2016 11:57 AM, Tetsuya Mukawa wrote:
>> On 2015/12/28 20:06, Tetsuya Mukawa wrote:
>>> On 2015/12/24 23:05, Tan, Jianfeng wrote:
>>>> Hi Tetsuya,
>>>>
>>>> After several days' studying your patch, I have some questions as
>>>> follows:
>>>>
>>>> 1. Is physically-contig memory really necessary?
>>>> This is a too strong requirement IMHO. IVSHMEM doesn't require this
>>>> in its original meaning. So how do you think of
>>>> Huawei Xie's idea of using virtual address for address translation?
>>>> (In addition, virtual address of mem_table could be
>>>> different in application and QTest, but this can be addressed
>>>> because SET_MEM_TABLE msg will be intercepted by
>>>> QTest)
>>> Hi Jianfeng,
>>>
>>> Thanks for your suggestion.
>>> Huawei's idea may solve contig-mem restriction.
>>> Let me have time to check it more.
>> Hi Jianfeng,
>>
>> I made sure we can remove the restriction with Huawei's idea.
>> One thing I concern is below.
>> If we don't use contiguous memory, this PMD will not work with other
>> 'physical' PMDs like e1000 PMD, virtio-net PMD, and etc.
>> (This is because allocated memory may not  be physically contiguous.)
>>
>> One of examples is that if we implement like above, in QEMU guest, we
>> can handle a host NIC directly, but in container, we will not be able to
>> handle the device.
>> This will be a restriction for this virtual addressing changing.
>>
>> Do you know an use case that the user wants to handle 'physical' PMD and
>> 'virtual' virtio-net PMD together?
>>
>> Tetsuya,
> Hi Tetsuya,
>
> I have no use case in hand, which handles 'physical' PMDs and
> 'virtual' virtio-net PMD together.
> (Pavel Fedin once tried to run ovs in container, but that case just
> uses virtual virtio devices, I
> don't know if he has plan to add 'physical' PMDs as well.)
>
> Actually, it's not completely contradictory to make them work
> together. Like this:
> a. containers with root privilege
> We can initialize memory as legacy way. (TODO: besides
> physical-contiguous, we try allocate
> virtual-contiguous big area for all memsegs as well.)

Hi Jianfeng,

Yes, I agree with you.
If the feature is really needed, we will be able to have work around.

>
> a.1 For vhost-net, before sending memory regions into kernel, we can
> merge those virtual-contiguous regions into one region.
> a.2 For vhost-user, we can merge memory regions in the vhost. The
> blocker is that for now, maximum fd num was restricted
> by VHOST_MEMORY_MAX_NREGIONS=8 (so in 2M-hugepage's case, 16M shared
> memory is not nearly enough).
>

With current your implementation, when 'virtual' virtio-net PMD is used,
'phys_addr' will be virtual address in EAL layer.

struct rte_memseg {
        phys_addr_t phys_addr;      /**< Start physical address. */
        union {
                void *addr;         /**< Start virtual address. */
                uint64_t addr_64;   /**< Makes sure addr is always 64
bits */
        };
        .......
};

How about choosing it in virtio-net PMD?
(In the case of 'virtual', just use 'addr' instead of using 'phys_addr'.)
For example, port0 may use physical address, but port1 may use virtual
address.

With this, of course, we don't have an issue with 'physical' virtio-net PMD.
Also, with 'virtual' virtio-net PMD, we can use virtual address and fd
that represents the big virtual address space.
(TODO: Need to change rte_memseg and EAL to keep fd and offset?)
Then, you don't worry about VHOST_MEMORY_MAX_NREGIONS, because we have
only one fd.

> b. containers without root privilege
> No need to worry about this problem, because it lacks of privilege to
> construct physical-contiguous memory.
>

Yes, we cannot run 'physical' PMDs in this type of container.
Anyway, I will check it more, if we really need it.

Thanks,
Tetsuya