Re: [PATCH 2/3] net/virtio_user: fix wrong sequence of messages

[Yuanhan Liu <yuanhan.liu@linux.intel.com>]

On Fri, Sep 09, 2016 at 01:50:16PM +0800, Tan, Jianfeng wrote:
> On 9/9/2016 12:19 PM, Yuanhan Liu wrote:
> 
> >>
> >>
> >>         Never mind, above fix on the vhost side will not take effect on existing
> >>         vpp-vhost implementations.
> >>
> >>     Actually, I was talking about the DPDK vhost implementation :)
> >>
> >>
> >>This patch is talking about vpp's native vhost implementation, not dpdk-vhost,
> >>and not the way vpp uses dpdk-vhost.
> >Yes, I know. What I meant is there was a "workaround" in DPDK vhost
> >implementation, and since you bring this issue on the table again,
> >it's a chance to think about how can we fix it.
> >
> >A rough idea come to my mind is we could check all the per-vring message
> >at the begining of vhost_user_msg_handler() and allocate related vq when
> >necessary (when it's the first vring message we got).
> >
> >Yeah, I know it's a bit ugly, but it at least gets rid of that "not-that-true"
> >assumption.
> 
> Sounds workable. So we'd define those vq-specific msgs, like:
> VHOST_USER_SET_VRING_NUM,
> VHOST_USER_SET_VRING_ADDR,
> VHOST_USER_SET_VRING_BASE,
> VHOST_USER_GET_VRING_BASE(?),
> VHOST_USER_SET_VRING_KICK,
> VHOST_USER_SET_VRING_CALL,
> VHOST_USER_SET_VRING_ENABLE,

Yes.

> >>         Still not working. VPP needs SET_VRING_CALL to create vq firstly.
> >>
> >>     Didn't get it. In the proposal, SET_FEATURES is sent before every other
> >>     messages, thus it should not cause the issue you described in this patch.
> >>
> >>
> >>OK. Let me try to explain. We take three vhost implementations into
> >>consideration: dpdk-2.2-vhost, dpdk-master-vhost, vpp-native-vhost.
> >>
> >>If set_feature before set_vring_call, dpdk-2.2-vhost will fail: inside
> >>set_feature handler, assigning header length to VQs which will be created in
> >>set_vring_call handler.
> >Oh, right. That was an in-correct implementation.
> >
> >>So we need to keep set_vring_call firstly.
> >>Then set_feature needs to be sent
> >>before any other msgs, this is what vpp-native-vhost requires. In all, the
> >>sequence is like this:
> >>1. set_vring_call,
> >>2. set_feature,
> >>3. other msgs
> >>
> >>
> >>     Besides, haven't we already sent SET_VRING_CALL before other messages
> >>     (well, execept the SET_FEATURES and SET_MEM_TABLE message)?
> >>
> >>
> >>Yes, set_vring_call is already in the first place, but we need to plugin
> >>set_feature between set_vring_call and other msgs. Previously, set_vring_call
> >>and other msgs are together.
> >Okay. Another thing I noticed is that virtio-user lacks some feature
> >negotiations, like GET_FEATURES and GET_PROTOCOL_FEATURES. I think you
> >might need add them back somewhen?
> 
> GET_FEATURES has been done in virtio_user_dev_init().

Oh, sorry, I missed that.

> GET_PROTOCOL_FEATURES
> is not supported yet. I see those features in PROTOCOL_FEATURES is for live
> migration (right?).

Not exactly. PROTOCOL_FEATURES was firstly introduced while MQ was
enabled. Thus it's no wonder MQ is the first protocol feature vhost
user supports:

    [yliu@yliu-dev ~/dpdk]$ gg PROTOCOL_F_ lib/librte_vhost/
    lib/librte_vhost/vhost_user.h:46:#define VHOST_USER_PROTOCOL_F_MQ 0
    lib/librte_vhost/vhost_user.h:47:#define VHOST_USER_PROTOCOL_F_LOG_SHMFD 1
    lib/librte_vhost/vhost_user.h:48:#define VHOST_USER_PROTOCOL_F_RARP 2

	--yliu

> Assuming that, anyone using container/process now
> enables live migration so far? I don't think so.
> 
> Thanks,
> Jianfeng
> 
> 
> >
> >	--yliu