Re: [Qemu-devel] [RFC 0/2] virtio-vhost-user: add virtio-vhost-user device

[Stefan Hajnoczi <stefanha@redhat.com>]

[-- Attachment #1: Type: text/plain, Size: 3608 bytes --]

On Mon, Feb 05, 2018 at 09:57:23AM +0000, Wang, Wei W wrote:
> On Friday, February 2, 2018 11:26 PM, Stefan Hajnoczi wrote:
> > On Tue, Jan 30, 2018 at 08:09:19PM +0800, Wei Wang wrote:
> > > Background:
> > > The vhost-user negotiation is split into 2 phases currently. The 1st
> > > phase happens when the connection is established, and we can find
> > > what's done in the 1st phase in vhost_user_init(). The 2nd phase
> > > happens when the master driver is loaded (e.g. run of virtio-net pmd)
> > > and set status to the device, and we can find what's done in the 2nd
> > > phase in vhost_dev_start(), which includes sending the memory info and
> > > virtqueue info. The socket is connected, till one of the QEMU devices
> > > exits, so pmd exiting won't end the QEMU side socket connection.
> > >
> > > Issues:
> > > Suppose we have both the vhost and virtio-net set up, and vhost pmd
> > > <-> virtio-net pmd communication works well. Now, vhost pmd exits
> > > (virtio-net pmd is still there). Some time later, we re-run vhost pmd,
> > > the vhost pmd doesn't know the virtqueue addresses of the virtio-net
> > > pmd, unless the virtio-net pmd reloads to start the 2nd phase of the
> > > vhost-user protocol. So the second run of the vhost pmd won't work.
> > 
> > This isn't a problem for virtio-vhost-user since the vhost-pmd resets the
> > virtio-vhost-user device when it restarts.  The vhost-user AF_UNIX socket
> > reconnects and negotiation restarts.
> 
> I'm not sure if you've agreed that vhost-user negotiation is split into two phases as described above. If not, it's also not difficult to check, thanks to the RTE_LOG put at the vhost_user_msg_handler:
> RTE_LOG(INFO, VHOST_CONFIG, "read message %s\n",
>                         vhost_message_str[msg.request.master]);
> It tells us what messages are received and when they are received. 
> 
> Before trying the virtio-vhost-user setup, please make sure the virtio-net side VM doesn't have the virtio-net kernel driver loaded (blacklist the module or disable it in .config).
> VM1: the VM with virtio-vhost-user
> VM2: the VM with virtio-net
> 
> 1) After we boot VM1 and the virtio-vhost-user pmd, we boot VM2, and we will see the following log in VM1:
> VHOST_CONFIG: new device, handle is 0
> VHOST_CONFIG: read message VHOST_USER_GET_FEATURES
> VHOST_CONFIG: read message VHOST_USER_GET_PROTOCOL_FEATURES
> VHOST_CONFIG: read message VHOST_USER_SET_PROTOCOL_FEATURES
> VHOST_CONFIG: read message VHOST_USER_GET_QUEUE_NUM
> VHOST_CONFIG: read message VHOST_USER_SET_OWNER
> VHOST_CONFIG: read message VHOST_USER_GET_FEATURES
> VHOST_CONFIG: read message VHOST_USER_SET_VRING_CALL
> VHOST_CONFIG: vring call idx:0 file:-1
> VHOST_CONFIG: read message VHOST_USER_SET_VRING_CALL
> VHOST_CONFIG: vring call idx:1 file:-1
> 
> Those messages are what I called phase1 negotiation. They are negotiated when the AF_UNIX socket connects.
> 
> 2) Then in VM2 we load the virtio-net pmd, we will see phase2 messages showing up in VM1, like:
> VHOST_CONFIG: read message VHOST_USER_SET_VRING_NUM
> VHOST_CONFIG: read message VHOST_USER_SET_VRING_BASE
> VHOST_CONFIG: read message VHOST_USER_SET_VRING_ADDR
> ...
> Those messages are sent to virtio-vhost-user only when the VM2's virtio-net pmd loads.
> 
> AF_UNIX socket reconnection only triggers phase1 negotiation. If virtio-net pmd in VM2 doesn't reload, virtio-vhost-user pmd won't get the above phase2 messages. Do you agree with the issue?

I'll reply on the subthread with Michael where you posted a similar
response.

Stefan

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 455 bytes --]