From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from eggs.gnu.org ([2001:4830:134:3::10]:44713) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1ZyGOd-0000kU-F4 for qemu-devel@nongnu.org; Mon, 16 Nov 2015 04:48:24 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1ZyGOa-0000n4-7e for qemu-devel@nongnu.org; Mon, 16 Nov 2015 04:48:23 -0500 Received: from mx1.redhat.com ([209.132.183.28]:50409) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1ZyGOZ-0000my-W8 for qemu-devel@nongnu.org; Mon, 16 Nov 2015 04:48:20 -0500 Date: Mon, 16 Nov 2015 11:48:16 +0200 From: "Michael S. Tsirkin" Message-ID: <20151116114721-mutt-send-email-mst@redhat.com> References: <1447615677-9302-1-git-send-email-mst@redhat.com> <20151116113126-mutt-send-email-victork@redhat.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20151116113126-mutt-send-email-victork@redhat.com> Subject: Re: [Qemu-devel] [PATCH] vhost-user: update spec description List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , To: Victor Kaplansky Cc: Jason Wang , =?us-ascii?B?PT9VVEYtOD9xP01hcmMtQW5kcj1DMz1BOT0yMEx1cmVhdT89?= , Yuanhan Liu , qemu-devel@nongnu.org, Changchun Ouyang On Mon, Nov 16, 2015 at 11:36:47AM +0200, Victor Kaplansky wrote: > On Sun, Nov 15, 2015 at 09:29:59PM +0200, Michael S. Tsirkin wrote: > > Clarify logging setup to make sure all clients comply in a way that is > > future-proof. Document how rings are started/stopped. > > > > Signed-off-by: Michael S. Tsirkin > > --- > > docs/specs/vhost-user.txt | 62 ++++++++++++++++++++++++++++++++++++++++------- > > 1 file changed, 53 insertions(+), 9 deletions(-) > > > > diff --git a/docs/specs/vhost-user.txt b/docs/specs/vhost-user.txt > > index 26dde2e..6a0ed8f 100644 > > --- a/docs/specs/vhost-user.txt > > +++ b/docs/specs/vhost-user.txt > > @@ -87,6 +87,14 @@ Depending on the request type, payload can be: > > User address: a 64-bit user address > > mmap offset: 64-bit offset where region starts in the mapped memory > > > > +* Log description > > + --------------------------- > > + | log size | log offset | > > + --------------------------- > > + log size: size of area used for logging > > + log offset: offset from start of supplied file descriptor > > + where logging starts (i.e. where guest address 0 would be logged) > > + > > In QEMU the vhost-user message is implemented with the following struct: > > > > typedef struct VhostUserMsg { > > @@ -138,6 +146,23 @@ As older slaves don't support negotiating protocol features, > > a feature bit was dedicated for this purpose: > > #define VHOST_USER_F_PROTOCOL_FEATURES 30 > > > > +Starting and stopping rings > > +---------------------- > > +Each ring is initialized in a stopped state, client must not process it until > > +ring is enabled. > > + > > +If VHOST_USER_F_PROTOCOL_FEATURES has been negotiated, client must start and > > +stop ring processing upon receiving VHOST_USER_SET_VRING_ENABLE with parameters > > +1 and 0 respoectively. > > + > > +If VHOST_USER_F_PROTOCOL_FEATURES has not been negotiated, client must start > > +ring processing upon receiving a kick (that is, detecting that file descriptor > > +is readable) on the descriptor specified by VHOST_USER_SET_VRING_KICK, and stop > > +ring processing upon receiving VHOST_USER_GET_VRING_BASE. > > + > > +While rings are running, client must support changing some configuration > > +aspects on the fly. > > + > > Multiple queue support > > ---------------------- > > > > @@ -162,9 +187,13 @@ the slave makes to the memory mapped regions. The client should mark > > the dirty pages in a log. Once it complies to this logging, it may > > declare the VHOST_F_LOG_ALL vhost feature. > > > > +To start/stop logging of data/used ring writes, server may send messages > > +VHOST_USER_SET_FEATURES with VHOST_F_LOG_ALL and VHOST_USER_SET_VRING_ADDR with > > +VHOST_VRING_F_LOG in ring's flags set to 1/0, respectively. > > + > > All the modifications to memory pointed by vring "descriptor" should > > be marked. Modifications to "used" vring should be marked if > > -VHOST_VRING_F_LOG is part of ring's features. > > +VHOST_VRING_F_LOG is part of ring's flags. > > > > Dirty pages are of size: > > #define VHOST_LOG_PAGE 0x1000 > > @@ -173,22 +202,35 @@ The log memory fd is provided in the ancillary data of > > VHOST_USER_SET_LOG_BASE message when the slave has > > VHOST_USER_PROTOCOL_F_LOG_SHMFD protocol feature. > > > > -The size of the log may be computed by using all the known guest > > -addresses. The log covers from address 0 to the maximum of guest > > +The size of the log is supplied as part of VhostUserMsg > > +which should be large enough to cover all known guest > > +addresses. Log starts at the supplied offset in the > > +supplied file descriptor. > > +The log covers from address 0 to the maximum of guest > > regions. In pseudo-code, to mark page at "addr" as dirty: > > > > page = addr / VHOST_LOG_PAGE > > log[page / 8] |= 1 << page % 8 > > > > +Where addr is the guest physical address. > > + > > Use atomic operations, as the log may be concurrently manipulated. > > > > +Note that when logging modifications to the used ring (when VHOST_VRING_F_LOG > > +is set for this ring), log_guest_addr should be used to calculate the log > > +offset: the write to first byte of the used ring is logged at this offset from > > +log start. Also note that this value might be outside the legal guest physical > > +address range (i.e. does not have to be covered by the VhostUserMemory table), > > +but the bit offset of the last byte of the ring must fall within > > +the size supplied by VhostUserLog. > > + > > VHOST_USER_SET_LOG_FD is an optional message with an eventfd in > > ancillary data, it may be used to inform the master that the log has > > been modified. > > > > -Once the source has finished migration, VHOST_USER_RESET_OWNER message > > -will be sent by the source. No further update must be done before the > > -destination takes over with new regions & rings. > > +Once the source has finished migration, rings will be stopped by > > +the source. No further update must be done before rings are > > +restarted. > > > > Protocol features > > ----------------- > > @@ -259,11 +301,11 @@ Message types > > * VHOST_USER_RESET_OWNER > > > > Id: 4 > > - Equivalent ioctl: VHOST_RESET_OWNER > > Master payload: N/A > > > > - Issued when a new connection is about to be closed. The Master will no > > - longer own this connection (and will usually close it). > > + This is no longer used. Used to be sent to request stopping > > + all rings, but some clients interpreted it to also discard > > + connection state (this interpretation would lead to bugs). > > The new code uses VHOST_USER_RESET_DEVICE as the name for request > Id 4. This was reverted before rc0. > Also it is not clear whether it is recommended just to > ignore the reset device request now? Since it's never sent, does it matter? > > > > * VHOST_USER_SET_MEM_TABLE > > > > @@ -388,6 +430,8 @@ Message types > > Master payload: vring state description > > > > Signal slave to enable or disable corresponding vring. > > + This request should be sent only when VHOST_USER_F_PROTOCOL_FEATURES > > + has been negotiated. > > > > * VHOST_USER_SEND_RARP > > > > -- > > MST > --Victor