Re: [PATCH] vhost-user.rst: Explicitly allow front-end to write to kick FDs

[Stefan Hajnoczi <stefanha@redhat.com>]

[-- Attachment #1: Type: text/plain, Size: 5682 bytes --]

On Tue, Apr 28, 2026 at 10:19:30AM -0700, Jorge Moreira wrote:
> On Tue, Apr 28, 2026 at 7:33 AM Stefano Garzarella <sgarzare@redhat.com> wrote:
> >
> > On Mon, Apr 27, 2026 at 03:48:44PM -0700, Jorge Moreira wrote:
> > >On Mon, Apr 27, 2026 at 3:45 PM Stefan Hajnoczi <stefanha@redhat.com> wrote:
> > >>
> > >> On Wed, Apr 22, 2026 at 12:20:52PM -0700, Jorge Moreira wrote:
> > >> > On Wed, Apr 22, 2026 at 1:32 AM Stefano Garzarella <sgarzare@redhat.com> wrote:
> > >> > >
> > >> > > On Wed, 22 Apr 2026 at 03:16, Jorge Moreira <jemoreira@google.com> wrote:
> > >> > > >
> > >> > > > On Tue, Apr 21, 2026 at 2:12 PM Stefan Hajnoczi <stefanha@redhat.com> wrote:
> > >> > > > >
> > >> > > > > On Mon, Apr 20, 2026 at 05:48:13PM -0700, Jorge Moreira wrote:
> > >> > > > > > While starting the vrings on SET_VRING_KICK could solve the state
> > >> > > > > > machine issue, it still won't notify the back-end that buffers are
> > >> > > > > > ready (the driver won't do this). Non-polling back-ends depend on this
> > >> > > > > > kick, especially for queues where data flows only from the driver to
> > >> > > > > > the back-end. Most implementations likely attempt to read from the
> > >> > > > > > queue only after receiving the kick.
> > >> > > > >
> > >> > > > > This is an interesting question to clarify in the spec.
> > >> > >
> > >> > > Yep, which is in part related to what I wrote in the other reply:
> > >> > > "I think the main issue to clarify is what the device should do
> > >> > > when the vrings are configured, but the driver has already been
> > >> > > initialized (which is usually the case after migration)."
> > >> > >
> > >> > > > >
> > >> > > > > Stefan
> > >> > > >
> > >> > > > This is the question that interests me most, to be honest. I'd rather
> > >> > > > have the discussion about when to activate the vrings in a different
> > >> > > > thread and keep this one focused on whether the front-end should send
> > >> > > > the kick or if the back-end is expected to check if there are "new"
> > >> > > > buffers in the vring after restore.
> > >> > > >
> > >> > >
> > >> > > IMO we don't need anything from the VMM. When the device receives
> > >> > > SET_VRING_KICK, it can check if the vring already contains buffers
> > >> > > (and this is the part we might need to clarify) and wake-up the other
> > >> > > threads (or always wake-ups them, as crosvm does IIUC, and let them
> > >> > > perform this check).
> > >> > > After sending the SET_VRING_KICK message to the device, the VMM has
> > >> > > the exact same knowledge of the vring state as the device, therefore,
> > >> > > it's still unclear to me why we need to inject that kick.
> > >> > >
> > >> > > Stefano
> > >> > >
> > >> >
> > >> > Is it possible to activate a vring after it has been deactivated with
> > >> > VHOST_USER_GET_VRING_BASE? If yes, does the front-end need to send the
> > >> > kick file descriptor again with VHOST_USER_SET_VRING_KICK to
> > >> > reactivate it?
> > >>
> > >> Hi Jorge and Stefano,
> > >> Yes, VHOST_USER_GET_VRING_BASE -> VHOST_USER_SET_VRING_KICK occurs when
> > >> a VM is paused and then resumed.
> > >>
> > >> You can stress test this by driving I/O using iperf (virtio-net) or fio
> > >> (virtio-blk) inside the guest and sending 'stop'/'cont' commands to
> > >> QEMU's monitor.
> > >>
> > >> Here is QEMU's code for starting (including re-starting) rings:
> > >> https://gitlab.com/qemu-project/qemu/-/blob/master/hw/virtio/vhost.c?ref_type=heads#L1341
> > >>
> > >> QEMU does not inject a kick. The back-end must check the rings itself.
> > >>
> > >> I'm not sure that all vhost-user back-ends actually check the rings. I
> > >> think back-ends should do it, but we should also update the spec with an
> > >> front-end implementation note recommending injecting a kick after
> > >> VHOST_USER_SET_VRING_KICK completes in order to maximize compatibility
> >
> > Okay, but since, as we've seen, no frontend currently implements this,
> > we need to make it clear that a backend shouldn't necessarily expect the
> > kick injected from every frontend, but should support it in some way
> > since some of them can inject it.
> >
> > IMHO especially new backend implementations shouldn't rely on the kick
> > injection.
> >
> > So, to summarize:
> > - the frontend should also send a kick to restart the queues
> > - the backend should restart the queues after VHOST_USER_SET_VRING_KICK,
> >    but it might also receive a kick
> >
> > >> with implementations that follow the current spec wording. And at the
> > >> same time I think the spec should also be changed to say that
> > >> VHOST_USER_SET_VRING_KICK starts the ring and back-ends SHOULD check the
> > >> vring upon processing the message.
> >
> > Yep, I think we are aligned.
> >
> > >>
> > >> That seems like it would clean up the issues without introducing
> > >> compatibility issues or making existing implementations non-compliant
> > >> with the updated spec.
> > >>
> > >> What do you think?
> >
> > LGTM!
> >
> > >
> > >Sounds good to me
> > >
> >
> > @Jorge do you want to propose this change?
> 
> You have a much better idea of where and how this should be written,
> it will save us a few rounds of review if one of you makes that
> change.

I can give it a shot and will CC you.

Also, I wanted to apologize for not being very forthcoming in this
discussion. I had an intuition about this issue but lacked the time to
research and double-check the details. As a result, I didn't respond
to all your points in detail. Sorry if it was frustrating.

Stefan

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 488 bytes --]