Discussion of the implementations of VIRTIO specification
 help / color / mirror / Atom feed
From: "Eugenio Pérez" <eperezma@redhat.com>
To: virtio-dev@lists.oasis-open.org,
	virtio-comment@lists.oasis-open.org, mst@redhat.com,
	jasowang@redhat.com
Cc: amikheev@nvidia.com, stefanha@redhat.com, shahafs@nvidia.com,
	oren@nvidia.com, pasic@linux.ibm.com, cohuck@redhat.com,
	bodong@nvidia.com,
	"Dr . David Alan Gilbert" <dgilbert@redhat.com>,
	parav@nvidia.com, mgurtovoy@nvidia.com
Subject: [PATCH v3 0/2] virtio: introduce STOP status bit
Date: Thu, 11 Nov 2021 19:58:10 +0100	[thread overview]
Message-ID: <20211111185812.2326093-1-eperezma@redhat.com> (raw)

This patch introduces a new status bit STOP. This can be used by the
driver to stop the device in order to safely fetch used descriptors
status, making sure the device will not fetch new available ones.

Its main use case is live migration, although it has other orthogonal
use cases. It can be used to safely discard requests that have not been
used: in other words, to rewind available descriptors.

Stopping the device in the live migration context is done by per-device
operations in vhost backends, but the introduction of STOP as a basic
virtio facility comes with advantages:
* All the device virtio-specific state is summarized in a single entity,
  making easier to reason about it.
* VMM does not need to implement device specific operations in the
  driver part.
* Work out of the box for devices that use pure virtio backends in some
  part of the device emulation chain (virtio_pci_vdpa or virtio_vdpa),
  in any transport the device can use.
* It's totally self-contained, solving the nested virtualization case
  straightforwardly.

To fully understand its position in the live migration case, it's needed
to note that the VMM acts as a part (or the whole) of the virtio device
from the guest point of view, and it can act as a part of the driver
from an external virtio device point of view. This is already the case
when using vhost-net, for example, where VMM exposes a combination of
backend and VMM features, and can mask them if needed.

To migrate an external device the VMM needs to retrieve its (guest
visible) status and make sure the device does not modify it or
communicate with the guest anymore. The STOP status bit achieves the
last part, and even the first one in case of a pure stateless device
using the split vring.

In its simpler way of working, the VMM masks the VIRTIO_F_STOP feature
to the guest, and also masks the STOP and STOP_FAILED status bit. This
way the VMM can stop and resume operation unilaterally, totally
transparent for the latter.

If we don't need the STOP status bit in the hypervisor but we want the
guest to be able to stop the device, the status can be passthrough.

If we want the guest to be able to stop and
resume the device by itself and VMM does not need LM, the flag and
status must not be masked. If we want both, we need VMM to be able to
override, taking into account the device status for the guest.

v3:
* Delete all virtqueue state saving and restoring, not needed at the
  moment.
* Add STOP_FAILED bit so device can fail the operation
* Add config interrupt to notify driver about the stop bit is set, so it
  can avoid busy waiting polling status.
* Expand device's required treatment to in-flight descriptors before
  setting the STOP bit.
* Add rewind capabilities.
* Add resume operation, clearing the STOP bit.
* Reword status clear bit / PCI reset contradictions, already present in
  the spec
* Specify device behavior if STOP status bit is set before DRIVER_OK

Signed-off-by: Jason Wang <jasowang@redhat.com>
Signed-off-by: Eugenio Pérez <eperezma@redhat.com>

Eugenio Pérez (1):
  content: Explain better the status clearing bits

Jason Wang (1):
  virtio: introduce STOP status bit

 content.tex | 90 +++++++++++++++++++++++++++++++++++++++++++++++++++--
 1 file changed, 87 insertions(+), 3 deletions(-)

-- 
2.27.0



             reply	other threads:[~2021-11-11 18:58 UTC|newest]

Thread overview: 43+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2021-11-11 18:58 Eugenio Pérez [this message]
2021-11-11 18:58 ` [PATCH v3 1/2] content: Explain better the status clearing bits Eugenio Pérez
2021-11-12  3:46   ` Jason Wang
2021-11-12 11:41     ` Eugenio Perez Martin
2021-11-12 10:34   ` [virtio-dev] " Cornelia Huck
2021-11-12 11:41     ` Eugenio Perez Martin
2021-11-11 18:58 ` [PATCH v3 2/2] virtio: introduce STOP status bit Eugenio Pérez
2021-11-12  4:18   ` Jason Wang
2021-11-12 10:50     ` Eugenio Perez Martin
2021-11-15  4:08       ` Jason Wang
2021-11-15 18:16         ` Eugenio Perez Martin
2021-11-16  6:56           ` Jason Wang
2021-11-16 14:50             ` Eugenio Perez Martin
2021-11-17  3:27               ` Jason Wang
2021-11-17  8:08                 ` Eugenio Perez Martin
2021-11-18  3:27                   ` Jason Wang
2021-11-18 15:59   ` Stefan Hajnoczi
2021-11-18 19:58     ` Eugenio Perez Martin
2021-11-23 12:16       ` Stefan Hajnoczi
2021-11-23 17:00         ` [virtio-dev] " Eugenio Perez Martin
2021-11-24 11:20           ` Stefan Hajnoczi
2021-11-24 16:41             ` Eugenio Perez Martin
2021-11-29 10:32               ` Stefan Hajnoczi
2021-11-25  2:57             ` Jason Wang
2021-11-29 10:29               ` Stefan Hajnoczi
2021-11-29 16:55                 ` Eugenio Perez Martin
2021-12-01 10:21                   ` Stefan Hajnoczi
2021-12-02  8:30                     ` Eugenio Perez Martin
2021-12-02  2:40                   ` Jason Wang
2021-12-02  9:44                     ` Stefan Hajnoczi
2021-12-03  2:09                       ` Jason Wang
2021-11-18 14:45 ` [PATCH v3 0/2] " Stefan Hajnoczi
2021-11-18 16:49   ` Eugenio Perez Martin
2021-11-23 11:33     ` Stefan Hajnoczi
2021-11-23 16:19       ` Eugenio Perez Martin
2021-11-24 15:26         ` Stefan Hajnoczi
2021-11-24 16:58           ` Eugenio Perez Martin
2021-11-25  3:05         ` Jason Wang
2021-11-25  7:24           ` Eugenio Perez Martin
2021-11-25  7:38             ` Jason Wang
2021-11-25  9:01               ` Eugenio Perez Martin
2021-11-25  9:10                 ` Eugenio Perez Martin
     [not found]                 ` <CACGkMEvD+Z7cYszhMzBsnEaC0K0kfnHxzFDEfjT_qLOFiMR-XA@mail.gmail.com>
2021-11-26  8:26                   ` Eugenio Perez Martin

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20211111185812.2326093-1-eperezma@redhat.com \
    --to=eperezma@redhat.com \
    --cc=amikheev@nvidia.com \
    --cc=bodong@nvidia.com \
    --cc=cohuck@redhat.com \
    --cc=dgilbert@redhat.com \
    --cc=jasowang@redhat.com \
    --cc=mgurtovoy@nvidia.com \
    --cc=mst@redhat.com \
    --cc=oren@nvidia.com \
    --cc=parav@nvidia.com \
    --cc=pasic@linux.ibm.com \
    --cc=shahafs@nvidia.com \
    --cc=stefanha@redhat.com \
    --cc=virtio-comment@lists.oasis-open.org \
    --cc=virtio-dev@lists.oasis-open.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox