Re: [virtio-comment] Re: [virtio] Re: [PATCH v10 04/10] admin: introduce virtio admin virtqueues

[Stefan Hajnoczi <stefanha@redhat.com>]

[-- Attachment #1: Type: text/plain, Size: 3343 bytes --]

On Tue, Mar 07, 2023 at 09:03:18AM +0100, Jiri Pirko wrote:
> Mon, Mar 06, 2023 at 07:37:31PM CET, mst@redhat.com wrote:
> >On Mon, Mar 06, 2023 at 06:03:40AM -0500, Stefan Hajnoczi wrote:
> >> On Sun, Mar 05, 2023 at 07:18:24PM -0500, Michael S. Tsirkin wrote:
> >> > On Sun, Mar 05, 2023 at 07:03:02PM -0500, Stefan Hajnoczi wrote:
> >> > > On Sun, Mar 05, 2023 at 04:38:59AM -0500, Michael S. Tsirkin wrote:
> >> > > > On Fri, Mar 03, 2023 at 03:21:33PM -0500, Stefan Hajnoczi wrote:
> >> > > > > What happens if a command takes 1 second to complete, is the device
> >> > > > > allowed to process the next command from the virtqueue during this time,
> >> > > > > possibly completing it before the first command?
> >> > > > > 
> >> > > > > This requires additional clarification in the spec because "they are
> >> > > > > processed by the device in the order in which they are queued" does not
> >> > > > > explain whether commands block the virtqueue (in order completion) or
> >> > > > > not (out of order completion).
> >> > > > 
> >> > > > Oh I begin to see. Hmm how does e.g. virtio scsi handle this?
> >> > > 
> >> > > virtio-scsi, virtio-blk, and NVMe requests may complete out of order.
> >> > > Several may be processed by the device at the same time.
> >> > 
> >> > Let's say I submit a write followed by read - is read
> >> > guaranteed to return an up to date info?
> >> 
> >> In general, no. The driver must wait for the write completion before
> >> submitting the read if it wants consistency.
> >> 
> >> Stefan
> >
> >I see.  I think it's a good design to follow then.
> 
> Hmm, is it suitable to have this approach for configuration interface?
> Storage device is a different beast, having parallel reads and writes
> makes complete sense for performance.
> 
> ->read a req
> ->read b req
> ->read c req
> <-read a rep
> <-read b rep
> <-read c rep
> 
> There is no dependency, even between writes.
> 
> But in case of configuration, does not make any sense to me.
> Why is it needed? To pass the burden of consistency of
> configuration to driver sounds odd at least.
> 
> I sense there is no concete idea about what the "admin virtqueue" should
> serve for exactly.

It's useful for long-running commands because they prevent other
commands from executing.

An example I've given is that deleting a group member might require
waiting for the group member's I/O activity to finish. If that I/O
activity cannot be cancelled instantaneously, then it could take an
unbounded amount of time to delete the group member. The device would be
unable to process futher admin commands.

Group member creation might have similar issues if it involves acquiring
remote resources (e.g. connecting to a Ceph cluster or allocating ports
on a distributed network switch). It can be impossible to defer resource
acquisition/initialization because because VIRTIO devices must be
available as soon as the driver can see them (i.e. how do populate
Configuration Space fields if you don't have the details of the resource
yet?).

So I have raised two questions:

1. What are the admin queue command completion semantics: in-order or
   out-of-order command completion?

2. Will there be long-running commands and how will we deal with them
   when they hang?

Stefan

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 488 bytes --]