Re: [virtio-dev] On doorbells (queue notifications)

["Alex Bennée" <alex.bennee@linaro.org>]

Cornelia Huck <cohuck@redhat.com> writes:

> On Wed, 15 Jul 2020 16:47:32 +0100
> Stefan Hajnoczi <stefanha@redhat.com> wrote:
>
>> On Wed, Jul 15, 2020 at 02:29:04PM +0100, Alex Bennée wrote:
>> > Stefan Hajnoczi <stefanha@redhat.com> writes:  
>> > > On Tue, Jul 14, 2020 at 10:43:36PM +0100, Alex Bennée wrote:  
>> > >> Finally I'm curious if this is just a problem avoided by the s390
>> > >> channel approach? Does the use of messages over a channel just avoid the
>> > >> sort of bouncing back and forth that other hypervisors have to do when
>> > >> emulating a device?  
>> > >
>> > > What does "bouncing back and forth" mean exactly?  
>> > 
>> > Context switching between guest and hypervisor.  
>> 
>> I have CCed Cornelia Huck, who can explain the lifecycle of an I/O
>> request on s390 channel I/O.
>
> Having read through this thread, I think this is mostly about
> notifications?

Yes - as I understand it they are the only things that really cause a
context switch between guest/hypervisor/host.

> These are not using channel programs (which are only
> used for things like feature negotiation, or emulating reading/writing
> a config space, which does not really exist for channel devices.)
>
> First, I/O and interrupts are highly abstracted on s390; much of the
> register accesses or writes done on other architectures is just not
> seen on s390.
>
> Traditionally, I/O interrupts on s390 are tied to a subchannel; you
> have a rather heavyweight process for that:
>
> guest								host
>
> 					put status into subchannel
> 					queue interrupt
> open up for I/O interrupt
> 					store some data into lowcore
> 					do PSW swap
> interrupt handler called
> read from lowcore
> call tsch for subchannel
> 					store subchannel status into
> 					control block
> process control block
> look at subchannel indicators
> virtio queue processing
>
> This is only used for configuration change notifications, or for very
> old legacy virtio implementations.
>
> There's an alternative mechanism not tied to a subchannel, called
> 'adapter interrupts'. (It is even used to implement MSI-X on s390x,
> which is why only virtio-pci devices using MSI-X are supported on
> s390x.) It uses two-staged indicators: a global indicator to show
> whether any secondary indicator is set, and secondary indicators (which
> are per virtqueue in the virtio case.)
>
> guest								host
>
> 					set queue indicator(s)
> 					set global indicator
> 					queue interrupt iff global
> 					indicator had not been set
> open up for I/O interrupt
> 					store some data into lowcore
> 					do PSW swap
> interrupt handler called
> read from lowcore
> look at indicators
> virtio queue processing
>
> This has less context switches than traditional I/O interrupts; but I
> think the main benefit comes from the ability to batch notifications:
> as long as the guest is still processing indicators, the host does not
> need to notify again, it can just set indicators (which is why the
> guest always needs to do two passes at processing.) We can already
> batch per-device indicators with the classic approach, but adapter
> interrupts allow to batch even across many devices.

Thanks for the explanation.

I'm curious why the data that's going to be read from lowcore isn't
loaded before the guest opens up (is this the same as unmasking?) for
the interrupt? Is this because the host has to set up the guest IRQ
itself?

-- 
Alex Bennée

---------------------------------------------------------------------
To unsubscribe, e-mail: virtio-dev-unsubscribe@lists.oasis-open.org
For additional commands, e-mail: virtio-dev-help@lists.oasis-open.org