Re: [RFC][PATCH] KVM: Introduce direct MSI message injection for in-kernel irqchips

["Michael S. Tsirkin" <mst@redhat.com>]

On Tue, Oct 25, 2011 at 09:24:17AM +0200, Jan Kiszka wrote:
> On 2011-10-24 19:23, Michael S. Tsirkin wrote:
> > On Mon, Oct 24, 2011 at 07:05:08PM +0200, Michael S. Tsirkin wrote:
> >> On Mon, Oct 24, 2011 at 06:10:28PM +0200, Jan Kiszka wrote:
> >>> On 2011-10-24 18:05, Michael S. Tsirkin wrote:
> >>>>> This is what I have in mind:
> >>>>>  - devices set PBA bit if MSI message cannot be sent due to mask (*)
> >>>>>  - core checks&clears PBA bit on unmask, injects message if bit was set
> >>>>>  - devices clear PBA bit if message reason is resolved before unmask (*)
> >>>>
> >>>> OK, but practically, when exactly does the device clear PBA?
> >>>
> >>> Consider a network adapter that signals messages in a RX ring: If the
> >>> corresponding vector is masked while the guest empties the ring, I
> >>> strongly assume that the device is supposed to take back the pending bit
> >>> in that case so that there is no interrupt inject on a later vector
> >>> unmask operation.
> >>>
> >>> Jan
> >>
> >> Do you mean virtio here?
> 
> Maybe, but I'm also thinking of fully emulated devices.

One thing seems certain: actual, assigned devices don't
have this fake "msi-x level" so they don't notify host
when that changes.

> > Do you expect this optimization to give
> >> a significant performance gain?
> 
> Hard to asses in general. But I have a silly guest here that obviously
> masks MSI vectors for each event. This currently not only kicks us into
> a heavy-weight exit, it also enforces serialization on qemu_global_mutex
> (while we have the rest already isolated).

It easy to see how MSIX mask support in kernel would help.
Not sure whether it's worth it to also add special APIs to
reduce the number of spurious interrupts for such silly guests.

> > 
> > It would also be challenging to implement this in
> > a race free manner. Clearing on interrupt status read
> > seems straight-forward.
> 
> With an in-kernel MSI-X MMIO handler, this race will be naturally
> unavoidable as there is no more global lock shared between table/PBA
> accesses and the device model. But, when using atomic bit ops, I don't
> think that will cause headache.
> 
> Jan

This is not the race I meant.  The challenge is for the device to
determine that it can clear the PBA.  atomic accesses on PBA won't help
here I think.


> -- 
> Siemens AG, Corporate Technology, CT T DE IT 1
> Corporate Competence Center Embedded Linux