Re: [PATCH v3 4/5] KVM: Kick resamplefd for split kernel irqchip

[Peter Xu <peterx@redhat.com>]

On Tue, Mar 17, 2020 at 03:06:46PM -0600, Alex Williamson wrote:

[...]

> > diff --git a/hw/intc/ioapic.c b/hw/intc/ioapic.c
> > index 15747fe2c2..81a17cc2b8 100644
> > --- a/hw/intc/ioapic.c
> > +++ b/hw/intc/ioapic.c
> > @@ -236,8 +236,29 @@ void ioapic_eoi_broadcast(int vector)
> >          for (n = 0; n < IOAPIC_NUM_PINS; n++) {
> >              entry = s->ioredtbl[n];
> >  
> > -            if ((entry & IOAPIC_VECTOR_MASK) != vector ||
> > -                ((entry >> IOAPIC_LVT_TRIGGER_MODE_SHIFT) & 1) != IOAPIC_TRIGGER_LEVEL) {
> > +            if ((entry & IOAPIC_VECTOR_MASK) != vector) {
> > +                continue;
> > +            }
> > +
> > +            /*
> > +             * When IOAPIC is in the userspace while APIC is still in
> > +             * the kernel (i.e., split irqchip), we have a trick to
> > +             * kick the resamplefd logic for registered irqfds from
> > +             * userspace to deactivate the IRQ.  When that happens, it
> > +             * means the irq bypassed userspace IOAPIC (so the irr and
> > +             * remote-irr of the table entry should be bypassed too
> > +             * even if interrupt come).  Still kick the resamplefds if
> > +             * they're bound to the IRQ, to make sure to EOI the
> > +             * interrupt for the hardware correctly.
> > +             *
> > +             * Note: We still need to go through the irr & remote-irr
> > +             * operations below because we don't know whether there're
> > +             * emulated devices that are using/sharing the same IRQ.
> > +             */
> > +            kvm_resample_fd_notify(n);
> > +
> > +            if (((entry >> IOAPIC_LVT_TRIGGER_MODE_SHIFT) & 1) !=
> > +                IOAPIC_TRIGGER_LEVEL) {
> >                  continue;
> >              }
> >  
> 
> What's the logic for sending resampler notifies before testing if the
> ioapic entry is in level triggered mode?  vfio won't use this for
> anything other than level triggered.  Inserting it between these checks
> confused me and in my testing wasn't necessary.  Thanks,

I put it there to match the kernel implementation, and IIUC Paolo
agreed with that too:

https://patchwork.kernel.org/patch/11407441/#23190969

Since we've discussed a few times here, I think I can talk a bit more
on how I understand this in case I was wrong...

Even if we have the fact that all the existing devices that use this
code should be using level-triggered IRQs, however... *If* there comes
an edge-triggered INTx device and we assign it using vfio-pci, vfio
should also mask the IRQ after it generates (according to
vfio_intx_handler), is that right?  Then we still need to kick the
resamplefd for that does-not-exist device too to make sure it'll work?

Thanks,

> 
> Alex
> 
> > diff --git a/include/sysemu/kvm.h b/include/sysemu/kvm.h
> > index 141342de98..583a976f8a 100644
> > --- a/include/sysemu/kvm.h
> > +++ b/include/sysemu/kvm.h
> > @@ -555,4 +555,8 @@ int kvm_set_one_reg(CPUState *cs, uint64_t id, void *source);
> >  int kvm_get_one_reg(CPUState *cs, uint64_t id, void *target);
> >  struct ppc_radix_page_info *kvm_get_radix_page_info(void);
> >  int kvm_get_max_memslots(void);
> > +
> > +/* Notify resamplefd for EOI of specific interrupts. */
> > +void kvm_resample_fd_notify(int gsi);
> > +
> >  #endif
> 

-- 
Peter Xu