Re: Multiple MSI, take 3

[Benjamin Herrenschmidt <benh@kernel.crashing.org>]

> It is edge triggered so it won't refire when unmasked (especially if we don't know).
> So it is easy to wind up in a state where the device is waiting for the software
> and the software is waiting for the device because an irq gets dropped.

Well, we are smarter than that. soft-masking is a know well-solved
problem. We just latch that something happened while masked and refire
when unmasked. Not terribly hard. We already do that in various
situations to mask edge interrupts.

> There are enough places that have problems that we have a fairly standard work around
> to the problem (listed above) by just taking the first irq (after we have disabled the
> irq) and setting it pending in software and then actually masking it in hardware.

Masking in HW is totally optional. I don't mask in HW on cell for
example, the HW just can't.

> That works, but it is still isn't quite correct.  Because we can run the
> interrupt handler once to often.

We only re-fire if it actually occured while "masked", that should take
care that we never fire once too much, no ?

>   For interrupts that are never shared and
> always in order with the DMA, generally don't require reading a status
> register on the card, and are otherwise highly optimized that might actually
> be a problem.

There must be some way of knowing what work is to do (ie, whether a DMA
q entry is completed, some kind of done bit, etc...). There generally is
at least, so that even in that case, spurrious MSIs are mostly a non
issue, but I don't think we have them here.
 
> Which is why I said that it doesn't look like even using an iommu can
> fix all of the issues with treating msi multi message mode messages
> as individual irqs.  We can get very close but not quite there.

I still mostly dislike the new approach, I prefer Matthew's original one
with SW masking of the MSIs. For example, if you have the MSIs be 'one'
interrupt, then you hit all of the logic in the IRQ core to make sure
only one happens at once. Might not be what you want, and -will- cause
some to be dropped... not nice.

Ben.