Re: MSI, fun for the whole family

[Roland Dreier <rdreier@cisco.com>]

 > > To my knowledge there is not any information that varies each time an
 > > interrupt fires.
 > 
 > Absolutely there is!  This is why MSI is so cool.
 > 
 > You get a tiny chunk of data from the hardware, across the PCI bus in
 > a single PCI transaction, sent [well, basically...] straight to the
 > driver __for each MSI interrupt__.  Rather than having a separate
 > interrupt line -- really an ugly OOB mechanism -- you get a bus
 > transaction as God intended, a bus transaction just like all the
 > others going across the PCI bus.

I think you've fundamentally misunderstood what the PCI spec for MSI
multi message means.  It is true that if the whole system agrees, then
an MSI-capable device that supports multiple messages might be allocated
a range of vectors (MSI is kind of stupid because it only allows
multiple messages to be generated by varying the low order bits -- MSI-X
fixes this limitation).  However, the way that these different messages
are handled is that they are all independent interrupt vectors.

Now, it is true that the kernel could do something crazy and collapse
all these interrupt vectors into a single "IRQ" and then tell the
interrupt handler which vector it was by passing some "metadata" in, but
why not just give each MSI message it's own IRQ?

By the way, for MSI on Linux this is theoretical, since MSI messages
have to be consecutive, and the kernel (on standard x86 at least) can't
really allocate a chunk of consecutive vectors.  However, Linux has
quite a few device drivers that handle devices that generate multiple
MSI-X messages, and the model of having each message be a different IRQ
works quite well.

You are correct that MSI/MSI-X has the advantage of being an in-band
transaction that obeys PCI ordering rules, rather than going through an
asynchronous separate wire that might reach the CPU before the DMA it is
signaling.

 > An AHCI PCI device uses MSI messages to inform the driver which <mask>
 > of 32 SATA ports have asserted an activity indication.

Actually from the spec it looks like it uses multiple MSI messages to
give each port its own interrupt.  There aren't enough bits available to
| together a bitwise port mask, and it wouldn't be sane to have eg 3
separate interrupt handlers for port 1 interrupts, port 2 interrupts,
and port 1 *and* 2 interrupts.

 - R.