Re: [Qemu-devel] [PATCH 1/3] spapr pci msi: rework

[Benjamin Herrenschmidt <benh@kernel.crashing.org>]

On Fri, 2013-06-21 at 12:31 +0200, Alexander Graf wrote:
> On 21.06.2013, at 11:22, Alexey Kardashevskiy wrote:
> 
> > Previously every PCI host bridge implemented its own MSI memory window
> > in order to catch msi_notify()/msix_notify() calls from various QEMU
> > MSI-capable devives such as virtio-pci or vfio and redirect them to
> > the guest via qemu_pulse_irq().
> 
> That's how hardware works, no?
> 
> > 
> > The encoded MSIMessage used to be encoded as:
> > * .addr - address in a MSI window, this is how QEMU knows which PHB
> > is the message for;
> > * .data - number of a device on a specific PHB and vector number.
> > 
> > As a PHB has a destriptor for every device, and every descriptor has
> > first IRQ number and the number of IRQs, it can calculate global IRQ
> > number to use in qemu_pulse_irq().
> 
> How does this work on real hardware?

Why would we care ?

The MSIs are established by an hcall anyway, the guest sees nothing of
that, we can encode what we want in there.

Now the way it works on real HW is different from Alex did originally.

The address of the MSI defines an MSI "port" which is purely a mechanism
for verifying that the device is authorized to do MSIs (to that port)
and associate a PE# with it, which is cross-match with the PHB-local MSI
index which is the message  value

That value is then ORed with a PHB-configured "BUID" which forms the
full global interrupt number.

At least on P7IOC ... P8 is a bit different. But we don't really need to
reproduce that exactly.

Ben.

> 
> Alex
> 
> > However the total number of IRQs is not really big (at the moment it is
> > 1024 IRQs which start from 4096) and the existing system looks overdesigned.
> > The patch simplifies it. Specifically:
> > 
> > 1. MSI windows were removed from PHB.
> > 2. Added one memory region for all MSIs.
> > 3. Now MSIMessage::addr is a number of first IRQ of a device,
> > MSIMessage:data is a number of a vector.
> > 
> > Putting IRQ number to .data and not using .addr would make it even simpler
> > for MSI-X but it will not work for MSI with multiple vectors unless a first
> > IRQ number of a device is aligned to the MSI vectors number.
> > 
> > The simplified scheme also allows easier MSIMessage->IRQ translation
> > for upcoming IRQFD support.
> > 
> > Signed-off-by: Alexey Kardashevskiy <aik@ozlabs.ru>