Re: [PATCH v2] PCI: Mark broken INTx masking for Mellanox devices

[Gavin Shan <gwshan@linux.vnet.ibm.com>]

On Mon, Aug 04, 2014 at 10:34:38AM +1000, Gavin Shan wrote:
>On Sun, Aug 03, 2014 at 09:08:06AM -0600, Alex Williamson wrote:
>>On Mon, 2014-08-04 at 00:30 +1000, Gavin Shan wrote:
>>> On Sun, Aug 03, 2014 at 08:57:39AM +0000, Eli Cohen wrote:
>>> >>>
>>> >>>What is the problem with masking the interrupts with the PCI command 
>>> >>>register? I'm asking because I want to understand in which devices we 
>>> >>>have the problem, and if it could be fixed by firmware guys.
>>> >>>What are the implications of having the quirk?
>>> >>>
>>> >
>>> >>The way to mask the interrupt through PCI command register isn't taking effect on IBM power platform. So we have to have the >quirk so that the interrupt could be masked from interrupt controller side with function disable_irq_nosync().
>>> >>
>>> >>If the interrupt can't be masked properly, we detect interrupt storm reported from host/guest when passing through those devices >via VFIO without suprise.
>>> >
>>> >Hi Gavin,
>>> >Does it have any effect on performance. Also, can you tell in which cases interrupts need to be masked?
>>> >
>>> 
>>> Eli, more code needed to be run for masking the LSI from interrupt controller
>>> side than from PCI command register.
>>> 
>>> I was passing through Mellanox devices from host to guest with VFIO, and I
>>> designated to use LSI in the guest side. More details could be found in
>>> drivers/vfio/pci/vfio_pci_intrs.c::vfio_intx_handler()
>>
>>INTx is relatively high overhead already for device assignment since the
>>interrupt is level triggered and needs to be masked on the host while
>>the guest is processing it.  The more important restriction imposed by
>>marking broken INTx masking is that the device needs an exclusive
>>interrupt line in order to be assigned to a guest.  That may be common
>>practice on IBM power, but on x86 it can make it much harder to
>>configure the system for this use case.  Thanks,
>>
>
>Power platform has the similar situation: Each PCI controller has 4
>LSIs shared by all child devices attached to the PHB. It would be
>racy if one LSI is shared by 2 or more devices. So masking LSI with
>PCI command register is the preferred mechanism. Unfortunately, it
>doesn't work on those 2 Mellanox devices. With the quirk, it's workable
>at least.
>

ping, Any more comments on this? :-)

Thanks,
Gavin