Re: [RFC PATCH] PCI: Disable MSI/MSI-X only if device is shutdown

[Fam Zheng <famz@redhat.com>]

On Thu, 03/12 23:09, Bandan Das wrote:
> Ccing Alex, he can probably confirm if my understanding is indeed correct.
> 
> Fam Zheng <famz@redhat.com> writes:
> 
> > On Thu, 03/12 19:56, Bandan Das wrote:
> >> Hi Fam,
> >> 
> >> Fam Zheng <famz@redhat.com> writes:
> >> 
> >> > If the device doesn't support shutdown, disabling interrupts may cause
> >> > trouble. For example, virtio-scsi-pci doesn't implement shutdown, and
> >> > after we disable MSI-X, futher notifications from device will be
> >> > delivered to IRQ, which is unexpected. This IRQ will not be cleared, and
> >> > may prevent us from making progress, by keep triggering interrupts.
> >> >
> >> > Signed-off-by: Fam Zheng <famz@redhat.com>
> >> > ---
> >> >  drivers/pci/pci-driver.c | 7 ++++---
> >> >  1 file changed, 4 insertions(+), 3 deletions(-)
> >> >
> >> > diff --git a/drivers/pci/pci-driver.c b/drivers/pci/pci-driver.c
> >> > index 3cb2210..fb29c96 100644
> >> > --- a/drivers/pci/pci-driver.c
> >> > +++ b/drivers/pci/pci-driver.c
> >> > @@ -448,10 +448,11 @@ static void pci_device_shutdown(struct device *dev)
> >> >  
> >> >  	pm_runtime_resume(dev);
> >> >  
> >> > -	if (drv && drv->shutdown)
> >> > +	if (drv && drv->shutdown) {
> >> >  		drv->shutdown(pci_dev);
> >> > -	pci_msi_shutdown(pci_dev);
> >> > -	pci_msix_shutdown(pci_dev);
> >> > +		pci_msi_shutdown(pci_dev);
> >> > +		pci_msix_shutdown(pci_dev);
> >> > +	}
> >> 
> >> If the driver doesn't provide a shutdown method and doesn't itself
> >> disable MSI/MSI-X, then pci_msi(x)_shutdown won't be called anymore,
> >> no ?
> >
> > Right.
> >
> >> 
> >> This is probably ok (but unclean) for a system shutdown, but could
> >> cause problems for something like kexec ?
> >
> > Doesn't the reset in kexec clean this up during initialization?
> 
> I guess it would take the same path as a reboot.

I don't understand, do you mean that no reset will be done before more
operations on the device?

> 
> > Without shutdown, anything in the driver is unclean anyway, for example
> > free_irq is not called.
> 
> True, And that is why the MSI/-X shutdown functions are called here
> because pci can't always rely on the individual device drivers to do
> the right thing, but atleast can make a best effort at cleaning up.

This virtio-scsi case shows us that intermediate state is bad, so I still think
we should call pci_msix_shutdown conditionally.

Fam