Re: [RFC V1 RESEND 0/6] Introduce dynamic allocation/freeing of MSI-X vectors

[Megha Dey <megha.dey@intel.com>]

On Thu, 2019-08-01 at 19:24 -0500, Bjorn Helgaas wrote:
> Hi Megha,
> 
> On Fri, Jun 21, 2019 at 05:19:32PM -0700, Megha Dey wrote:
> > 
> > Currently, MSI-X vector enabling and allocation for a PCIe device
> > is
> > static i.e. a device driver gets only one chance to enable a
> > specific
> > number of MSI-X vectors, usually during device probe. Also, in many
> > cases, drivers usually reserve more than required number of vectors
> > anticipating their use, which unnecessarily blocks resources that
> > could have been made available to other devices. Lastly, there is
> > no
> > way for drivers to reserve more vectors, if the MSI-x has already
> > been
> > enabled for that device.
> >  
> > Hence, a dynamic MSI-X kernel infrastructure can benefit drivers by
> > deferring MSI-X allocation to post probe phase, where actual demand
> > information is available.
> >  
> > This patchset enables the dynamic allocation/de-allocation of MSI-X
> > vectors by introducing 2 new APIs:
> > pci_alloc_irq_vectors_dyn() and pci_free_irq_vectors_grp():
> > 
> > We have had requests from some of the NIC/RDMA users who have lots
> > of
> > interrupt resources and would like to allocate them on demand,
> > instead of using an all or none approach.
> > 
> > The APIs are fairly well tested (multiple
> > allocations/deallocations),
> > but we have no early adopters yet. Hence, sending this series as an
> > RFC for review and comments.
> > 
> > The patches are based out of Linux 5.2-rc5.
> > 
> > Megha Dey (6):
> >   PCI/MSI: New structures/macros for dynamic MSI-X allocation
> >   PCI/MSI: Dynamic allocation of MSI-X vectors by group
> >   x86: Introduce the dynamic teardown function
> >   PCI/MSI: Introduce new structure to manage MSI-x entries
> s/MSI-x/MSI-X/
> If "entries" here means the same as "vectors" above, please use the
> same word.
> 

Hi Bjorn,

Well, here entries basically mean the msi_desc entries. I thought MSI
vectors are used for each address/data pair, hence used the term
entries. Will update it to vectors to ensure uniformity.

> > 
> >   PCI/MSI: Free MSI-X resources by group
> Is "resources" the same as "vectors"?
> 

Yes, will update this in V2.

> > 
> >   Documentation: PCI/MSI: Document dynamic MSI-X infrastructure
> When you post a v2 after addressing Thomas' comments, please make
> these subject lines imperative sentences beginning with a descriptive
> verb.  You can run "git log --oneline drivers/pci" to see the style.

Sure, I will update the subject lines in V2.

> If you're adding a specific interface or structure, mention it by
> name
> in the subject if it's practical.  The "x86" line needs a little more
> context; I assume it should include "IRQ", "MSI-X", or something.
> 

True, will change this in V2.

> > 
> >  Documentation/PCI/MSI-HOWTO.txt |  38 +++++
> >  arch/x86/include/asm/x86_init.h |   1 +
> >  arch/x86/kernel/x86_init.c      |   6 +
> >  drivers/pci/msi.c               | 363
> > +++++++++++++++++++++++++++++++++++++---
> >  drivers/pci/probe.c             |   9 +
> >  include/linux/device.h          |   3 +
> >  include/linux/msi.h             |  13 ++
> >  include/linux/pci.h             |  61 +++++++
> >  kernel/irq/msi.c                |  34 +++-
> >  9 files changed, 497 insertions(+), 31 deletions(-)
> >