Re: [PATCH 2 08/24] hpsa: choose number of reply queues more intelligently.

[scameron@beardog.cce.hp.com]

On Mon, Jun 02, 2014 at 09:52:06AM -0500, scameron@beardog.cce.hp.com wrote:
> On Mon, Jun 02, 2014 at 02:27:51AM -0700, Christoph Hellwig wrote:
> > On Thu, May 29, 2014 at 10:53:02AM -0500, Stephen M. Cameron wrote:
> > > From: Stephen M. Cameron <scameron@beardog.cce.hp.com>
> > > 
> > > No sense having 8 or 16 reply queues if you only have 4 cpus,
> > > and likewise no sense limiting to 8 reply queues if you have
> > > many more cpus.
> > 
> > I've applied this as it looks good as-is, but shouldn't we also
> > cap the number of MSI-X vectors in common code so that we avoid
> > adding this as boilerplate code to lots of drivers?
> 
> Maybe so.  Thinking about CPU hotplug, is num_online_cpus()
> the right cap?  Maybe num_possible_cpus() is better in case
> additional cpus coming online are anticipated?

On second thought, might there be PCI devices which have multiple
MSIX vectors to signal different things?  I know for smart array
the idea has occasionally been floated to have an additional vector
to signal something to the host that is not a command completion,
although we never implemented such a thing.  Had we done so,
we would then need num_online_cpus() + 1 vectors.

So maybe it's not a good idea to put such a limit in the generic
pci code in case some device works in such a way.

-- steve


> 
> > 
> > > 
> > > Signed-off-by: Stephen M. Cameron <scameron@beardog.cce.hp.com>
> > > Reviewed-by: Mike Miller <michael.miller@canonical.com>
> > > Reviewed-by: Scott Teel <scott.teel@hp.com>
> > > ---
> > >  drivers/scsi/hpsa.c     |    2 ++
> > >  drivers/scsi/hpsa_cmd.h |    2 +-
> > >  2 files changed, 3 insertions(+), 1 deletions(-)
> > > 
> > > diff --git a/drivers/scsi/hpsa.c b/drivers/scsi/hpsa.c
> > > index b1ecfd8..b903e86 100644
> > > --- a/drivers/scsi/hpsa.c
> > > +++ b/drivers/scsi/hpsa.c
> > > @@ -6157,6 +6157,8 @@ static void hpsa_interrupt_mode(struct ctlr_info *h)
> > >  	if (pci_find_capability(h->pdev, PCI_CAP_ID_MSIX)) {
> > >  		dev_info(&h->pdev->dev, "MSIX\n");
> > >  		h->msix_vector = MAX_REPLY_QUEUES;
> > > +		if (h->msix_vector > num_online_cpus())
> > > +			h->msix_vector = num_online_cpus();
> > >  		err = pci_enable_msix(h->pdev, hpsa_msix_entries,
> > >  				      h->msix_vector);
> > >  		if (err > 0) {
> > > diff --git a/drivers/scsi/hpsa_cmd.h b/drivers/scsi/hpsa_cmd.h
> > > index db89245..104b67b 100644
> > > --- a/drivers/scsi/hpsa_cmd.h
> > > +++ b/drivers/scsi/hpsa_cmd.h
> > > @@ -615,7 +615,7 @@ struct TransTable_struct {
> > >  	u32            RepQCount;
> > >  	u32            RepQCtrAddrLow32;
> > >  	u32            RepQCtrAddrHigh32;
> > > -#define MAX_REPLY_QUEUES 8
> > > +#define MAX_REPLY_QUEUES 64
> > >  	struct vals32  RepQAddr[MAX_REPLY_QUEUES];
> > >  };
> > >  
> > > 
> > > --
> > > To unsubscribe from this list: send the line "unsubscribe linux-scsi" in
> > > the body of a message to majordomo@vger.kernel.org
> > > More majordomo info at  http://vger.kernel.org/majordomo-info.html
> > ---end quoted text---