From: helgaas@kernel.org (Bjorn Helgaas)
Subject: [PATCH V2 1/3] PCI/MSI: preference to returning -ENOSPC from pci_alloc_irq_vectors_affinity
Date: Wed, 2 Jan 2019 15:02:02 -0600 [thread overview]
Message-ID: <20190102210202.GC126384@google.com> (raw)
In-Reply-To: <20190101052458.GA17588@ming.t460p>
On Tue, Jan 01, 2019@01:24:59PM +0800, Ming Lei wrote:
> On Mon, Dec 31, 2018@04:00:59PM -0600, Bjorn Helgaas wrote:
> > On Sat, Dec 29, 2018@11:26:48AM +0800, Ming Lei wrote:
...
> > > Users of pci_alloc_irq_vectors_affinity() may try to reduce irq
> > > vectors and allocate vectors again in case that -ENOSPC is returned, such
> > > as NVMe, so we need to respect the current interface and give preference to
> > > -ENOSPC.
> >
> > I thought the whole point of the (min_vecs, max_vecs) tuple was to
> > avoid this sort of "reduce and try again" iteration in the callers.
>
> As Keith replied, in case of NVMe, we have to keep min_vecs same with
> max_vecs.
Keith said:
> The min/max vecs doesn't work correctly when using the irq_affinity
> nr_sets because rebalancing the set counts is driver specific. To
> get around that, drivers using nr_sets have to set min and max to
> the same value and handle the "reduce and try again".
Sorry I saw that, but didn't follow it at first. After a little
archaeology, I see that 6da4b3ab9a6e ("genirq/affinity: Add support
for allocating interrupt sets") added nr_sets and some validation
tests (if affd.nr_sets, min_vecs == max_vecs) for using it in the API.
That's sort of a wart on the API, but I don't know if we should live
with it or try to clean it up somehow.
At the very least, this seems like something that could be documented
somewhere in Documentation/PCI/MSI-HOWTO.txt, which mentions
PCI_IRQ_AFFINITY, but doesn't cover struct irq_affinity or
pci_alloc_irq_vectors_affinity() at all, let alone this wrinkle about
affd.nr_sets/min_vecs/max_vecs.
Obviously that would not be part of *this* patch.
Bjorn
WARNING: multiple messages have this Message-ID (diff)
From: Bjorn Helgaas <helgaas@kernel.org>
To: Ming Lei <ming.lei@redhat.com>
Cc: linux-nvme@lists.infradead.org, Christoph Hellwig <hch@lst.de>,
Jens Axboe <axboe@fb.com>, Keith Busch <keith.busch@intel.com>,
linux-pci@vger.kernel.org
Subject: Re: [PATCH V2 1/3] PCI/MSI: preference to returning -ENOSPC from pci_alloc_irq_vectors_affinity
Date: Wed, 2 Jan 2019 15:02:02 -0600 [thread overview]
Message-ID: <20190102210202.GC126384@google.com> (raw)
In-Reply-To: <20190101052458.GA17588@ming.t460p>
On Tue, Jan 01, 2019 at 01:24:59PM +0800, Ming Lei wrote:
> On Mon, Dec 31, 2018 at 04:00:59PM -0600, Bjorn Helgaas wrote:
> > On Sat, Dec 29, 2018 at 11:26:48AM +0800, Ming Lei wrote:
...
> > > Users of pci_alloc_irq_vectors_affinity() may try to reduce irq
> > > vectors and allocate vectors again in case that -ENOSPC is returned, such
> > > as NVMe, so we need to respect the current interface and give preference to
> > > -ENOSPC.
> >
> > I thought the whole point of the (min_vecs, max_vecs) tuple was to
> > avoid this sort of "reduce and try again" iteration in the callers.
>
> As Keith replied, in case of NVMe, we have to keep min_vecs same with
> max_vecs.
Keith said:
> The min/max vecs doesn't work correctly when using the irq_affinity
> nr_sets because rebalancing the set counts is driver specific. To
> get around that, drivers using nr_sets have to set min and max to
> the same value and handle the "reduce and try again".
Sorry I saw that, but didn't follow it at first. After a little
archaeology, I see that 6da4b3ab9a6e ("genirq/affinity: Add support
for allocating interrupt sets") added nr_sets and some validation
tests (if affd.nr_sets, min_vecs == max_vecs) for using it in the API.
That's sort of a wart on the API, but I don't know if we should live
with it or try to clean it up somehow.
At the very least, this seems like something that could be documented
somewhere in Documentation/PCI/MSI-HOWTO.txt, which mentions
PCI_IRQ_AFFINITY, but doesn't cover struct irq_affinity or
pci_alloc_irq_vectors_affinity() at all, let alone this wrinkle about
affd.nr_sets/min_vecs/max_vecs.
Obviously that would not be part of *this* patch.
Bjorn
next prev parent reply other threads:[~2019-01-02 21:02 UTC|newest]
Thread overview: 32+ messages / expand[flat|nested] mbox.gz Atom feed top
2018-12-29 3:26 [PATCH V2 0/3] nvme pci: two fixes on nvme_setup_irqs Ming Lei
2018-12-29 3:26 ` Ming Lei
2018-12-29 3:26 ` [PATCH V2 1/3] PCI/MSI: preference to returning -ENOSPC from pci_alloc_irq_vectors_affinity Ming Lei
2018-12-29 3:26 ` Ming Lei
2018-12-31 22:00 ` Bjorn Helgaas
2018-12-31 22:00 ` Bjorn Helgaas
2018-12-31 22:41 ` Keith Busch
2018-12-31 22:41 ` Keith Busch
2019-01-01 5:24 ` Ming Lei
2019-01-01 5:24 ` Ming Lei
2019-01-02 21:02 ` Bjorn Helgaas [this message]
2019-01-02 21:02 ` Bjorn Helgaas
2019-01-02 22:46 ` Keith Busch
2019-01-02 22:46 ` Keith Busch
2018-12-29 3:26 ` [PATCH V2 2/3] nvme pci: fix nvme_setup_irqs() Ming Lei
2018-12-29 3:26 ` [PATCH V2 3/3] nvme pci: introduce module parameter of 'default_queues' Ming Lei
2018-12-31 21:24 ` Bjorn Helgaas
2019-01-01 5:47 ` Ming Lei
2019-01-02 2:14 ` Shan Hai
[not found] ` <20190102073607.GA25590@ming.t460p>
[not found] ` <d59007c6-af13-318c-5c9d-438ad7d9149d@oracle.com>
[not found] ` <20190102083901.GA26881@ming.t460p>
2019-01-03 2:04 ` Shan Hai
2019-01-02 20:11 ` Bjorn Helgaas
2019-01-03 2:12 ` Ming Lei
2019-01-03 2:52 ` Shan Hai
2019-01-03 3:11 ` Shan Hai
2019-01-03 3:31 ` Ming Lei
2019-01-03 4:36 ` Shan Hai
2019-01-03 10:34 ` Ming Lei
2019-01-04 2:53 ` Shan Hai
2019-01-03 4:51 ` Shan Hai
2019-01-03 3:21 ` Ming Lei
2019-01-14 13:13 ` [PATCH V2 0/3] nvme pci: two fixes on nvme_setup_irqs John Garry
2019-01-14 13:13 ` John Garry
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20190102210202.GC126384@google.com \
--to=helgaas@kernel.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.