From: Keith Busch <keith.busch@intel.com>
To: linux-nvme@lists.infradead.org, linux-block@vger.kernel.org,
Jens Axboe <axboe@kernel.dk>, Jens Axboe <axboe@fb.com>,
Christoph Hellwig <hch@lst.de>,
Thomas Gleixner <tglx@linutronix.de>
Cc: Marc Merlin <marc@merlins.org>, Keith Busch <keith.busch@intel.com>
Subject: [PATCH 1/6] irq/affinity: Assign all online CPUs to vectors
Date: Wed, 4 Jan 2017 17:41:06 -0500 [thread overview]
Message-ID: <1483569671-1462-2-git-send-email-keith.busch@intel.com> (raw)
In-Reply-To: <1483569671-1462-1-git-send-email-keith.busch@intel.com>
This patch makes sure all online CPUs are assigned to vectors in
cases where the nodes don't have the same number of online CPUs.
The calculation for how many vectors needs to be assigned should account
for the number of CPUs for a particular node on each round of assignment
in order to ensure all online CPUs are assigned a vector when they don't
evenly divide, and calculate extras accordingly.
Since we attempt to divide evently among the nodes, this may still result
in unused vectors if some nodes have fewer CPUs than nodes previosuly
set up, but at least every online CPU will be assigned to something.
Signed-off-by: Keith Busch <keith.busch@intel.com>
---
kernel/irq/affinity.c | 9 ++++++---
1 file changed, 6 insertions(+), 3 deletions(-)
diff --git a/kernel/irq/affinity.c b/kernel/irq/affinity.c
index 4544b11..b25dce0 100644
--- a/kernel/irq/affinity.c
+++ b/kernel/irq/affinity.c
@@ -96,17 +96,19 @@ irq_create_affinity_masks(int nvecs, const struct irq_affinity *affd)
/* Spread the vectors per node */
vecs_per_node = affv / nodes;
- /* Account for rounding errors */
- extra_vecs = affv - (nodes * vecs_per_node);
for_each_node_mask(n, nodemsk) {
- int ncpus, v, vecs_to_assign = vecs_per_node;
+ int ncpus, v, vecs_to_assign;
/* Get the cpus on this node which are in the mask */
cpumask_and(nmsk, cpu_online_mask, cpumask_of_node(n));
/* Calculate the number of cpus per vector */
ncpus = cpumask_weight(nmsk);
+ vecs_to_assign = min(vecs_per_node, ncpus);
+
+ /* Account for rounding errors */
+ extra_vecs = ncpus - vecs_to_assign;
for (v = 0; curvec < last_affv && v < vecs_to_assign;
curvec++, v++) {
@@ -123,6 +125,7 @@ irq_create_affinity_masks(int nvecs, const struct irq_affinity *affd)
if (curvec >= last_affv)
break;
+ vecs_per_node = (affv - curvec) / --nodes;
}
done:
--
2.5.5
WARNING: multiple messages have this Message-ID (diff)
From: keith.busch@intel.com (Keith Busch)
Subject: [PATCH 1/6] irq/affinity: Assign all online CPUs to vectors
Date: Wed, 4 Jan 2017 17:41:06 -0500 [thread overview]
Message-ID: <1483569671-1462-2-git-send-email-keith.busch@intel.com> (raw)
In-Reply-To: <1483569671-1462-1-git-send-email-keith.busch@intel.com>
This patch makes sure all online CPUs are assigned to vectors in
cases where the nodes don't have the same number of online CPUs.
The calculation for how many vectors needs to be assigned should account
for the number of CPUs for a particular node on each round of assignment
in order to ensure all online CPUs are assigned a vector when they don't
evenly divide, and calculate extras accordingly.
Since we attempt to divide evently among the nodes, this may still result
in unused vectors if some nodes have fewer CPUs than nodes previosuly
set up, but at least every online CPU will be assigned to something.
Signed-off-by: Keith Busch <keith.busch at intel.com>
---
kernel/irq/affinity.c | 9 ++++++---
1 file changed, 6 insertions(+), 3 deletions(-)
diff --git a/kernel/irq/affinity.c b/kernel/irq/affinity.c
index 4544b11..b25dce0 100644
--- a/kernel/irq/affinity.c
+++ b/kernel/irq/affinity.c
@@ -96,17 +96,19 @@ irq_create_affinity_masks(int nvecs, const struct irq_affinity *affd)
/* Spread the vectors per node */
vecs_per_node = affv / nodes;
- /* Account for rounding errors */
- extra_vecs = affv - (nodes * vecs_per_node);
for_each_node_mask(n, nodemsk) {
- int ncpus, v, vecs_to_assign = vecs_per_node;
+ int ncpus, v, vecs_to_assign;
/* Get the cpus on this node which are in the mask */
cpumask_and(nmsk, cpu_online_mask, cpumask_of_node(n));
/* Calculate the number of cpus per vector */
ncpus = cpumask_weight(nmsk);
+ vecs_to_assign = min(vecs_per_node, ncpus);
+
+ /* Account for rounding errors */
+ extra_vecs = ncpus - vecs_to_assign;
for (v = 0; curvec < last_affv && v < vecs_to_assign;
curvec++, v++) {
@@ -123,6 +125,7 @@ irq_create_affinity_masks(int nvecs, const struct irq_affinity *affd)
if (curvec >= last_affv)
break;
+ vecs_per_node = (affv - curvec) / --nodes;
}
done:
--
2.5.5
next prev parent reply other threads:[~2017-01-04 22:41 UTC|newest]
Thread overview: 44+ messages / expand[flat|nested] mbox.gz Atom feed top
2017-01-04 22:41 [PATCH 0/6] NVMe related fixes Keith Busch
2017-01-04 22:41 ` Keith Busch
2017-01-04 22:41 ` Keith Busch [this message]
2017-01-04 22:41 ` [PATCH 1/6] irq/affinity: Assign all online CPUs to vectors Keith Busch
2017-01-13 20:21 ` Sagi Grimberg
2017-01-13 20:21 ` Sagi Grimberg
2017-01-23 18:30 ` Christoph Hellwig
2017-01-23 18:30 ` Christoph Hellwig
2017-01-04 22:41 ` [PATCH 2/6] irq/affinity: Assign offline CPUs a vector Keith Busch
2017-01-04 22:41 ` Keith Busch
2017-01-08 10:01 ` Christoph Hellwig
2017-01-08 10:01 ` Christoph Hellwig
2017-01-13 20:26 ` Sagi Grimberg
2017-01-13 20:26 ` Sagi Grimberg
2017-01-04 22:41 ` [PATCH 3/6] nvme/pci: Start queues after tagset is updated Keith Busch
2017-01-04 22:41 ` Keith Busch
2017-01-13 20:38 ` Sagi Grimberg
2017-01-13 20:38 ` Sagi Grimberg
2017-01-23 18:31 ` Christoph Hellwig
2017-01-23 18:31 ` Christoph Hellwig
2017-01-04 22:41 ` [PATCH 4/6] blk-mq: Update queue map when changing queue count Keith Busch
2017-01-04 22:41 ` Keith Busch
2017-01-13 20:39 ` Sagi Grimberg
2017-01-13 20:39 ` Sagi Grimberg
2017-01-23 18:32 ` Christoph Hellwig
2017-01-23 18:32 ` Christoph Hellwig
2017-01-04 22:41 ` [PATCH 5/6] blk-mq: Fix queue freeze deadlock Keith Busch
2017-01-04 22:41 ` Keith Busch
2017-01-05 7:33 ` Bart Van Assche
2017-01-05 7:33 ` Bart Van Assche
2017-01-17 17:53 ` Keith Busch
2017-01-17 17:53 ` Keith Busch
2017-01-13 21:05 ` Sagi Grimberg
2017-01-13 21:05 ` Sagi Grimberg
2017-01-17 18:00 ` Keith Busch
2017-01-17 18:00 ` Keith Busch
2017-01-19 7:54 ` Sagi Grimberg
2017-01-19 7:54 ` Sagi Grimberg
2017-01-04 22:41 ` [PATCH 6/6] blk-mq: Remove unused variable Keith Busch
2017-01-04 22:41 ` Keith Busch
2017-01-08 10:02 ` Christoph Hellwig
2017-01-08 10:02 ` Christoph Hellwig
2017-01-13 21:05 ` Sagi Grimberg
2017-01-13 21:05 ` Sagi Grimberg
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=1483569671-1462-2-git-send-email-keith.busch@intel.com \
--to=keith.busch@intel.com \
--cc=axboe@fb.com \
--cc=axboe@kernel.dk \
--cc=hch@lst.de \
--cc=linux-block@vger.kernel.org \
--cc=linux-nvme@lists.infradead.org \
--cc=marc@merlins.org \
--cc=tglx@linutronix.de \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.