From: Bart Van Assche <bvanassche@acm.org>
To: Sagi Grimberg <sagig@dev.mellanox.co.il>,
Christoph Hellwig <hch@infradead.org>
Cc: Jens Axboe <axboe@kernel.dk>, Sagi Grimberg <sagig@mellanox.com>,
Sebastian Parschauer <sebastian.riemer@profitbricks.com>,
Robert Elliott <Elliott@hp.com>,
Ming Lei <ming.lei@canonical.com>,
"linux-scsi@vger.kernel.org" <linux-scsi@vger.kernel.org>,
linux-rdma <linux-rdma@vger.kernel.org>
Subject: Re: [PATCH v2 12/12] IB/srp: Add multichannel support
Date: Thu, 30 Oct 2014 15:36:56 +0100 [thread overview]
Message-ID: <54524D08.4040203@acm.org> (raw)
In-Reply-To: <545248F8.8020102@dev.mellanox.co.il>
On 10/30/14 15:19, Sagi Grimberg wrote:
> My impression here that in the default settings, on a 1 NUMA node with
> 8 cores, 2 different srp connections (using 4 channels each) will be
> associated with comp vectors 0-3. while it could potentially use
> vectors 4-7 and reduce possible mutual interference. right?
Hello Sagi,
That's correct. For this example if use of all completion vectors is
desired additional configuration is required, e.g. by setting ch_count
to 8 in /etc/modprobe.d/ib_srp.conf. By the way, I'm not sure it is
possible to avoid manual configuration and tuning entirely. As an
example, with a six core CPU at the initiator side and with
hyperthreading enabled (12 CPU threads in total) I see higher IOPS
results with ch_count=6 compared to ch_count=8 or ch_count=12. I have
not tried to determine why but maybe this is because ch_count values
that are below the number of CPU threads cause some interrupt coalescing.
Bart.
next prev parent reply other threads:[~2014-10-30 14:37 UTC|newest]
Thread overview: 82+ messages / expand[flat|nested] mbox.gz Atom feed top
2014-10-07 13:01 [PATCH v2 0/12] IB/srp: Add multichannel support Bart Van Assche
2014-10-07 13:03 ` [PATCH v2 02/12] blk-mq: Add blk_mq_unique_tag() Bart Van Assche
2014-10-11 11:08 ` Christoph Hellwig
2014-10-13 9:21 ` Bart Van Assche
[not found] ` <543B99B2.1010307-HInyCGIudOg@public.gmane.org>
2014-10-13 10:15 ` Christoph Hellwig
2014-10-19 16:14 ` Sagi Grimberg
[not found] ` <5433E493.9030304-HInyCGIudOg@public.gmane.org>
2014-10-28 1:55 ` Martin K. Petersen
2014-10-07 13:04 ` [PATCH v2 04/12] scsi_tcq.h: Add support for multiple hardware queues Bart Van Assche
2014-10-19 16:12 ` Sagi Grimberg
[not found] ` <5443E2DF.1040605-LDSdmyG8hGV8YrgS2mwiifqBs+8SCbDb@public.gmane.org>
2014-10-20 12:01 ` Bart Van Assche
[not found] ` <5444F995.5080407-HInyCGIudOg@public.gmane.org>
2014-10-21 8:49 ` Christoph Hellwig
2014-10-21 8:59 ` Sagi Grimberg
2014-10-28 2:06 ` Martin K. Petersen
[not found] ` <5433E43D.3010107-HInyCGIudOg@public.gmane.org>
2014-10-07 13:02 ` [PATCH v2 01/12] blk-mq: Use all available " Bart Van Assche
2014-10-07 14:37 ` Jens Axboe
[not found] ` <5433FA8F.3050100-tSWWG44O7X1aa/9Udqfwiw@public.gmane.org>
2014-10-08 13:21 ` Bart Van Assche
[not found] ` <54353A74.7040406-HInyCGIudOg@public.gmane.org>
2014-10-11 11:11 ` Christoph Hellwig
[not found] ` <20141011111114.GB9593-wEGCiKHe2LqWVfeAwA7xHQ@public.gmane.org>
2014-10-13 9:45 ` Bart Van Assche
[not found] ` <543B9F47.2090204-HInyCGIudOg@public.gmane.org>
2014-10-17 13:20 ` Christoph Hellwig
[not found] ` <20141017132053.GF16538-wEGCiKHe2LqWVfeAwA7xHQ@public.gmane.org>
2014-10-17 14:11 ` Sagi Grimberg
2014-10-07 13:03 ` [PATCH v2 03/12] scsi-mq: Add support for multiple " Bart Van Assche
[not found] ` <5433E4AB.8030306-HInyCGIudOg@public.gmane.org>
2014-10-19 15:54 ` Sagi Grimberg
2014-10-28 2:01 ` Martin K. Petersen
2014-10-29 12:22 ` Bart Van Assche
2014-10-29 12:27 ` Bart Van Assche
[not found] ` <5450DD49.6090108-HInyCGIudOg@public.gmane.org>
2014-10-30 0:53 ` Martin K. Petersen
2014-10-07 13:04 ` [PATCH v2 05/12] IB/srp: Move ib_destroy_cm_id() call into srp_free_ch_ib() Bart Van Assche
2014-10-07 13:04 ` [PATCH v2 06/12] IB/srp: Remove stale connection retry mechanism Bart Van Assche
2014-10-07 13:05 ` [PATCH v2 09/12] IB/srp: Separate target and channel variables Bart Van Assche
2014-10-19 16:48 ` Sagi Grimberg
2014-10-07 13:06 ` [PATCH v2 11/12] IB/srp: Eliminate free_reqs list Bart Van Assche
[not found] ` <5433E56E.6010600-HInyCGIudOg@public.gmane.org>
2014-10-17 10:59 ` Christoph Hellwig
[not found] ` <20141017105939.GB7819-wEGCiKHe2LqWVfeAwA7xHQ@public.gmane.org>
2014-10-19 16:59 ` Sagi Grimberg
2014-10-20 11:47 ` Bart Van Assche
2014-10-21 8:49 ` Christoph Hellwig
2014-10-07 13:05 ` [PATCH v2 07/12] IB/srp: Avoid that I/O hangs due to a cable pull during LUN scanning Bart Van Assche
2014-10-19 16:27 ` Sagi Grimberg
[not found] ` <5443E66F.7050901-LDSdmyG8hGV8YrgS2mwiifqBs+8SCbDb@public.gmane.org>
2014-10-20 12:15 ` Bart Van Assche
2014-10-21 8:50 ` Christoph Hellwig
2014-10-07 13:05 ` [PATCH v2 08/12] IB/srp: Introduce two new srp_target_port member variables Bart Van Assche
2014-10-19 16:30 ` Sagi Grimberg
2014-10-07 13:06 ` [PATCH v2 10/12] IB/srp: Use block layer tags Bart Van Assche
[not found] ` <5433E557.3010505-HInyCGIudOg@public.gmane.org>
2014-10-17 10:58 ` Christoph Hellwig
[not found] ` <20141017105858.GA7819-wEGCiKHe2LqWVfeAwA7xHQ@public.gmane.org>
2014-10-20 11:44 ` Bart Van Assche
2014-10-22 22:03 ` Elliott, Robert (Server Storage)
[not found] ` <94D0CD8314A33A4D9D801C0FE68B4029593212E0-wwDBVnaDRpYSZAcGdq5asR6epYMZPwEe5NbjCUgZEJk@public.gmane.org>
2014-10-23 7:16 ` Bart Van Assche
2014-10-23 17:43 ` Webb Scales
[not found] ` <54493E5A.7050803-VXdhtT5mjnY@public.gmane.org>
2014-10-24 6:45 ` Bart Van Assche
[not found] ` <5449F571.7080308-HInyCGIudOg@public.gmane.org>
2014-10-24 15:40 ` Webb Scales
2014-10-23 8:47 ` Christoph Hellwig
2014-10-24 4:43 ` Elliott, Robert (Server Storage)
2014-10-24 6:45 ` Christoph Hellwig
[not found] ` <20141024064514.GA15654-wEGCiKHe2LqWVfeAwA7xHQ@public.gmane.org>
2014-10-31 17:34 ` Hannes Reinecke
2014-11-03 7:52 ` Kashyap Desai
2014-11-03 8:25 ` Christoph Hellwig
2014-10-07 13:07 ` [PATCH v2 12/12] IB/srp: Add multichannel support Bart Van Assche
2014-10-17 11:01 ` EH action after scsi_remove_host, was: " Christoph Hellwig
2014-10-20 13:53 ` Bart Van Assche
2014-10-21 8:51 ` Christoph Hellwig
2014-10-17 11:06 ` Christoph Hellwig
[not found] ` <20141017110627.GD7819-wEGCiKHe2LqWVfeAwA7xHQ@public.gmane.org>
2014-10-20 11:57 ` Bart Van Assche
2014-10-21 8:49 ` Christoph Hellwig
[not found] ` <5433E585.607-HInyCGIudOg@public.gmane.org>
2014-10-19 17:36 ` Sagi Grimberg
2014-10-20 12:56 ` Bart Van Assche
[not found] ` <54450690.709-HInyCGIudOg@public.gmane.org>
2014-10-21 9:10 ` Sagi Grimberg
[not found] ` <544622FE.5040906-LDSdmyG8hGV8YrgS2mwiifqBs+8SCbDb@public.gmane.org>
2014-10-28 18:32 ` Sagi Grimberg
[not found] ` <544FE13A.60807-LDSdmyG8hGV8YrgS2mwiifqBs+8SCbDb@public.gmane.org>
2014-10-29 10:52 ` Bart Van Assche
2014-10-30 14:19 ` Sagi Grimberg
2014-10-30 14:36 ` Bart Van Assche [this message]
[not found] ` <54524D08.4040203-HInyCGIudOg@public.gmane.org>
2014-10-30 15:06 ` Sagi Grimberg
[not found] ` <545253E3.7000009-LDSdmyG8hGV8YrgS2mwiifqBs+8SCbDb@public.gmane.org>
2014-10-30 15:19 ` Bart Van Assche
[not found] ` <545256E5.9010501-HInyCGIudOg@public.gmane.org>
2014-10-30 17:33 ` Sagi Grimberg
2014-10-31 9:19 ` Bart Van Assche
[not found] ` <5453541D.7040206-HInyCGIudOg@public.gmane.org>
2014-11-02 13:03 ` Sagi Grimberg
2014-11-03 1:46 ` Elliott, Robert (Server Storage)
2014-11-04 11:46 ` Bart Van Assche
[not found] ` <5458BC8B.40202-HInyCGIudOg@public.gmane.org>
2014-11-04 12:15 ` Sagi Grimberg
[not found] ` <5458C344.2040109-LDSdmyG8hGV8YrgS2mwiifqBs+8SCbDb@public.gmane.org>
2014-11-05 4:57 ` Elliott, Robert (Server Storage)
[not found] ` <94D0CD8314A33A4D9D801C0FE68B40295937104F-2m9nI20wMFwSZAcGdq5asR6epYMZPwEe5NbjCUgZEJk@public.gmane.org>
2014-11-05 11:22 ` Sagi Grimberg
2014-10-21 9:14 ` Sagi Grimberg
2014-10-29 12:36 ` Bart Van Assche
2014-10-30 14:22 ` Sagi Grimberg
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=54524D08.4040203@acm.org \
--to=bvanassche@acm.org \
--cc=Elliott@hp.com \
--cc=axboe@kernel.dk \
--cc=hch@infradead.org \
--cc=linux-rdma@vger.kernel.org \
--cc=linux-scsi@vger.kernel.org \
--cc=ming.lei@canonical.com \
--cc=sagig@dev.mellanox.co.il \
--cc=sagig@mellanox.com \
--cc=sebastian.riemer@profitbricks.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).