From: Bart Van Assche <Bart.VanAssche@sandisk.com>
To: "osandov@osandov.com" <osandov@osandov.com>
Cc: "linux-kernel@vger.kernel.org" <linux-kernel@vger.kernel.org>,
"linux-block@vger.kernel.org" <linux-block@vger.kernel.org>,
"osandov@fb.com" <osandov@fb.com>, "axboe@fb.com" <axboe@fb.com>,
"axboe@kernel.dk" <axboe@kernel.dk>,
"paolo.valente@linaro.org" <paolo.valente@linaro.org>
Subject: Re: [PATCHSET v4] blk-mq-scheduling framework
Date: Thu, 22 Dec 2016 17:39:56 +0000 [thread overview]
Message-ID: <1482428377.2682.13.camel@sandisk.com> (raw)
In-Reply-To: <20161222171224.GA26830@vader>
T24gVGh1LCAyMDE2LTEyLTIyIGF0IDA5OjEyIC0wODAwLCBPbWFyIFNhbmRvdmFsIHdyb3RlOg0K
PiBPbiBUaHUsIERlYyAyMiwgMjAxNiBhdCAwNDo1NzozNlBNICswMDAwLCBCYXJ0IFZhbiBBc3Nj
aGUgd3JvdGU6DQo+ID4gT24gVGh1LCAyMDE2LTEyLTIyIGF0IDA4OjUyIC0wODAwLCBPbWFyIFNh
bmRvdmFsIHdyb3RlOg0KPiA+ID4gVGhpcyBhcHByb2FjaCBvY2N1cnJlZCB0byB1cywgYnV0IHdl
IGNvdWxkbid0IGZpZ3VyZSBvdXQgYSB3YXkgdG8gbWFrZQ0KPiA+ID4gYmxrX21xX3RhZ190b19y
cSgpIHdvcmsgd2l0aCBpdC4gRnJvbSBza2ltbWluZyBvdmVyIHRoZSBwYXRjaGVzLCBJDQo+ID4g
PiBkaWRuJ3Qgc2VlIGEgc29sdXRpb24gdG8gdGhhdCBwcm9ibGVtLg0KPiA+IA0KPiA+IENhbiB5
b3UgY2xhcmlmeSB5b3VyIGNvbW1lbnQ/IFNpbmNlIG15IHBhdGNoZXMgaW5pdGlhbGl6ZSBib3Ro
IHRhZ3MtPnJxc1tdDQo+ID4gYW5kIHNjaGVkX3RhZ3MtPnJxc1tdIHRoZSBmdW5jdGlvbiBibGtf
bXFfdGFnX3RvX3JxKCkgc2hvdWxkIHN0aWxsIHdvcmsuDQo+IA0KPiBTb3JyeSwgeW91J3JlIHJp
Z2h0LCBpdCBkb2VzIHdvcmssIGJ1dCB0YWdzLT5ycXNbXSBlbmRzIHVwIGJlaW5nIHRoZQ0KPiBl
eHRyYSBsb29rdXAgdGFibGUuIEkgc3VzcGVjdCB0aGF0IHRoZSBydW50aW1lIG92ZXJoZWFkIG9m
IGtlZXBpbmcgdGhhdA0KPiB1cCB0byBkYXRlIGNvdWxkIGJlIHdvcnNlIHRoYW4gY29weWluZyB0
aGUgcnEgZmllbGRzIGlmIHlvdSBoYXZlIGxvdHMgb2YNCj4gQ1BVcyBidXQgb25seSBvbmUgaGFy
ZHdhcmUgcXVldWUuDQoNCkhlbGxvIE9tYXIsDQoNCkknbSBub3Qgc3VyZSB0aGF0IGFueXRoaW5n
IGNhbiBiZSBkb25lIGlmIHRoZSBudW1iZXIgb2YgQ1BVcyB0aGF0IGlzIHN1Ym1pdHRpbmcNCkkv
TyBpcyBsYXJnZSBjb21wYXJlZCB0byB0aGUgcXVldWUgZGVwdGggc28gSSBkb24ndCB0aGluayB3
ZSBzaG91bGQgc3BlbmQgb3VyDQp0aW1lIG9uIHRoYXQgY2FzZS4gSWYgdGhlIHF1ZXVlIGRlcHRo
IGlzIGxhcmdlIGVub3VnaCB0aGVuIHRoZSBzYml0bWFwIGNvZGUgd2lsbA0KYWxsb2NhdGUgdGFn
cyBzdWNoIHRoYXQgZGlmZmVyZW50IENQVXMgdXNlIGRpZmZlcmVudCBycXNbXSBlbGVtZW50cy4N
Cg0KVGhlIGFkdmFudGFnZXMgb2YgdGhlIGFwcHJvYWNoIEkgcHJvcG9zZWQgYXJlIHN1Y2ggdGhh
dCBJIGFtIGNvbnZpbmNlZCB0aGF0IGlzDQp3aGF0IHdlIHNob3VsZCBzdGFydCBmcm9tIGFuZCBh
ZGRyZXNzIGNvbnRlbnRpb24gb24gdGhlIHRhZ3MtPnJxc1tdIGFycmF5IGlmIGl0DQptZWFzdXJl
bWVudHMgc2hvdyB0aGF0IGl0IGlzIG5lY2Vzc2FyeSB0byBhZGRyZXNzIGl0Lg0KDQpCYXJ0Lg==
WARNING: multiple messages have this Message-ID (diff)
From: Bart Van Assche <Bart.VanAssche@sandisk.com>
To: "osandov@osandov.com" <osandov@osandov.com>
Cc: "linux-kernel@vger.kernel.org" <linux-kernel@vger.kernel.org>,
"linux-block@vger.kernel.org" <linux-block@vger.kernel.org>,
"osandov@fb.com" <osandov@fb.com>, "axboe@fb.com" <axboe@fb.com>,
"axboe@kernel.dk" <axboe@kernel.dk>,
"paolo.valente@linaro.org" <paolo.valente@linaro.org>
Subject: Re: [PATCHSET v4] blk-mq-scheduling framework
Date: Thu, 22 Dec 2016 17:39:56 +0000 [thread overview]
Message-ID: <1482428377.2682.13.camel@sandisk.com> (raw)
In-Reply-To: <20161222171224.GA26830@vader>
On Thu, 2016-12-22 at 09:12 -0800, Omar Sandoval wrote:
> On Thu, Dec 22, 2016 at 04:57:36PM +0000, Bart Van Assche wrote:
> > On Thu, 2016-12-22 at 08:52 -0800, Omar Sandoval wrote:
> > > This approach occurred to us, but we couldn't figure out a way to make
> > > blk_mq_tag_to_rq() work with it. From skimming over the patches, I
> > > didn't see a solution to that problem.
> >
> > Can you clarify your comment? Since my patches initialize both tags->rqs[]
> > and sched_tags->rqs[] the function blk_mq_tag_to_rq() should still work.
>
> Sorry, you're right, it does work, but tags->rqs[] ends up being the
> extra lookup table. I suspect that the runtime overhead of keeping that
> up to date could be worse than copying the rq fields if you have lots of
> CPUs but only one hardware queue.
Hello Omar,
I'm not sure that anything can be done if the number of CPUs that is submitting
I/O is large compared to the queue depth so I don't think we should spend our
time on that case. If the queue depth is large enough then the sbitmap code will
allocate tags such that different CPUs use different rqs[] elements.
The advantages of the approach I proposed are such that I am convinced that is
what we should start from and address contention on the tags->rqs[] array if it
measurements show that it is necessary to address it.
Bart.
next prev parent reply other threads:[~2016-12-22 17:39 UTC|newest]
Thread overview: 115+ messages / expand[flat|nested] mbox.gz Atom feed top
2016-12-17 0:12 [PATCHSET v4] blk-mq-scheduling framework Jens Axboe
2016-12-17 0:12 ` [PATCH 1/8] block: move existing elevator ops to union Jens Axboe
2016-12-17 0:12 ` [PATCH 2/8] blk-mq: make mq_ops a const pointer Jens Axboe
2016-12-17 0:12 ` [PATCH 3/8] block: move rq_ioc() to blk.h Jens Axboe
2016-12-20 10:12 ` Paolo Valente
2016-12-20 10:12 ` Paolo Valente
2016-12-20 15:46 ` Jens Axboe
2016-12-20 22:14 ` Jens Axboe
2016-12-17 0:12 ` [PATCH 4/8] blk-mq: un-export blk_mq_free_hctx_request() Jens Axboe
2016-12-17 0:12 ` [PATCH 5/8] blk-mq: export some helpers we need to the scheduling framework Jens Axboe
2016-12-17 0:12 ` [PATCH 6/8] blk-mq-sched: add framework for MQ capable IO schedulers Jens Axboe
2016-12-20 11:55 ` Paolo Valente
2016-12-20 11:55 ` Paolo Valente
2016-12-20 15:45 ` Jens Axboe
2016-12-21 2:22 ` Jens Axboe
2016-12-22 15:20 ` Paolo Valente
2016-12-22 15:20 ` Paolo Valente
2016-12-22 9:59 ` Paolo Valente
2016-12-22 9:59 ` Paolo Valente
2016-12-22 11:13 ` Paolo Valente
2016-12-22 11:13 ` Paolo Valente
2017-01-17 2:47 ` Jens Axboe
2017-01-17 2:47 ` Jens Axboe
2017-01-17 10:13 ` Paolo Valente
2017-01-17 10:13 ` Paolo Valente
2017-01-17 12:38 ` Jens Axboe
2017-01-17 12:38 ` Jens Axboe
2016-12-23 10:12 ` Paolo Valente
2016-12-23 10:12 ` Paolo Valente
2017-01-17 2:47 ` Jens Axboe
2017-01-17 9:17 ` Paolo Valente
2017-01-17 9:17 ` Paolo Valente
2016-12-17 0:12 ` [PATCH 7/8] mq-deadline: add blk-mq adaptation of the deadline IO scheduler Jens Axboe
2016-12-20 9:34 ` Paolo Valente
2016-12-20 9:34 ` Paolo Valente
2016-12-20 15:46 ` Jens Axboe
2016-12-21 11:59 ` Bart Van Assche
2016-12-21 11:59 ` Bart Van Assche
2016-12-21 14:22 ` Jens Axboe
2016-12-22 16:07 ` Paolo Valente
2016-12-22 16:07 ` Paolo Valente
2017-01-17 2:47 ` Jens Axboe
2017-01-17 2:47 ` Jens Axboe
2016-12-22 16:49 ` Paolo Valente
2016-12-22 16:49 ` Paolo Valente
2017-01-17 2:47 ` Jens Axboe
2017-01-20 11:07 ` Paolo Valente
2017-01-20 11:07 ` Paolo Valente
2017-01-20 14:26 ` Jens Axboe
2017-01-20 13:14 ` Paolo Valente
2017-01-20 13:14 ` Paolo Valente
2017-01-20 13:18 ` Paolo Valente
2017-01-20 13:18 ` Paolo Valente
2017-01-20 14:28 ` Jens Axboe
2017-01-20 14:28 ` Jens Axboe
2017-02-01 11:11 ` Paolo Valente
2017-02-01 11:11 ` Paolo Valente
2017-02-02 5:19 ` Jens Axboe
2017-02-02 9:19 ` Paolo Valente
2017-02-02 9:19 ` Paolo Valente
2017-02-02 15:30 ` Jens Axboe
2017-02-02 15:30 ` Jens Axboe
2017-02-02 21:15 ` Paolo Valente
2017-02-02 21:15 ` Paolo Valente
2017-02-02 21:32 ` Jens Axboe
2017-02-02 21:32 ` Jens Axboe
2017-02-07 17:27 ` Paolo Valente
2017-02-07 17:27 ` Paolo Valente
2017-02-01 11:56 ` Paolo Valente
2017-02-01 11:56 ` Paolo Valente
2017-02-02 5:20 ` Jens Axboe
2017-02-16 10:46 ` Paolo Valente
2017-02-16 10:46 ` Paolo Valente
2017-02-16 15:35 ` Jens Axboe
2017-02-16 15:35 ` Jens Axboe
2016-12-17 0:12 ` [PATCH 8/8] blk-mq-sched: allow setting of default " Jens Axboe
2016-12-19 11:32 ` [PATCHSET v4] blk-mq-scheduling framework Paolo Valente
2016-12-19 11:32 ` Paolo Valente
2016-12-19 15:20 ` Jens Axboe
2016-12-19 15:20 ` Jens Axboe
2016-12-19 15:33 ` Jens Axboe
2016-12-19 18:21 ` Paolo Valente
2016-12-19 18:21 ` Paolo Valente
2016-12-19 21:05 ` Jens Axboe
2016-12-19 21:05 ` Jens Axboe
2016-12-22 15:28 ` Paolo Valente
2016-12-22 15:28 ` Paolo Valente
2017-01-17 2:47 ` Jens Axboe
2017-01-17 10:43 ` Paolo Valente
2017-01-17 10:44 ` Paolo Valente
2017-01-17 10:47 ` Paolo Valente
2017-01-17 10:49 ` Paolo Valente
2017-01-17 10:49 ` Paolo Valente
2017-01-18 16:14 ` Paolo Valente
2017-01-18 16:14 ` Paolo Valente
2017-01-18 16:21 ` Jens Axboe
2017-01-18 16:21 ` Jens Axboe
2017-01-23 17:04 ` Paolo Valente
2017-01-23 17:04 ` Paolo Valente
2017-01-23 17:42 ` Jens Axboe
2017-01-23 17:42 ` Jens Axboe
2017-01-25 8:46 ` Paolo Valente
2017-01-25 8:46 ` Paolo Valente
2017-01-25 16:13 ` Jens Axboe
2017-01-25 16:13 ` Jens Axboe
2017-01-26 14:23 ` Paolo Valente
2017-01-26 14:23 ` Paolo Valente
2016-12-22 16:23 ` Bart Van Assche
2016-12-22 16:52 ` Omar Sandoval
2016-12-22 16:52 ` Omar Sandoval
2016-12-22 16:57 ` Bart Van Assche
2016-12-22 16:57 ` Bart Van Assche
2016-12-22 17:12 ` Omar Sandoval
2016-12-22 17:39 ` Bart Van Assche [this message]
2016-12-22 17:39 ` Bart Van Assche
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=1482428377.2682.13.camel@sandisk.com \
--to=bart.vanassche@sandisk.com \
--cc=axboe@fb.com \
--cc=axboe@kernel.dk \
--cc=linux-block@vger.kernel.org \
--cc=linux-kernel@vger.kernel.org \
--cc=osandov@fb.com \
--cc=osandov@osandov.com \
--cc=paolo.valente@linaro.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.