From: Christoph Hellwig <hch@lst.de>
To: Sagi Grimberg <sagi@grimberg.me>
Cc: Israel Rukshin <israelr@mellanox.com>,
Linux-nvme <linux-nvme-bounces@lists.infradead.org>,
linux-block@vger.kernel.org, Max Gurtovoy <maxg@mellanox.com>,
Christoph Hellwig <hch@lst.de>, Jens Axboe <axboe@kernel.dk>
Subject: Re: [PATCH] blk-mq: Iterate also over sched_tags requests at blk_mq_tagset_iter()
Date: Mon, 23 Oct 2017 08:26:27 +0200 [thread overview]
Message-ID: <20171023062627.GD16899@lst.de> (raw)
In-Reply-To: <2c27d643-a7d9-3077-2875-cea768cbf386@grimberg.me>
On Sun, Oct 22, 2017 at 09:32:00PM +0300, Sagi Grimberg wrote:
>
>> Currently, blk_mq_tagset_iter() iterate over initial hctx tags only.
>> In case scheduler is used, it doesn't iterate the hctx scheduler tags
>> and the static request aren't been updated.
>> For example, while using NVMe over Fabrics RDMA host, this cause us not to
>> reinit the scheduler requests and thus not re-register all the memory regions
>> during the tagset re-initialization in the reconnect flow.
>
> I think this is a sign that we should cease from embedding memory
> regions on the pre-allocated requests. Its too much resources
> that we waste. In our case, tags are not really cheap given
> that they take a physical HW resource (rdma memory region).
>
> I think we should switch (again) to a pool design instead. I guess its
> time for a generic MR pool that will serve nvmf, xprt, srp, iser and
> friends.
Liks drivers/infiniband/core/mr_pool.c? :)
next prev parent reply other threads:[~2017-10-23 6:26 UTC|newest]
Thread overview: 9+ messages / expand[flat|nested] mbox.gz Atom feed top
2017-10-22 13:47 [PATCH] blk-mq: Iterate also over sched_tags requests at blk_mq_tagset_iter() Israel Rukshin
2017-10-22 18:32 ` Sagi Grimberg
2017-10-23 6:26 ` Christoph Hellwig [this message]
2017-10-23 7:27 ` Sagi Grimberg
2017-10-23 7:30 ` Christoph Hellwig
2017-10-23 6:38 ` Bart Van Assche
2017-10-23 7:52 ` Sagi Grimberg
2017-10-23 9:31 ` Max Gurtovoy
2017-10-23 10:00 ` Ming Lei
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20171023062627.GD16899@lst.de \
--to=hch@lst.de \
--cc=axboe@kernel.dk \
--cc=israelr@mellanox.com \
--cc=linux-block@vger.kernel.org \
--cc=linux-nvme-bounces@lists.infradead.org \
--cc=maxg@mellanox.com \
--cc=sagi@grimberg.me \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox