From: Leon Romanovsky <leon@kernel.org>
To: Sagi Grimberg <sagi@grimberg.me>
Cc: Max Gurtovoy <mgurtovoy@nvidia.com>,
Krishnamraju Eraparaju <krishna2@chelsio.com>,
linux-rdma@vger.kernel.org,
Potnuri Bharat Teja <bharat@chelsio.com>,
Max Gurtovoy <maxg@mellanox.com>
Subject: Re: reduce iSERT Max IO size
Date: Fri, 9 Oct 2020 16:07:07 +0300 [thread overview]
Message-ID: <20201009130707.GR13580@unreal> (raw)
In-Reply-To: <5ab4fea9-fefb-d138-cc3b-03f87cd6ee66@grimberg.me>
On Thu, Oct 08, 2020 at 09:20:41AM -0700, Sagi Grimberg wrote:
>
> > > > > I think max IO size, at iSER initiator, depends on
> > > > > "max_fast_reg_page_list_len".
> > > > > currently, below are the supported "max_fast_reg_page_list_len" of
> > > > > various iwarp drivers:
> > > > >
> > > > > iw_cxgb4: 128 pages
> > > > > Softiwarp: 256 pages
> > > > > i40iw: 512 pages
> > > > > qedr: couldn't find.
> > > > >
> > > > > For iwarp case, if 512 is the max pages supported by all iwarp drivers,
> > > > > then provisioning a gigantic MR pool at target(to accommodate never used
> > > > > 16MiB IO) wouldn't be a overkill?
> > > >
> > > > For RoCE/IB Mellanox HCAs we support 16MiB IO size and even more. We
> > > > limited to 16MiB in iSER/iSERT.
> > > >
> > > > Sagi,
> > > >
> > > > what about adding a module parameter for this as we did in iSER initiator ?
> > >
> > > I don't think we have any other choice...
> >
> > Sagi,
> >
> > I didn't read whole thread and know little about ULPs, but wonder if isn't
> > it possible to check device type (iWARP/RoCE) during iSERT initialization
> > and create MR pool only after device is recognized?
>
> Its already done this way. The problem is that there is no handshake
> procedure in iSER between the host and the target for what the maximum
> transfer size would be, so currently isert sets up to a worse case of
> 16MB. This has implications on memory requirements.
Thanks for the explanation.
next prev parent reply other threads:[~2020-10-09 13:07 UTC|newest]
Thread overview: 17+ messages / expand[flat|nested] mbox.gz Atom feed top
2020-09-22 10:44 reduce iSERT Max IO size Krishnamraju Eraparaju
2020-09-23 8:57 ` Sagi Grimberg
2020-10-02 17:10 ` Krishnamraju Eraparaju
2020-10-02 20:29 ` Sagi Grimberg
2020-10-03 3:36 ` Krishnamraju Eraparaju
2020-10-03 13:12 ` Max Gurtovoy
2020-10-03 21:45 ` Max Gurtovoy
2020-10-07 3:36 ` Krishnamraju Eraparaju
2020-10-07 12:56 ` Max Gurtovoy
2020-10-07 23:50 ` Sagi Grimberg
2020-10-08 5:30 ` Leon Romanovsky
2020-10-08 16:20 ` Sagi Grimberg
2020-10-09 13:07 ` Leon Romanovsky [this message]
2020-10-08 13:12 ` Bernard Metzler
2020-10-08 18:59 ` Krishnamraju Eraparaju
2020-10-08 22:47 ` Max Gurtovoy
2020-10-09 3:06 ` Krishnamraju Eraparaju
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20201009130707.GR13580@unreal \
--to=leon@kernel.org \
--cc=bharat@chelsio.com \
--cc=krishna2@chelsio.com \
--cc=linux-rdma@vger.kernel.org \
--cc=maxg@mellanox.com \
--cc=mgurtovoy@nvidia.com \
--cc=sagi@grimberg.me \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).