From: Leon Romanovsky <leon@kernel.org>
To: Sagi Grimberg <sagi@grimberg.me>
Cc: Max Gurtovoy <mgurtovoy@nvidia.com>,
Krishnamraju Eraparaju <krishna2@chelsio.com>,
linux-rdma@vger.kernel.org,
Potnuri Bharat Teja <bharat@chelsio.com>,
Max Gurtovoy <maxg@mellanox.com>
Subject: Re: reduce iSERT Max IO size
Date: Fri, 9 Oct 2020 16:07:07 +0300 [thread overview]
Message-ID: <20201009130707.GR13580@unreal> (raw)
In-Reply-To: <5ab4fea9-fefb-d138-cc3b-03f87cd6ee66@grimberg.me>
On Thu, Oct 08, 2020 at 09:20:41AM -0700, Sagi Grimberg wrote:
>
> > > > > I think max IO size, at iSER initiator, depends on
> > > > > "max_fast_reg_page_list_len".
> > > > > currently, below are the supported "max_fast_reg_page_list_len" of
> > > > > various iwarp drivers:
> > > > >
> > > > > iw_cxgb4: 128 pages
> > > > > Softiwarp: 256 pages
> > > > > i40iw: 512 pages
> > > > > qedr: couldn't find.
> > > > >
> > > > > For iwarp case, if 512 is the max pages supported by all iwarp drivers,
> > > > > then provisioning a gigantic MR pool at target(to accommodate never used
> > > > > 16MiB IO) wouldn't be a overkill?
> > > >
> > > > For RoCE/IB Mellanox HCAs we support 16MiB IO size and even more. We
> > > > limited to 16MiB in iSER/iSERT.
> > > >
> > > > Sagi,
> > > >
> > > > what about adding a module parameter for this as we did in iSER initiator ?
> > >
> > > I don't think we have any other choice...
> >
> > Sagi,
> >
> > I didn't read whole thread and know little about ULPs, but wonder if isn't
> > it possible to check device type (iWARP/RoCE) during iSERT initialization
> > and create MR pool only after device is recognized?
>
> Its already done this way. The problem is that there is no handshake
> procedure in iSER between the host and the target for what the maximum
> transfer size would be, so currently isert sets up to a worse case of
> 16MB. This has implications on memory requirements.
Thanks for the explanation.
next prev parent reply other threads:[~2020-10-09 13:07 UTC|newest]
Thread overview: 17+ messages / expand[flat|nested] mbox.gz Atom feed top
2020-09-22 10:44 reduce iSERT Max IO size Krishnamraju Eraparaju
2020-09-23 8:57 ` Sagi Grimberg
2020-10-02 17:10 ` Krishnamraju Eraparaju
2020-10-02 20:29 ` Sagi Grimberg
2020-10-03 3:36 ` Krishnamraju Eraparaju
2020-10-03 13:12 ` Max Gurtovoy
2020-10-03 21:45 ` Max Gurtovoy
2020-10-07 3:36 ` Krishnamraju Eraparaju
2020-10-07 12:56 ` Max Gurtovoy
2020-10-07 23:50 ` Sagi Grimberg
2020-10-08 5:30 ` Leon Romanovsky
2020-10-08 16:20 ` Sagi Grimberg
2020-10-09 13:07 ` Leon Romanovsky [this message]
2020-10-08 13:12 ` Bernard Metzler
2020-10-08 18:59 ` Krishnamraju Eraparaju
2020-10-08 22:47 ` Max Gurtovoy
2020-10-09 3:06 ` Krishnamraju Eraparaju
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20201009130707.GR13580@unreal \
--to=leon@kernel.org \
--cc=bharat@chelsio.com \
--cc=krishna2@chelsio.com \
--cc=linux-rdma@vger.kernel.org \
--cc=maxg@mellanox.com \
--cc=mgurtovoy@nvidia.com \
--cc=sagi@grimberg.me \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.