From: Sagi Grimberg <sagig@dev.mellanox.co.il>
To: Or Gerlitz <ogerlitz@mellanox.com>,
Sagi Grimberg <sagig@mellanox.com>,
linux-rdma@vger.kernel.org, target-devel@vger.kernel.org
Cc: Steve Wise <swise@opengridcomputing.com>,
"Nicholas A. Bellinger" <nab@linux-iscsi.org>,
Doug Ledford <dledford@redhat.com>,
Yishai Hadas <yishaih@mellanox.com>
Subject: Re: [PATCH 1/2] mlx4: Expose correct max_sge_rd limit
Date: Tue, 27 Oct 2015 18:03:43 +0200 [thread overview]
Message-ID: <562FA05F.4050805@dev.mellanox.co.il> (raw)
In-Reply-To: <562F8C8F.8040000@mellanox.com>
On 27/10/2015 16:39, Or Gerlitz wrote:
> On 10/27/2015 11:40 AM, Sagi Grimberg wrote:
>> mlx4 devices (ConnectX-2, ConnectX-3) can not issue
>> max_sge in a single RDMA_READ request (resulting in
>> a completion error). Thus, expose lower max_sge_rd
>> to avoid this issue.
>
> Sagi,
Hey Or,
> Still, this patch is currently pure WA b/c we didn't do RCA (Root Cause
> Analysis)
So from my discussions with the HW folks a RDMA_READ wqe cannot exceed
512B. The wqe control segment is 16 bytes, the rdma section is 12 bytes
(rkey + raddr) and each sge is 16 bytes so the computation is:
(512B-16B-12B)/16B = 30.
The reason is that the HW needs to fetch the rdma_read wqe on the RX
path (rdma_read response) and it has a limited buffer at that point.
Perhaps a dedicated #define for that is needed here.
I'll add that in the change log in v1.
Cheers,
Sagi.
next prev parent reply other threads:[~2015-10-27 16:03 UTC|newest]
Thread overview: 11+ messages / expand[flat|nested] mbox.gz Atom feed top
2015-10-27 9:40 [PATCH 0/2] Expose max_sge_rd correctly Sagi Grimberg
2015-10-27 9:40 ` [PATCH 1/2] mlx4: Expose correct max_sge_rd limit Sagi Grimberg
[not found] ` <1445938846-9240-2-git-send-email-sagig-VPRAkNaXOzVWk0Htik3J/w@public.gmane.org>
2015-10-27 14:39 ` Or Gerlitz
2015-10-27 16:03 ` Sagi Grimberg [this message]
[not found] ` <562FA05F.4050805-LDSdmyG8hGV8YrgS2mwiifqBs+8SCbDb@public.gmane.org>
2015-10-27 16:12 ` Or Gerlitz
[not found] ` <562FA273.5090700-VPRAkNaXOzVWk0Htik3J/w@public.gmane.org>
2015-10-27 16:17 ` Sagi Grimberg
2015-10-27 16:24 ` Bart Van Assche
[not found] ` <562FA544.4040508-XdAiOPVOjttBDgjK7y7TUQ@public.gmane.org>
2015-10-27 16:31 ` Sagi Grimberg
[not found] ` <562FA6E8.4040005-LDSdmyG8hGV8YrgS2mwiifqBs+8SCbDb@public.gmane.org>
2015-10-27 17:37 ` Eli Cohen
2015-10-27 9:40 ` [PATCH 2/2] iser-target: Remove explicit mlx4 work-around Sagi Grimberg
[not found] ` <1445938846-9240-1-git-send-email-sagig-VPRAkNaXOzVWk0Htik3J/w@public.gmane.org>
2015-10-27 14:26 ` [PATCH 0/2] Expose max_sge_rd correctly Steve Wise
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=562FA05F.4050805@dev.mellanox.co.il \
--to=sagig@dev.mellanox.co.il \
--cc=dledford@redhat.com \
--cc=linux-rdma@vger.kernel.org \
--cc=nab@linux-iscsi.org \
--cc=ogerlitz@mellanox.com \
--cc=sagig@mellanox.com \
--cc=swise@opengridcomputing.com \
--cc=target-devel@vger.kernel.org \
--cc=yishaih@mellanox.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.