linux-rdma.vger.kernel.org archive mirror
 help / color / mirror / Atom feed
From: Steve Wise <swise-7bPotxP6k4+P2YhJcF5u+vpXobYPEAuW@public.gmane.org>
To: Sagi Grimberg
	<sagig-LDSdmyG8hGV8YrgS2mwiifqBs+8SCbDb@public.gmane.org>,
	Christoph Hellwig <hch-wEGCiKHe2LqWVfeAwA7xHQ@public.gmane.org>
Cc: dledford-H+wXaHxf7aLQT0dZR+AlfA@public.gmane.org,
	infinipath-ral2JQCrhuEAvxtiuMwx3w@public.gmane.org,
	sagig-VPRAkNaXOzVWk0Htik3J/w@public.gmane.org,
	ogerlitz-VPRAkNaXOzVWk0Htik3J/w@public.gmane.org,
	roid-VPRAkNaXOzVWk0Htik3J/w@public.gmane.org,
	linux-rdma-u79uwXL29TY76Z2rM5mHXA@public.gmane.org,
	eli-VPRAkNaXOzVWk0Htik3J/w@public.gmane.org,
	target-devel-u79uwXL29TY76Z2rM5mHXA@public.gmane.org,
	linux-nfs-u79uwXL29TY76Z2rM5mHXA@public.gmane.org,
	bfields-uC3wQj2KruNg9hUCZPvPmw@public.gmane.org
Subject: Re: [PATCH V6 6/9] isert: Rename IO functions to more descriptive names
Date: Sun, 26 Jul 2015 12:32:54 -0500	[thread overview]
Message-ID: <55B519C6.2030205@opengridcomputing.com> (raw)
In-Reply-To: <55B4BDE3.8040801-LDSdmyG8hGV8YrgS2mwiifqBs+8SCbDb@public.gmane.org>

On 7/26/2015 6:00 AM, Sagi Grimberg wrote:
> On 7/26/2015 1:43 PM, Christoph Hellwig wrote:
>> On Sun, Jul 26, 2015 at 01:08:16PM +0300, Sagi Grimberg wrote:
>>> I've given this some thought and I think we should avoid splitting
>>> logic from PI and iWARP. The reason (other than code duplication) is
>>> that currently the iser target support only up to 1MB IOs. I have some
>>> code (not done yet) to support larger IOs by using multiple
>>> registrations  per IO (with or without PI).
>>
>> Just curious: How is this going to work with iSER only having a single
>> rkey/offset/len field?
>>
>
> Good question,
>
> On the wire iser sends a single rkey, but the target is allowed to
> transfer the data however it wants to.
>
> Say that the local target HCA supports only 32 pages (128K bytes for 4K
> pages) registration and the initiator sent:
> rkey=0x1234
> address=0xffffaaaa
> length=512K
>
> The target would allocate a 512K buffer and:
> register offset 0-128K to lkey=0x1
> register offset 128K-256K to lkey=0x2
> register offset 256K-384K to lkey=0x3
> register offset 384K-512K to lkey=0x4
>
> then constructs sg_list as:
> sg_list[0] = {addr=buf, length=128K, lkey=0x1}
> sg_list[1] = {addr=buf+128K, length=128K, lkey=0x2}
> sg_list[2] = {addr=buf+256K, length=128K, lkey=0x3}
> sg_list[3] = {addr=buf+384K, length=128K, lkey=0x4}
>
> Then set rdma_read wr with:
> rdma_r_wr.sg_list=&sg_list
> rdma_r_wr.rdma.addr=0xffffaaaa
> rdma_r_wr.rdma.rkey=0x1234
>
> post_send(rdma_r_wr);
>
> Ideally, the post contains a chain of all 4 registrations and the
> rdma_read (and an opportunistic good scsi response).

Just to be clear: This example is for IB only, correct?  IW would 
require rkeys with REMOTE_WRITE and 4 read wrs.  And you're ignoring 
invalidation wrs (or read-with-inv) in the example...

--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

  parent reply	other threads:[~2015-07-26 17:32 UTC|newest]

Thread overview: 45+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2015-07-24 16:18 [PATCH V6 0/9] iSER support for iWARP Steve Wise
     [not found] ` <20150724161331.25617.8475.stgit-PBeJgSbIpn97NCTnQtmixQ@public.gmane.org>
2015-07-24 16:18   ` [PATCH V6 1/9] RDMA/iser: Limit sg tablesize and max_sectors to device fastreg max depth Steve Wise
     [not found]     ` <20150724161820.25617.63886.stgit-PBeJgSbIpn97NCTnQtmixQ@public.gmane.org>
2015-07-24 16:41       ` Jason Gunthorpe
2015-07-24 18:40         ` Steve Wise
2015-07-24 19:14           ` Jason Gunthorpe
2015-07-26  9:58             ` Sagi Grimberg
2015-07-26  9:57           ` Sagi Grimberg
2015-07-24 16:18   ` [PATCH V6 2/9] mlx4, mlx5, mthca: Expose max_sge_rd correctly Steve Wise
2015-07-24 16:18   ` [PATCH V6 3/9] ipath,qib: " Steve Wise
2015-07-24 16:18   ` [PATCH V6 4/9] svcrdma: Use max_sge_rd for destination read depths Steve Wise
     [not found]     ` <20150724161837.25617.48584.stgit-PBeJgSbIpn97NCTnQtmixQ@public.gmane.org>
2015-07-24 18:51       ` Steve Wise
2015-07-26  9:58         ` Sagi Grimberg
     [not found]           ` <55B4AF63.3040108-LDSdmyG8hGV8YrgS2mwiifqBs+8SCbDb@public.gmane.org>
2015-07-26 10:42             ` Christoph Hellwig
2015-07-24 16:18   ` [PATCH V6 6/9] isert: Rename IO functions to more descriptive names Steve Wise
     [not found]     ` <20150724161848.25617.26092.stgit-PBeJgSbIpn97NCTnQtmixQ@public.gmane.org>
2015-07-26 10:08       ` Sagi Grimberg
     [not found]         ` <55B4B190.7070305-LDSdmyG8hGV8YrgS2mwiifqBs+8SCbDb@public.gmane.org>
2015-07-26 10:43           ` Christoph Hellwig
     [not found]             ` <20150726104328.GB18944-wEGCiKHe2LqWVfeAwA7xHQ@public.gmane.org>
2015-07-26 11:00               ` Sagi Grimberg
2015-07-26 15:53                 ` Christoph Hellwig
2015-07-26 16:44                   ` Sagi Grimberg
     [not found]                 ` <55B4BDE3.8040801-LDSdmyG8hGV8YrgS2mwiifqBs+8SCbDb@public.gmane.org>
2015-07-26 17:32                   ` Steve Wise [this message]
2015-07-26 17:40                     ` Sagi Grimberg
     [not found]                       ` <55B51B7A.8030008-LDSdmyG8hGV8YrgS2mwiifqBs+8SCbDb@public.gmane.org>
2015-07-26 20:15                         ` Steve Wise
2015-07-26 20:17           ` Steve Wise
     [not found]             ` <55B5404F.5040404-7bPotxP6k4+P2YhJcF5u+vpXobYPEAuW@public.gmane.org>
2015-07-27 21:45               ` Steve Wise
2015-08-03 19:32               ` Steve Wise
2015-08-04 17:26                 ` Sagi Grimberg
     [not found]                   ` <55C0F5B7.4040100-LDSdmyG8hGV8YrgS2mwiifqBs+8SCbDb@public.gmane.org>
2015-08-04 17:44                     ` Steve Wise
2015-08-05 21:23                   ` Steve Wise
2015-08-06 15:37                     ` Sagi Grimberg
     [not found]                       ` <55C37F43.6080106-LDSdmyG8hGV8YrgS2mwiifqBs+8SCbDb@public.gmane.org>
2015-08-12 22:15                         ` Steve Wise
2015-08-13 13:09                           ` Sagi Grimberg
2015-07-24 16:18 ` [PATCH V6 5/9] RDMA/isert: Limit read depth based on the device max_sge_rd capability Steve Wise
2015-07-24 16:18 ` [PATCH V6 7/9] isert: Use the device's max fastreg page list depth Steve Wise
2015-07-24 16:18 ` [PATCH V6 8/9] isert: Use local_dma_lkey whenever possible Steve Wise
     [not found]   ` <20150724161859.25617.17286.stgit-PBeJgSbIpn97NCTnQtmixQ@public.gmane.org>
2015-07-24 16:49     ` Jason Gunthorpe
     [not found]       ` <20150724164909.GB25480-ePGOBjL8dl3ta4EC/59zMFaTQe2KTcn/@public.gmane.org>
2015-07-24 18:41         ` Steve Wise
2015-07-24 16:19 ` [PATCH V6 9/9] isert: Support iWARP transports using FRMRs Steve Wise
2015-07-24 16:57   ` Jason Gunthorpe
2015-07-24 18:48     ` Steve Wise
2015-07-24 19:24       ` Jason Gunthorpe
2015-07-24 19:57         ` Steve Wise
2015-07-24 22:11           ` Steve Wise
2015-07-24 22:38             ` Jason Gunthorpe
     [not found]         ` <20150724192411.GC26225-ePGOBjL8dl3ta4EC/59zMFaTQe2KTcn/@public.gmane.org>
2015-07-26 10:23           ` Sagi Grimberg
2015-07-27 17:07             ` Jason Gunthorpe

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=55B519C6.2030205@opengridcomputing.com \
    --to=swise-7bpotxp6k4+p2yhjcf5u+vpxobypeauw@public.gmane.org \
    --cc=bfields-uC3wQj2KruNg9hUCZPvPmw@public.gmane.org \
    --cc=dledford-H+wXaHxf7aLQT0dZR+AlfA@public.gmane.org \
    --cc=eli-VPRAkNaXOzVWk0Htik3J/w@public.gmane.org \
    --cc=hch-wEGCiKHe2LqWVfeAwA7xHQ@public.gmane.org \
    --cc=infinipath-ral2JQCrhuEAvxtiuMwx3w@public.gmane.org \
    --cc=linux-nfs-u79uwXL29TY76Z2rM5mHXA@public.gmane.org \
    --cc=linux-rdma-u79uwXL29TY76Z2rM5mHXA@public.gmane.org \
    --cc=ogerlitz-VPRAkNaXOzVWk0Htik3J/w@public.gmane.org \
    --cc=roid-VPRAkNaXOzVWk0Htik3J/w@public.gmane.org \
    --cc=sagig-LDSdmyG8hGV8YrgS2mwiifqBs+8SCbDb@public.gmane.org \
    --cc=sagig-VPRAkNaXOzVWk0Htik3J/w@public.gmane.org \
    --cc=target-devel-u79uwXL29TY76Z2rM5mHXA@public.gmane.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).