[PATCH] svcrdma: Fence LOCAL_INV work requests

[PATCH] svcrdma: Fence LOCAL_INV work requests

[Steve Wise]

This applies on top of:

	commit e5a070216356dbcb03607cb264cc3104e17339b3
	Author: Steve Wise <swise@opengridcomputing.com>
	Date:   Wed May 28 15:12:01 2014 -0500

	    svcrdma: refactor marshalling logic

Fencing forces the invalidate to only happen after all subsequent send
work requests have been completed.

Signed-off-by: Steve Wise <swise@opengridcomputing.com>
---

 net/sunrpc/xprtrdma/svc_rdma_recvfrom.c |    2 +-
 1 files changed, 1 insertions(+), 1 deletions(-)

diff --git a/net/sunrpc/xprtrdma/svc_rdma_recvfrom.c b/net/sunrpc/xprtrdma/svc_rdma_recvfrom.c
index 52d9f2c..8f92a61 100644
--- a/net/sunrpc/xprtrdma/svc_rdma_recvfrom.c
+++ b/net/sunrpc/xprtrdma/svc_rdma_recvfrom.c
@@ -338,7 +338,7 @@ static int rdma_read_chunk_frmr(struct svcxprt_rdma *xprt,
 		memset(&inv_wr, 0, sizeof(inv_wr));
 		inv_wr.wr_id = (unsigned long)ctxt;
 		inv_wr.opcode = IB_WR_LOCAL_INV;
-		inv_wr.send_flags = IB_SEND_SIGNALED;
+		inv_wr.send_flags = IB_SEND_SIGNALED | IB_SEND_FENCE;
 		inv_wr.ex.invalidate_rkey = frmr->mr->lkey;
 	}
 	ctxt->wr_op = read_wr.opcode;

[PATCH] svcrdma: Fence LOCAL_INV work requests

[Steve Wise]

This applies on top of:

	commit e5a070216356dbcb03607cb264cc3104e17339b3
	Author: Steve Wise <swise-7bPotxP6k4+P2YhJcF5u+vpXobYPEAuW@public.gmane.org>
	Date:   Wed May 28 15:12:01 2014 -0500

	    svcrdma: refactor marshalling logic

Fencing forces the invalidate to only happen after all subsequent send
work requests have been completed.

Signed-off-by: Steve Wise <swise-7bPotxP6k4+P2YhJcF5u+vpXobYPEAuW@public.gmane.org>
---

 net/sunrpc/xprtrdma/svc_rdma_recvfrom.c |    2 +-
 1 files changed, 1 insertions(+), 1 deletions(-)

diff --git a/net/sunrpc/xprtrdma/svc_rdma_recvfrom.c b/net/sunrpc/xprtrdma/svc_rdma_recvfrom.c
index 52d9f2c..8f92a61 100644
--- a/net/sunrpc/xprtrdma/svc_rdma_recvfrom.c
+++ b/net/sunrpc/xprtrdma/svc_rdma_recvfrom.c
@@ -338,7 +338,7 @@ static int rdma_read_chunk_frmr(struct svcxprt_rdma *xprt,
 		memset(&inv_wr, 0, sizeof(inv_wr));
 		inv_wr.wr_id = (unsigned long)ctxt;
 		inv_wr.opcode = IB_WR_LOCAL_INV;
-		inv_wr.send_flags = IB_SEND_SIGNALED;
+		inv_wr.send_flags = IB_SEND_SIGNALED | IB_SEND_FENCE;
 		inv_wr.ex.invalidate_rkey = frmr->mr->lkey;
 	}
 	ctxt->wr_op = read_wr.opcode;

--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

RE: [PATCH] svcrdma: Fence LOCAL_INV work requests

[Devesh Sharma]

UmVwb3J0ZWQgYnkgOiBEZXZlc2ggU2hhcm1hDQpBY2snZWQgYnk6IERldmVzaCBTaGFybWENCg0K
LVJlZ2FyZHMNCiBEZXZlc2gNCg0KPiAtLS0tLU9yaWdpbmFsIE1lc3NhZ2UtLS0tLQ0KPiBGcm9t
OiBsaW51eC1yZG1hLW93bmVyQHZnZXIua2VybmVsLm9yZyBbbWFpbHRvOmxpbnV4LXJkbWEtDQo+
IG93bmVyQHZnZXIua2VybmVsLm9yZ10gT24gQmVoYWxmIE9mIFN0ZXZlIFdpc2UNCj4gU2VudDog
TW9uZGF5LCBKdW5lIDAyLCAyMDE0IDEwOjQ1IFBNDQo+IFRvOiBiZmllbGRzQGZpZWxkc2VzLm9y
Zw0KPiBDYzogbGludXgtbmZzQHZnZXIua2VybmVsLm9yZzsgbGludXgtcmRtYUB2Z2VyLmtlcm5l
bC5vcmc7DQo+IHRvbUBvcGVuZ3JpZGNvbXB1dGluZy5jb20NCj4gU3ViamVjdDogW1BBVENIXSBz
dmNyZG1hOiBGZW5jZSBMT0NBTF9JTlYgd29yayByZXF1ZXN0cw0KPiANCj4gVGhpcyBhcHBsaWVz
IG9uIHRvcCBvZjoNCj4gDQo+IAljb21taXQgZTVhMDcwMjE2MzU2ZGJjYjAzNjA3Y2IyNjRjYzMx
MDRlMTczMzliMw0KPiAJQXV0aG9yOiBTdGV2ZSBXaXNlIDxzd2lzZUBvcGVuZ3JpZGNvbXB1dGlu
Zy5jb20+DQo+IAlEYXRlOiAgIFdlZCBNYXkgMjggMTU6MTI6MDEgMjAxNCAtMDUwMA0KPiANCj4g
CSAgICBzdmNyZG1hOiByZWZhY3RvciBtYXJzaGFsbGluZyBsb2dpYw0KPiANCj4gRmVuY2luZyBm
b3JjZXMgdGhlIGludmFsaWRhdGUgdG8gb25seSBoYXBwZW4gYWZ0ZXIgYWxsIHN1YnNlcXVlbnQg
c2VuZCB3b3JrDQo+IHJlcXVlc3RzIGhhdmUgYmVlbiBjb21wbGV0ZWQuDQo+IA0KPiBTaWduZWQt
b2ZmLWJ5OiBTdGV2ZSBXaXNlIDxzd2lzZUBvcGVuZ3JpZGNvbXB1dGluZy5jb20+DQo+IC0tLQ0K
PiANCj4gIG5ldC9zdW5ycGMveHBydHJkbWEvc3ZjX3JkbWFfcmVjdmZyb20uYyB8ICAgIDIgKy0N
Cj4gIDEgZmlsZXMgY2hhbmdlZCwgMSBpbnNlcnRpb25zKCspLCAxIGRlbGV0aW9ucygtKQ0KPiAN
Cj4gZGlmZiAtLWdpdCBhL25ldC9zdW5ycGMveHBydHJkbWEvc3ZjX3JkbWFfcmVjdmZyb20uYw0K
PiBiL25ldC9zdW5ycGMveHBydHJkbWEvc3ZjX3JkbWFfcmVjdmZyb20uYw0KPiBpbmRleCA1MmQ5
ZjJjLi44ZjkyYTYxIDEwMDY0NA0KPiAtLS0gYS9uZXQvc3VucnBjL3hwcnRyZG1hL3N2Y19yZG1h
X3JlY3Zmcm9tLmMNCj4gKysrIGIvbmV0L3N1bnJwYy94cHJ0cmRtYS9zdmNfcmRtYV9yZWN2ZnJv
bS5jDQo+IEBAIC0zMzgsNyArMzM4LDcgQEAgc3RhdGljIGludCByZG1hX3JlYWRfY2h1bmtfZnJt
cihzdHJ1Y3Qgc3ZjeHBydF9yZG1hDQo+ICp4cHJ0LA0KPiAgCQltZW1zZXQoJmludl93ciwgMCwg
c2l6ZW9mKGludl93cikpOw0KPiAgCQlpbnZfd3Iud3JfaWQgPSAodW5zaWduZWQgbG9uZyljdHh0
Ow0KPiAgCQlpbnZfd3Iub3Bjb2RlID0gSUJfV1JfTE9DQUxfSU5WOw0KPiAtCQlpbnZfd3Iuc2Vu
ZF9mbGFncyA9IElCX1NFTkRfU0lHTkFMRUQ7DQo+ICsJCWludl93ci5zZW5kX2ZsYWdzID0gSUJf
U0VORF9TSUdOQUxFRCB8IElCX1NFTkRfRkVOQ0U7DQo+ICAJCWludl93ci5leC5pbnZhbGlkYXRl
X3JrZXkgPSBmcm1yLT5tci0+bGtleTsNCj4gIAl9DQo+ICAJY3R4dC0+d3Jfb3AgPSByZWFkX3dy
Lm9wY29kZTsNCj4gDQo+IC0tDQo+IFRvIHVuc3Vic2NyaWJlIGZyb20gdGhpcyBsaXN0OiBzZW5k
IHRoZSBsaW5lICJ1bnN1YnNjcmliZSBsaW51eC1yZG1hIiBpbiB0aGUNCj4gYm9keSBvZiBhIG1l
c3NhZ2UgdG8gbWFqb3Jkb21vQHZnZXIua2VybmVsLm9yZyBNb3JlIG1ham9yZG9tbyBpbmZvIGF0
DQo+IGh0dHA6Ly92Z2VyLmtlcm5lbC5vcmcvbWFqb3Jkb21vLWluZm8uaHRtbA0K

RE: [PATCH] svcrdma: Fence LOCAL_INV work requests

[Devesh Sharma]

[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #1: Type: text/plain; charset="utf-8", Size: 1948 bytes --]

Reported by : Devesh Sharma
Ack'ed by: Devesh Sharma

-Regards
 Devesh

> -----Original Message-----
> From: linux-rdma-owner@vger.kernel.org [mailto:linux-rdma-
> owner@vger.kernel.org] On Behalf Of Steve Wise
> Sent: Monday, June 02, 2014 10:45 PM
> To: bfields@fieldses.org
> Cc: linux-nfs@vger.kernel.org; linux-rdma@vger.kernel.org;
> tom@opengridcomputing.com
> Subject: [PATCH] svcrdma: Fence LOCAL_INV work requests
> 
> This applies on top of:
> 
> 	commit e5a070216356dbcb03607cb264cc3104e17339b3
> 	Author: Steve Wise <swise@opengridcomputing.com>
> 	Date:   Wed May 28 15:12:01 2014 -0500
> 
> 	    svcrdma: refactor marshalling logic
> 
> Fencing forces the invalidate to only happen after all subsequent send work
> requests have been completed.
> 
> Signed-off-by: Steve Wise <swise@opengridcomputing.com>
> ---
> 
>  net/sunrpc/xprtrdma/svc_rdma_recvfrom.c |    2 +-
>  1 files changed, 1 insertions(+), 1 deletions(-)
> 
> diff --git a/net/sunrpc/xprtrdma/svc_rdma_recvfrom.c
> b/net/sunrpc/xprtrdma/svc_rdma_recvfrom.c
> index 52d9f2c..8f92a61 100644
> --- a/net/sunrpc/xprtrdma/svc_rdma_recvfrom.c
> +++ b/net/sunrpc/xprtrdma/svc_rdma_recvfrom.c
> @@ -338,7 +338,7 @@ static int rdma_read_chunk_frmr(struct svcxprt_rdma
> *xprt,
>  		memset(&inv_wr, 0, sizeof(inv_wr));
>  		inv_wr.wr_id = (unsigned long)ctxt;
>  		inv_wr.opcode = IB_WR_LOCAL_INV;
> -		inv_wr.send_flags = IB_SEND_SIGNALED;
> +		inv_wr.send_flags = IB_SEND_SIGNALED | IB_SEND_FENCE;
>  		inv_wr.ex.invalidate_rkey = frmr->mr->lkey;
>  	}
>  	ctxt->wr_op = read_wr.opcode;
> 
> --
> To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the
> body of a message to majordomo@vger.kernel.org More majordomo info at
> http://vger.kernel.org/majordomo-info.html
N‹§²æìr¸›yúèšØb²X¬¶Ç§vØ^–)Þº{.nÇ+‰·¥Š{±­ÙšŠ{ayº\x1dʇڙë,j\a­¢f£¢·hš‹»öì\x17/oSc¾™Ú³9˜uÀ¦æå‰È&jw¨®\x03(­éšŽŠÝ¢j"ú\x1a¶^[m§ÿïêäz¹Þ–Šàþf£¢·hšˆ§~ˆmš

Re: [PATCH] svcrdma: Fence LOCAL_INV work requests

[Bart Van Assche]

On 06/02/14 19:14, Steve Wise wrote:
> Fencing forces the invalidate to only happen after all subsequent send
> work requests have been completed.

Shouldn't that read "prior" instead of "subsequent" ?

Bart.

Re: [PATCH] svcrdma: Fence LOCAL_INV work requests

[Bart Van Assche]

On 06/02/14 19:14, Steve Wise wrote:
> Fencing forces the invalidate to only happen after all subsequent send
> work requests have been completed.

Shouldn't that read "prior" instead of "subsequent" ?

Bart.
--
To unsubscribe from this list: send the line "unsubscribe linux-nfs" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

RE: [PATCH] svcrdma: Fence LOCAL_INV work requests

[Steve Wise]

> -----Original Message-----
> From: Bart Van Assche [mailto:bvanassche@acm.org]
> Sent: Thursday, June 05, 2014 2:23 AM
> To: Steve Wise; bfields@fieldses.org
> Cc: linux-nfs@vger.kernel.org; linux-rdma@vger.kernel.org; tom@opengridcomputing.com
> Subject: Re: [PATCH] svcrdma: Fence LOCAL_INV work requests
> 
> On 06/02/14 19:14, Steve Wise wrote:
> > Fencing forces the invalidate to only happen after all subsequent send
> > work requests have been completed.
> 
> Shouldn't that read "prior" instead of "subsequent" ?
>

Yes.  
 

RE: [PATCH] svcrdma: Fence LOCAL_INV work requests

[Steve Wise]

> -----Original Message-----
> From: Bart Van Assche [mailto:bvanassche-HInyCGIudOg@public.gmane.org]
> Sent: Thursday, June 05, 2014 2:23 AM
> To: Steve Wise; bfields-uC3wQj2KruNg9hUCZPvPmw@public.gmane.org
> Cc: linux-nfs-u79uwXL29TY76Z2rM5mHXA@public.gmane.org; linux-rdma-u79uwXL29TY76Z2rM5mHXA@public.gmane.org; tom-7bPotxP6k4+P2YhJcF5u+vpXobYPEAuW@public.gmane.org
> Subject: Re: [PATCH] svcrdma: Fence LOCAL_INV work requests
> 
> On 06/02/14 19:14, Steve Wise wrote:
> > Fencing forces the invalidate to only happen after all subsequent send
> > work requests have been completed.
> 
> Shouldn't that read "prior" instead of "subsequent" ?
>

Yes.  
 

--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH] svcrdma: Fence LOCAL_INV work requests

[Steve Wise]

On 6/5/2014 8:55 AM, Steve Wise wrote
>> -----Original Message-----
>> From: Bart Van Assche [mailto:bvanassche@acm.org]
>> Sent: Thursday, June 05, 2014 2:23 AM
>> To: Steve Wise; bfields@fieldses.org
>> Cc: linux-nfs@vger.kernel.org; linux-rdma@vger.kernel.org; tom@opengridcomputing.com
>> Subject: Re: [PATCH] svcrdma: Fence LOCAL_INV work requests
>>
>> On 06/02/14 19:14, Steve Wise wrote:
>>> Fencing forces the invalidate to only happen after all subsequent send
>>> work requests have been completed.
>> Shouldn't that read "prior" instead of "subsequent" ?
>>
> Yes.
>   

I posted V2 of the patch to fix the comment.

Thanks,

Steve.

Re: [PATCH] svcrdma: Fence LOCAL_INV work requests

[Steve Wise]

On 6/5/2014 8:55 AM, Steve Wise wrote
>> -----Original Message-----
>> From: Bart Van Assche [mailto:bvanassche-HInyCGIudOg@public.gmane.org]
>> Sent: Thursday, June 05, 2014 2:23 AM
>> To: Steve Wise; bfields-uC3wQj2KruNg9hUCZPvPmw@public.gmane.org
>> Cc: linux-nfs-u79uwXL29TY76Z2rM5mHXA@public.gmane.org; linux-rdma-u79uwXL29TY76Z2rM5mHXA@public.gmane.org; tom-7bPotxP6k4+P2YhJcF5u+vpXobYPEAuW@public.gmane.org
>> Subject: Re: [PATCH] svcrdma: Fence LOCAL_INV work requests
>>
>> On 06/02/14 19:14, Steve Wise wrote:
>>> Fencing forces the invalidate to only happen after all subsequent send
>>> work requests have been completed.
>> Shouldn't that read "prior" instead of "subsequent" ?
>>
> Yes.
>   

I posted V2 of the patch to fix the comment.

Thanks,

Steve.
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html