From: Trond Myklebust <trondmy@primarydata.com>
To: "chuck.lever@oracle.com" <chuck.lever@oracle.com>
Cc: "anna.schumaker@netapp.com" <anna.schumaker@netapp.com>,
"linux-rdma@vger.kernel.org" <linux-rdma@vger.kernel.org>,
"linux-nfs@vger.kernel.org" <linux-nfs@vger.kernel.org>
Subject: Re: [PATCH v3 12/12] sunrpc: Allow keepalive ping on a credit-full transport
Date: Thu, 9 Feb 2017 20:13:58 +0000 [thread overview]
Message-ID: <1486671236.5570.4.camel@primarydata.com> (raw)
In-Reply-To: <4E4245D4-8F9C-4CF3-8B2D-E4528B9E791F@oracle.com>
T24gVGh1LCAyMDE3LTAyLTA5IGF0IDE0OjQyIC0wNTAwLCBDaHVjayBMZXZlciB3cm90ZToNCj4g
PiBPbiBGZWIgOSwgMjAxNywgYXQgMTA6MzcgQU0sIENodWNrIExldmVyIDxjaHVjay5sZXZlckBv
cmFjbGUuY29tPg0KPiA+IHdyb3RlOg0KPiA+IA0KPiA+ID4gDQo+ID4gPiBPbiBGZWIgOCwgMjAx
NywgYXQgNzo0OCBQTSwgVHJvbmQgTXlrbGVidXN0IDx0cm9uZG15QHByaW1hcnlkYXRhLg0KPiA+
ID4gY29tPiB3cm90ZToNCj4gPiA+IA0KPiA+ID4gT24gV2VkLCAyMDE3LTAyLTA4IGF0IDE5OjE5
IC0wNTAwLCBDaHVjayBMZXZlciB3cm90ZToNCj4gPiA+ID4gPiBPbiBGZWIgOCwgMjAxNywgYXQg
NzowNSBQTSwgVHJvbmQgTXlrbGVidXN0IDx0cm9uZG15QHByaW1hcnlkDQo+ID4gPiA+ID4gYXRh
LmNvDQo+ID4gPiA+ID4gbT4gd3JvdGU6DQo+ID4gPiA+ID4gDQo+ID4gPiA+ID4gT24gV2VkLCAy
MDE3LTAyLTA4IGF0IDE3OjAxIC0wNTAwLCBDaHVjayBMZXZlciB3cm90ZToNCj4gPiA+ID4gPiA+
IEFsbG93IFJQQy1vdmVyLVJETUEgdG8gc2VuZCBOVUxMIHBpbmdzIGV2ZW4gd2hlbiB0aGUNCj4g
PiA+ID4gPiA+IHRyYW5zcG9ydA0KPiA+ID4gPiA+ID4gaGFzDQo+ID4gPiA+ID4gPiBoaXQgaXRz
IGNyZWRpdCBsaW1pdC4gT25lIFJQQy1vdmVyLVJETUEgY3JlZGl0IGlzIHJlc2VydmVkDQo+ID4g
PiA+ID4gPiBmb3INCj4gPiA+ID4gPiA+IG9wZXJhdGlvbnMgbGlrZSBrZWVwYWxpdmUuDQo+ID4g
PiA+ID4gPiANCj4gPiA+ID4gPiA+IEZvciB0cmFuc3BvcnRzIHRoYXQgY29udmV5IE5GU3Y0LCBp
dCBzZWVtcyBsaWtlIGxlYXNlDQo+ID4gPiA+ID4gPiByZW5ld2FsDQo+ID4gPiA+ID4gPiB3b3Vs
ZA0KPiA+ID4gPiA+ID4gYWxzbyBiZSBhIGNhbmRpZGF0ZSBmb3IgdXNpbmcgYSBwcmlvcml0eSB0
cmFuc3BvcnQgc2xvdC4NCj4gPiA+ID4gPiA+IEknZCBsaWtlDQo+ID4gPiA+ID4gPiB0bw0KPiA+
ID4gPiA+ID4gc2VlIGEgbWVjaGFuaXNtIGJldHRlciB0aGFuIFJQQ1JETUFfUFJJT1JJVFkgdGhh
dCBjYW4NCj4gPiA+ID4gPiA+IGVuc3VyZSBvbmx5DQo+ID4gPiA+ID4gPiBvbmUgcHJpb3JpdHkg
b3BlcmF0aW9uIGlzIGluIHVzZSBhdCBhIHRpbWUuDQo+ID4gPiA+ID4gPiANCj4gPiA+ID4gPiA+
IFNpZ25lZC1vZmYtYnk6IENodWNrIExldmVyIDxjaHVjay5sZXZlckBvcmFjbGUuY29tPg0KPiA+
ID4gPiA+ID4gLS0tDQo+ID4gPiA+ID4gPiBpbmNsdWRlL2xpbnV4L3N1bnJwYy9zY2hlZC5owqDC
oMKgwqB8wqDCoMKgwqAyICsrDQo+ID4gPiA+ID4gPiBuZXQvc3VucnBjL3hwcnQuY8KgwqDCoMKg
wqDCoMKgwqDCoMKgwqDCoMKgwqDCoHzCoMKgwqDCoDQgKysrKw0KPiA+ID4gPiA+ID4gbmV0L3N1
bnJwYy94cHJ0cmRtYS90cmFuc3BvcnQuYyB8wqDCoMKgwqAzICsrLQ0KPiA+ID4gPiA+ID4gbmV0
L3N1bnJwYy94cHJ0cmRtYS92ZXJicy5jwqDCoMKgwqDCoHzCoMKgwqAxMyArKysrKysrKy0tLS0t
DQo+ID4gPiA+ID4gPiA0IGZpbGVzIGNoYW5nZWQsIDE2IGluc2VydGlvbnMoKyksIDYgZGVsZXRp
b25zKC0pDQo+ID4gPiA+ID4gPiANCj4gPiA+ID4gPiA+IGRpZmYgLS1naXQgYS9pbmNsdWRlL2xp
bnV4L3N1bnJwYy9zY2hlZC5oDQo+ID4gPiA+ID4gPiBiL2luY2x1ZGUvbGludXgvc3VucnBjL3Nj
aGVkLmgNCj4gPiA+ID4gPiA+IGluZGV4IDEzODIyZTYuLmZjZWExNTggMTAwNjQ0DQo+ID4gPiA+
ID4gPiAtLS0gYS9pbmNsdWRlL2xpbnV4L3N1bnJwYy9zY2hlZC5oDQo+ID4gPiA+ID4gPiArKysg
Yi9pbmNsdWRlL2xpbnV4L3N1bnJwYy9zY2hlZC5oDQo+ID4gPiA+ID4gPiBAQCAtMTI3LDYgKzEy
Nyw3IEBAIHN0cnVjdCBycGNfdGFza19zZXR1cCB7DQo+ID4gPiA+ID4gPiAjZGVmaW5lIFJQQ19U
QVNLX1RJTUVPVVQJMHgxMDAwCQkvKg0KPiA+ID4gPiA+ID4gZmFpbA0KPiA+ID4gPiA+ID4gd2l0
aA0KPiA+ID4gPiA+ID4gRVRJTUVET1VUIG9uIHRpbWVvdXQgKi8NCj4gPiA+ID4gPiA+ICNkZWZp
bmUgUlBDX1RBU0tfTk9DT05ORUNUCTB4MjAwMAkJLyoNCj4gPiA+ID4gPiA+IHJldHVybg0KPiA+
ID4gPiA+ID4gRU5PVENPTk4gaWYgbm90IGNvbm5lY3RlZCAqLw0KPiA+ID4gPiA+ID4gI2RlZmlu
ZSBSUENfVEFTS19OT19SRVRSQU5TX1RJTUVPVVQJMHg0MDAwCQkNCj4gPiA+ID4gPiA+IC8qDQo+
ID4gPiA+ID4gPiB3YWl0IGZvcmV2ZXIgZm9yIGEgcmVwbHkgKi8NCj4gPiA+ID4gPiA+ICsjZGVm
aW5lIFJQQ19UQVNLX05PX0NPTkcJMHg4MDAwCQkvKg0KPiA+ID4gPiA+ID4gc2tpcA0KPiA+ID4g
PiA+ID4gY29uZ2VzdGlvbiBjb250cm9sICovDQo+ID4gPiA+ID4gPiANCj4gPiA+ID4gPiA+ICNk
ZWZpbmUgUlBDX1RBU0tfU09GVFBJTkcJKFJQQ19UQVNLX1NPRlQgfA0KPiA+ID4gPiA+ID4gUlBD
X1RBU0tfU09GVENPTk4pDQo+ID4gPiA+ID4gPiANCj4gPiA+ID4gPiA+IEBAIC0xMzcsNiArMTM4
LDcgQEAgc3RydWN0IHJwY190YXNrX3NldHVwIHsNCj4gPiA+ID4gPiA+ICNkZWZpbmUgUlBDX0lT
X1NPRlQodCkJCSgodCktPnRrX2ZsYWdzICYNCj4gPiA+ID4gPiA+IChSUENfVEFTS19TT0ZUfFJQ
Q19UQVNLX1RJTUVPVVQpKQ0KPiA+ID4gPiA+ID4gI2RlZmluZSBSUENfSVNfU09GVENPTk4odCkJ
KCh0KS0+dGtfZmxhZ3MgJg0KPiA+ID4gPiA+ID4gUlBDX1RBU0tfU09GVENPTk4pDQo+ID4gPiA+
ID4gPiAjZGVmaW5lIFJQQ19XQVNfU0VOVCh0KQkJKCh0KS0+dGtfZmxhZ3MgJg0KPiA+ID4gPiA+
ID4gUlBDX1RBU0tfU0VOVCkNCj4gPiA+ID4gPiA+ICsjZGVmaW5lIFJQQ19TS0lQX0NPTkcodCkJ
KCh0KS0+dGtfZmxhZ3MgJg0KPiA+ID4gPiA+ID4gUlBDX1RBU0tfTk9fQ09ORykNCj4gPiA+ID4g
PiA+IA0KPiA+ID4gPiA+ID4gI2RlZmluZSBSUENfVEFTS19SVU5OSU5HCTANCj4gPiA+ID4gPiA+
ICNkZWZpbmUgUlBDX1RBU0tfUVVFVUVECQkxDQo+ID4gPiA+ID4gPiBkaWZmIC0tZ2l0IGEvbmV0
L3N1bnJwYy94cHJ0LmMgYi9uZXQvc3VucnBjL3hwcnQuYw0KPiA+ID4gPiA+ID4gaW5kZXggYjUz
MGEyOC4uYTQ3N2VlNiAxMDA2NDQNCj4gPiA+ID4gPiA+IC0tLSBhL25ldC9zdW5ycGMveHBydC5j
DQo+ID4gPiA+ID4gPiArKysgYi9uZXQvc3VucnBjL3hwcnQuYw0KPiA+ID4gPiA+ID4gQEAgLTM5
Miw2ICszOTIsMTAgQEAgc3RhdGljIGlubGluZSB2b2lkDQo+ID4gPiA+ID4gPiB4cHJ0X3JlbGVh
c2Vfd3JpdGUoc3RydWN0DQo+ID4gPiA+ID4gPiBycGNfeHBydCAqeHBydCwgc3RydWN0IHJwY190
YXNrICp0YQ0KPiA+ID4gPiA+ID4gew0KPiA+ID4gPiA+ID4gCXN0cnVjdCBycGNfcnFzdCAqcmVx
ID0gdGFzay0+dGtfcnFzdHA7DQo+ID4gPiA+ID4gPiANCj4gPiA+ID4gPiA+ICsJaWYgKFJQQ19T
S0lQX0NPTkcodGFzaykpIHsNCj4gPiA+ID4gPiA+ICsJCXJlcS0+cnFfY29uZyA9IDA7DQo+ID4g
PiA+ID4gPiArCQlyZXR1cm4gMTsNCj4gPiA+ID4gPiA+ICsJfQ0KPiA+ID4gPiA+IA0KPiA+ID4g
PiA+IFdoeSBub3QganVzdCBoYXZlIHRoZSBSRE1BIGxheWVyIGNhbGwgeHBydF9yZXNlcnZlX3hw
cnQoKQ0KPiA+ID4gPiA+IChhbmQNCj4gPiA+ID4gPiB4cHJ0X3JlbGVhc2VfeHBydCgpKSBpZiB0
aGlzIGZsYWcgaXMgc2V0PyBJdCBzZWVtcyB0byBtZSB0aGF0DQo+ID4gPiA+ID4geW91DQo+ID4g
PiA+ID4gd2lsbA0KPiA+ID4gPiA+IG5lZWQgc29tZSBraW5kIG9mIGV4dHJhIGNvbmdlc3Rpb24g
Y29udHJvbCBpbiB0aGUgUkRNQSBsYXllcg0KPiA+ID4gPiA+IGFueXdheQ0KPiA+ID4gPiA+IHNp
bmNlIHlvdSBvbmx5IGhhdmUgb25lIHJlc2VydmVkIGNyZWRpdCBmb3IgdGhlc2UgcHJpdmlsZWdl
ZA0KPiA+ID4gPiA+IHRhc2tzDQo+ID4gPiA+ID4gKG9yDQo+ID4gPiA+ID4gZGlkIEkgbWlzcyB3
aGVyZSB0aGF0IGlzIGJlaW5nIGdhdGVkPykuDQo+ID4gPiA+IA0KPiA+ID4gPiBUaGFua3MgZm9y
IHRoZSByZXZpZXcuDQo+ID4gPiA+IA0KPiA+ID4gPiBTZWUgUlBDUkRNQV9JQV9SU1ZEX0NSRURJ
VCBpbiAxMS8xMi4gSXQncyBhIGhhY2sgSSdtIG5vdA0KPiA+ID4gPiB0ZXJyaWJseSBoYXBweSB3
aXRoLg0KPiA+ID4gPiANCj4gPiA+ID4gU28sIEkgdGhpbmsgeW91IGFyZSBzdWdnZXN0aW5nIHJl
cGxhY2luZyB4cHJ0cmRtYSdzDQo+ID4gPiA+IC0+cmVzZXJ2ZV94cHJ0IHdpdGggc29tZXRoaW5n
IGxpa2U6DQo+ID4gPiA+IA0KPiA+ID4gPiBpbnQgeHBydF9yZG1hX3Jlc2VydmVfeHBydCh4cHJ0
LCB0YXNrKQ0KPiA+ID4gPiB7DQo+ID4gPiA+IMKgwqDCoMKgwqBpZiAoUlBDX1NLSVBfQ09ORyh0
YXNrKSkNCj4gPiA+ID4gwqDCoMKgwqDCoMKgwqDCoMKgwqByZXR1cm4geHBydF9yZXNlcnZlX3hw
cnQoeHBydCwgdGFzayk7DQo+ID4gPiA+IMKgwqDCoMKgwqByZXR1cm4geHBydF9yZXNlcnZlX3hw
cnRfY29uZyh4cHJ0LCB0YXNrKTsNCj4gPiA+ID4gfQ0KPiA+ID4gPiANCj4gPiA+ID4gYW5kIGxp
a2V3aXNlIGZvciAtPnJlbGVhc2VfeHBydCA/DQo+ID4gPiANCj4gPiA+IFJpZ2h0Lg0KPiANCj4g
VGhpcyBzZWVtcyB0byB3b3JrIGZpbmUgZm9yIHRoZSBub3JtYWwgY2FzZXMuDQo+IA0KPiBJJ20g
Y29uZnVzZWQgYWJvdXQgaG93IHRvIGNvbnN0cnVjdCB4cHJ0X3JkbWFfcmVsZWFzZV94cHJ0KCkN
Cj4gc28gaXQgbmV2ZXIgcmVsZWFzZXMgYSBub3JtYWwgUlBDIHRhc2sgd2hlbiBhIFNLSVBfQ09O
Rw0KPiB0YXNrIGNvbXBsZXRlcyBhbmQgdGhlIGNyZWRpdCBsaW1pdCBpcyBzdGlsbCBmdWxsLg0K
PiANCj4gSWYgaXQgc2hvdWxkIHNlbmQgYSBub3JtYWwgdGFzayB1c2luZyB0aGUgcmVzZXJ2ZWQg
Y3JlZGl0DQo+IGFuZCB0aGF0IHRhc2sgaGFuZ3MgdG9vLCB3ZSdyZSBpbiBleGFjdGx5IHRoZSBw
b3NpdGlvbg0KPiB3ZSB3YW50ZWQgdG8gYXZvaWQuDQo+IA0KPiBNeSBvcmlnaW5hbCBzb2x1dGlv
biBtaWdodCBoYXZlIGhhZCBhIHNpbWlsYXIgcHJvYmxlbSwNCj4gY29tZSB0byB0aGluayBvZiBp
dC4NCj4gDQo+IA0KDQpUaGF0J3MgdHJ1ZS4uLiBZb3UgbWF5IG5lZWQgdG8gc2V0IHVwIGEgc2Vw
YXJhdGUgd2FpdHF1ZXVlIHRoYXQgaXMNCnJlc2VydmVkIGZvciBTS0lQX0NPTkcgdGFza3MuIEFn
YWluLCBpdCBtYWtlcyBzZW5zZSB0byBrZWVwIHRoYXQgaW4gdGhlDQpSRE1BIGNvZGUuDQoNCi0t
IA0KDQoNCgkNCgkNCg0KDQpUcm9uZCBNeWtsZWJ1c3QNClByaW5jaXBhbCBTeXN0ZW0gQXJjaGl0
ZWN0DQo0MzAwIEVsIENhbWlubyBSZWFsIHwgU3VpdGUgMTAwDQpMb3MgQWx0b3MsIENBwqDCoDk0
MDIyDQpXOiA2NTAtNDIyLTM4MDANCkM6IDgwMS05MjEtNDU4M8KgDQp3d3cucHJpbWFyeWRhdGEu
Y29tDQoNCg0KDQo=
WARNING: multiple messages have this Message-ID (diff)
From: Trond Myklebust <trondmy-7I+n7zu2hftEKMMhf/gKZA@public.gmane.org>
To: "chuck.lever-QHcLZuEGTsvQT0dZR+AlfA@public.gmane.org"
<chuck.lever-QHcLZuEGTsvQT0dZR+AlfA@public.gmane.org>
Cc: "anna.schumaker-HgOvQuBEEgTQT0dZR+AlfA@public.gmane.org"
<anna.schumaker-HgOvQuBEEgTQT0dZR+AlfA@public.gmane.org>,
"linux-rdma-u79uwXL29TY76Z2rM5mHXA@public.gmane.org"
<linux-rdma-u79uwXL29TY76Z2rM5mHXA@public.gmane.org>,
"linux-nfs-u79uwXL29TY76Z2rM5mHXA@public.gmane.org"
<linux-nfs-u79uwXL29TY76Z2rM5mHXA@public.gmane.org>
Subject: Re: [PATCH v3 12/12] sunrpc: Allow keepalive ping on a credit-full transport
Date: Thu, 9 Feb 2017 20:13:58 +0000 [thread overview]
Message-ID: <1486671236.5570.4.camel@primarydata.com> (raw)
In-Reply-To: <4E4245D4-8F9C-4CF3-8B2D-E4528B9E791F-QHcLZuEGTsvQT0dZR+AlfA@public.gmane.org>
On Thu, 2017-02-09 at 14:42 -0500, Chuck Lever wrote:
> > On Feb 9, 2017, at 10:37 AM, Chuck Lever <chuck.lever@oracle.com>
> > wrote:
> >
> > >
> > > On Feb 8, 2017, at 7:48 PM, Trond Myklebust <trondmy@primarydata.
> > > com> wrote:
> > >
> > > On Wed, 2017-02-08 at 19:19 -0500, Chuck Lever wrote:
> > > > > On Feb 8, 2017, at 7:05 PM, Trond Myklebust <trondmy@primaryd
> > > > > ata.co
> > > > > m> wrote:
> > > > >
> > > > > On Wed, 2017-02-08 at 17:01 -0500, Chuck Lever wrote:
> > > > > > Allow RPC-over-RDMA to send NULL pings even when the
> > > > > > transport
> > > > > > has
> > > > > > hit its credit limit. One RPC-over-RDMA credit is reserved
> > > > > > for
> > > > > > operations like keepalive.
> > > > > >
> > > > > > For transports that convey NFSv4, it seems like lease
> > > > > > renewal
> > > > > > would
> > > > > > also be a candidate for using a priority transport slot.
> > > > > > I'd like
> > > > > > to
> > > > > > see a mechanism better than RPCRDMA_PRIORITY that can
> > > > > > ensure only
> > > > > > one priority operation is in use at a time.
> > > > > >
> > > > > > Signed-off-by: Chuck Lever <chuck.lever@oracle.com>
> > > > > > ---
> > > > > > include/linux/sunrpc/sched.h | 2 ++
> > > > > > net/sunrpc/xprt.c | 4 ++++
> > > > > > net/sunrpc/xprtrdma/transport.c | 3 ++-
> > > > > > net/sunrpc/xprtrdma/verbs.c | 13 ++++++++-----
> > > > > > 4 files changed, 16 insertions(+), 6 deletions(-)
> > > > > >
> > > > > > diff --git a/include/linux/sunrpc/sched.h
> > > > > > b/include/linux/sunrpc/sched.h
> > > > > > index 13822e6..fcea158 100644
> > > > > > --- a/include/linux/sunrpc/sched.h
> > > > > > +++ b/include/linux/sunrpc/sched.h
> > > > > > @@ -127,6 +127,7 @@ struct rpc_task_setup {
> > > > > > #define RPC_TASK_TIMEOUT 0x1000 /*
> > > > > > fail
> > > > > > with
> > > > > > ETIMEDOUT on timeout */
> > > > > > #define RPC_TASK_NOCONNECT 0x2000 /*
> > > > > > return
> > > > > > ENOTCONN if not connected */
> > > > > > #define RPC_TASK_NO_RETRANS_TIMEOUT 0x4000
> > > > > > /*
> > > > > > wait forever for a reply */
> > > > > > +#define RPC_TASK_NO_CONG 0x8000 /*
> > > > > > skip
> > > > > > congestion control */
> > > > > >
> > > > > > #define RPC_TASK_SOFTPING (RPC_TASK_SOFT |
> > > > > > RPC_TASK_SOFTCONN)
> > > > > >
> > > > > > @@ -137,6 +138,7 @@ struct rpc_task_setup {
> > > > > > #define RPC_IS_SOFT(t) ((t)->tk_flags &
> > > > > > (RPC_TASK_SOFT|RPC_TASK_TIMEOUT))
> > > > > > #define RPC_IS_SOFTCONN(t) ((t)->tk_flags &
> > > > > > RPC_TASK_SOFTCONN)
> > > > > > #define RPC_WAS_SENT(t) ((t)->tk_flags &
> > > > > > RPC_TASK_SENT)
> > > > > > +#define RPC_SKIP_CONG(t) ((t)->tk_flags &
> > > > > > RPC_TASK_NO_CONG)
> > > > > >
> > > > > > #define RPC_TASK_RUNNING 0
> > > > > > #define RPC_TASK_QUEUED 1
> > > > > > diff --git a/net/sunrpc/xprt.c b/net/sunrpc/xprt.c
> > > > > > index b530a28..a477ee6 100644
> > > > > > --- a/net/sunrpc/xprt.c
> > > > > > +++ b/net/sunrpc/xprt.c
> > > > > > @@ -392,6 +392,10 @@ static inline void
> > > > > > xprt_release_write(struct
> > > > > > rpc_xprt *xprt, struct rpc_task *ta
> > > > > > {
> > > > > > struct rpc_rqst *req = task->tk_rqstp;
> > > > > >
> > > > > > + if (RPC_SKIP_CONG(task)) {
> > > > > > + req->rq_cong = 0;
> > > > > > + return 1;
> > > > > > + }
> > > > >
> > > > > Why not just have the RDMA layer call xprt_reserve_xprt()
> > > > > (and
> > > > > xprt_release_xprt()) if this flag is set? It seems to me that
> > > > > you
> > > > > will
> > > > > need some kind of extra congestion control in the RDMA layer
> > > > > anyway
> > > > > since you only have one reserved credit for these privileged
> > > > > tasks
> > > > > (or
> > > > > did I miss where that is being gated?).
> > > >
> > > > Thanks for the review.
> > > >
> > > > See RPCRDMA_IA_RSVD_CREDIT in 11/12. It's a hack I'm not
> > > > terribly happy with.
> > > >
> > > > So, I think you are suggesting replacing xprtrdma's
> > > > ->reserve_xprt with something like:
> > > >
> > > > int xprt_rdma_reserve_xprt(xprt, task)
> > > > {
> > > > if (RPC_SKIP_CONG(task))
> > > > return xprt_reserve_xprt(xprt, task);
> > > > return xprt_reserve_xprt_cong(xprt, task);
> > > > }
> > > >
> > > > and likewise for ->release_xprt ?
> > >
> > > Right.
>
> This seems to work fine for the normal cases.
>
> I'm confused about how to construct xprt_rdma_release_xprt()
> so it never releases a normal RPC task when a SKIP_CONG
> task completes and the credit limit is still full.
>
> If it should send a normal task using the reserved credit
> and that task hangs too, we're in exactly the position
> we wanted to avoid.
>
> My original solution might have had a similar problem,
> come to think of it.
>
>
That's true... You may need to set up a separate waitqueue that is
reserved for SKIP_CONG tasks. Again, it makes sense to keep that in the
RDMA code.
--
Trond Myklebust
Principal System Architect
4300 El Camino Real | Suite 100
Los Altos, CA 94022
W: 650-422-3800
C: 801-921-4583
www.primarydata.com
next prev parent reply other threads:[~2017-02-09 20:14 UTC|newest]
Thread overview: 42+ messages / expand[flat|nested] mbox.gz Atom feed top
2017-02-08 21:59 [PATCH v3 00/12] NFS/RDMA client-side patches for 4.11 Chuck Lever
2017-02-08 21:59 ` Chuck Lever
2017-02-08 21:59 ` [PATCH v3 01/12] xprtrdma: Fix Read chunk padding Chuck Lever
2017-02-08 21:59 ` Chuck Lever
2017-02-08 21:59 ` [PATCH v3 02/12] xprtrdma: Per-connection pad optimization Chuck Lever
2017-02-08 21:59 ` Chuck Lever
2017-02-08 22:00 ` [PATCH v3 03/12] xprtrdma: Disable pad optimization by default Chuck Lever
2017-02-08 22:00 ` Chuck Lever
2017-02-08 22:00 ` [PATCH v3 04/12] xprtrdma: Reduce required number of send SGEs Chuck Lever
2017-02-08 22:00 ` Chuck Lever
2017-02-08 22:00 ` [PATCH v3 05/12] xprtrdma: Shrink send SGEs array Chuck Lever
2017-02-08 22:00 ` Chuck Lever
2017-02-08 22:00 ` [PATCH v3 06/12] xprtrdma: Properly recover FRWRs with in-flight FASTREG WRs Chuck Lever
2017-02-08 22:00 ` Chuck Lever
2017-02-08 22:00 ` [PATCH v3 07/12] xprtrdma: Handle stale connection rejection Chuck Lever
2017-02-08 22:00 ` Chuck Lever
2017-02-08 22:00 ` [PATCH v3 08/12] xprtrdma: Refactor management of mw_list field Chuck Lever
2017-02-08 22:00 ` Chuck Lever
2017-02-08 22:00 ` [PATCH v3 09/12] sunrpc: Allow xprt->ops->timer method to sleep Chuck Lever
2017-02-08 22:00 ` Chuck Lever
2017-02-08 23:48 ` Trond Myklebust
2017-02-08 23:48 ` Trond Myklebust
2017-02-08 22:00 ` [PATCH v3 10/12] sunrpc: Enable calls to rpc_call_null_helper() from other modules Chuck Lever
2017-02-08 22:00 ` Chuck Lever
2017-02-08 22:01 ` [PATCH v3 11/12] xprtrdma: Detect unreachable NFS/RDMA servers more reliably Chuck Lever
2017-02-08 22:01 ` Chuck Lever
2017-02-08 22:01 ` [PATCH v3 12/12] sunrpc: Allow keepalive ping on a credit-full transport Chuck Lever
2017-02-08 22:01 ` Chuck Lever
2017-02-09 0:05 ` Trond Myklebust
2017-02-09 0:05 ` Trond Myklebust
2017-02-09 0:19 ` Chuck Lever
2017-02-09 0:19 ` Chuck Lever
2017-02-09 0:48 ` Trond Myklebust
2017-02-09 0:48 ` Trond Myklebust
2017-02-09 15:37 ` Chuck Lever
2017-02-09 15:37 ` Chuck Lever
2017-02-09 19:42 ` Chuck Lever
2017-02-09 19:42 ` Chuck Lever
2017-02-09 20:13 ` Trond Myklebust [this message]
2017-02-09 20:13 ` Trond Myklebust
2017-02-09 20:39 ` Chuck Lever
2017-02-09 20:39 ` Chuck Lever
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=1486671236.5570.4.camel@primarydata.com \
--to=trondmy@primarydata.com \
--cc=anna.schumaker@netapp.com \
--cc=chuck.lever@oracle.com \
--cc=linux-nfs@vger.kernel.org \
--cc=linux-rdma@vger.kernel.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.