From: Trond Myklebust <trondmy@primarydata.com>
To: "chuck.lever@oracle.com" <chuck.lever@oracle.com>
Cc: "anna.schumaker@netapp.com" <anna.schumaker@netapp.com>,
"linux-rdma@vger.kernel.org" <linux-rdma@vger.kernel.org>,
"linux-nfs@vger.kernel.org" <linux-nfs@vger.kernel.org>
Subject: Re: [PATCH v3 12/12] sunrpc: Allow keepalive ping on a credit-full transport
Date: Thu, 9 Feb 2017 00:48:54 +0000 [thread overview]
Message-ID: <1486601331.11028.5.camel@primarydata.com> (raw)
In-Reply-To: <9D6B8B44-9C23-427C-9E06-7C92302EB04D@oracle.com>
T24gV2VkLCAyMDE3LTAyLTA4IGF0IDE5OjE5IC0wNTAwLCBDaHVjayBMZXZlciB3cm90ZToNCj4g
PiBPbiBGZWIgOCwgMjAxNywgYXQgNzowNSBQTSwgVHJvbmQgTXlrbGVidXN0IDx0cm9uZG15QHBy
aW1hcnlkYXRhLmNvDQo+ID4gbT4gd3JvdGU6DQo+ID4gDQo+ID4gT24gV2VkLCAyMDE3LTAyLTA4
IGF0IDE3OjAxIC0wNTAwLCBDaHVjayBMZXZlciB3cm90ZToNCj4gPiA+IEFsbG93IFJQQy1vdmVy
LVJETUEgdG8gc2VuZCBOVUxMIHBpbmdzIGV2ZW4gd2hlbiB0aGUgdHJhbnNwb3J0DQo+ID4gPiBo
YXMNCj4gPiA+IGhpdCBpdHMgY3JlZGl0IGxpbWl0LiBPbmUgUlBDLW92ZXItUkRNQSBjcmVkaXQg
aXMgcmVzZXJ2ZWQgZm9yDQo+ID4gPiBvcGVyYXRpb25zIGxpa2Uga2VlcGFsaXZlLg0KPiA+ID4g
DQo+ID4gPiBGb3IgdHJhbnNwb3J0cyB0aGF0IGNvbnZleSBORlN2NCwgaXQgc2VlbXMgbGlrZSBs
ZWFzZSByZW5ld2FsDQo+ID4gPiB3b3VsZA0KPiA+ID4gYWxzbyBiZSBhIGNhbmRpZGF0ZSBmb3Ig
dXNpbmcgYSBwcmlvcml0eSB0cmFuc3BvcnQgc2xvdC4gSSdkIGxpa2UNCj4gPiA+IHRvDQo+ID4g
PiBzZWUgYSBtZWNoYW5pc20gYmV0dGVyIHRoYW4gUlBDUkRNQV9QUklPUklUWSB0aGF0IGNhbiBl
bnN1cmUgb25seQ0KPiA+ID4gb25lIHByaW9yaXR5IG9wZXJhdGlvbiBpcyBpbiB1c2UgYXQgYSB0
aW1lLg0KPiA+ID4gDQo+ID4gPiBTaWduZWQtb2ZmLWJ5OiBDaHVjayBMZXZlciA8Y2h1Y2subGV2
ZXJAb3JhY2xlLmNvbT4NCj4gPiA+IC0tLQ0KPiA+ID4gwqBpbmNsdWRlL2xpbnV4L3N1bnJwYy9z
Y2hlZC5owqDCoMKgwqB8wqDCoMKgwqAyICsrDQo+ID4gPiDCoG5ldC9zdW5ycGMveHBydC5jwqDC
oMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgfMKgwqDCoMKgNCArKysrDQo+ID4gPiDCoG5ldC9z
dW5ycGMveHBydHJkbWEvdHJhbnNwb3J0LmMgfMKgwqDCoMKgMyArKy0NCj4gPiA+IMKgbmV0L3N1
bnJwYy94cHJ0cmRtYS92ZXJicy5jwqDCoMKgwqDCoHzCoMKgwqAxMyArKysrKysrKy0tLS0tDQo+
ID4gPiDCoDQgZmlsZXMgY2hhbmdlZCwgMTYgaW5zZXJ0aW9ucygrKSwgNiBkZWxldGlvbnMoLSkN
Cj4gPiA+IA0KPiA+ID4gZGlmZiAtLWdpdCBhL2luY2x1ZGUvbGludXgvc3VucnBjL3NjaGVkLmgN
Cj4gPiA+IGIvaW5jbHVkZS9saW51eC9zdW5ycGMvc2NoZWQuaA0KPiA+ID4gaW5kZXggMTM4MjJl
Ni4uZmNlYTE1OCAxMDA2NDQNCj4gPiA+IC0tLSBhL2luY2x1ZGUvbGludXgvc3VucnBjL3NjaGVk
LmgNCj4gPiA+ICsrKyBiL2luY2x1ZGUvbGludXgvc3VucnBjL3NjaGVkLmgNCj4gPiA+IEBAIC0x
MjcsNiArMTI3LDcgQEAgc3RydWN0IHJwY190YXNrX3NldHVwIHsNCj4gPiA+IMKgI2RlZmluZSBS
UENfVEFTS19USU1FT1VUCTB4MTAwMAkJLyogZmFpbA0KPiA+ID4gd2l0aA0KPiA+ID4gRVRJTUVE
T1VUIG9uIHRpbWVvdXQgKi8NCj4gPiA+IMKgI2RlZmluZSBSUENfVEFTS19OT0NPTk5FQ1QJMHgy
MDAwCQkvKg0KPiA+ID4gcmV0dXJuDQo+ID4gPiBFTk9UQ09OTiBpZiBub3QgY29ubmVjdGVkICov
DQo+ID4gPiDCoCNkZWZpbmUgUlBDX1RBU0tfTk9fUkVUUkFOU19USU1FT1VUCTB4NDAwMAkJDQo+
ID4gPiAvKg0KPiA+ID4gd2FpdCBmb3JldmVyIGZvciBhIHJlcGx5ICovDQo+ID4gPiArI2RlZmlu
ZSBSUENfVEFTS19OT19DT05HCTB4ODAwMAkJLyogc2tpcA0KPiA+ID4gY29uZ2VzdGlvbiBjb250
cm9sICovDQo+ID4gPiDCoA0KPiA+ID4gwqAjZGVmaW5lIFJQQ19UQVNLX1NPRlRQSU5HCShSUENf
VEFTS19TT0ZUIHwNCj4gPiA+IFJQQ19UQVNLX1NPRlRDT05OKQ0KPiA+ID4gwqANCj4gPiA+IEBA
IC0xMzcsNiArMTM4LDcgQEAgc3RydWN0IHJwY190YXNrX3NldHVwIHsNCj4gPiA+IMKgI2RlZmlu
ZSBSUENfSVNfU09GVCh0KQkJKCh0KS0+dGtfZmxhZ3MgJg0KPiA+ID4gKFJQQ19UQVNLX1NPRlR8
UlBDX1RBU0tfVElNRU9VVCkpDQo+ID4gPiDCoCNkZWZpbmUgUlBDX0lTX1NPRlRDT05OKHQpCSgo
dCktPnRrX2ZsYWdzICYNCj4gPiA+IFJQQ19UQVNLX1NPRlRDT05OKQ0KPiA+ID4gwqAjZGVmaW5l
IFJQQ19XQVNfU0VOVCh0KQkJKCh0KS0+dGtfZmxhZ3MgJg0KPiA+ID4gUlBDX1RBU0tfU0VOVCkN
Cj4gPiA+ICsjZGVmaW5lIFJQQ19TS0lQX0NPTkcodCkJKCh0KS0+dGtfZmxhZ3MgJg0KPiA+ID4g
UlBDX1RBU0tfTk9fQ09ORykNCj4gPiA+IMKgDQo+ID4gPiDCoCNkZWZpbmUgUlBDX1RBU0tfUlVO
TklORwkwDQo+ID4gPiDCoCNkZWZpbmUgUlBDX1RBU0tfUVVFVUVECQkxDQo+ID4gPiBkaWZmIC0t
Z2l0IGEvbmV0L3N1bnJwYy94cHJ0LmMgYi9uZXQvc3VucnBjL3hwcnQuYw0KPiA+ID4gaW5kZXgg
YjUzMGEyOC4uYTQ3N2VlNiAxMDA2NDQNCj4gPiA+IC0tLSBhL25ldC9zdW5ycGMveHBydC5jDQo+
ID4gPiArKysgYi9uZXQvc3VucnBjL3hwcnQuYw0KPiA+ID4gQEAgLTM5Miw2ICszOTIsMTAgQEAg
c3RhdGljIGlubGluZSB2b2lkIHhwcnRfcmVsZWFzZV93cml0ZShzdHJ1Y3QNCj4gPiA+IHJwY194
cHJ0ICp4cHJ0LCBzdHJ1Y3QgcnBjX3Rhc2sgKnRhDQo+ID4gPiDCoHsNCj4gPiA+IMKgCXN0cnVj
dCBycGNfcnFzdCAqcmVxID0gdGFzay0+dGtfcnFzdHA7DQo+ID4gPiDCoA0KPiA+ID4gKwlpZiAo
UlBDX1NLSVBfQ09ORyh0YXNrKSkgew0KPiA+ID4gKwkJcmVxLT5ycV9jb25nID0gMDsNCj4gPiA+
ICsJCXJldHVybiAxOw0KPiA+ID4gKwl9DQo+ID4gDQo+ID4gV2h5IG5vdCBqdXN0IGhhdmUgdGhl
IFJETUEgbGF5ZXIgY2FsbCB4cHJ0X3Jlc2VydmVfeHBydCgpIChhbmQNCj4gPiB4cHJ0X3JlbGVh
c2VfeHBydCgpKSBpZiB0aGlzIGZsYWcgaXMgc2V0PyBJdCBzZWVtcyB0byBtZSB0aGF0IHlvdQ0K
PiA+IHdpbGwNCj4gPiBuZWVkIHNvbWUga2luZCBvZiBleHRyYSBjb25nZXN0aW9uIGNvbnRyb2wg
aW4gdGhlIFJETUEgbGF5ZXIgYW55d2F5DQo+ID4gc2luY2UgeW91IG9ubHkgaGF2ZSBvbmUgcmVz
ZXJ2ZWQgY3JlZGl0IGZvciB0aGVzZSBwcml2aWxlZ2VkIHRhc2tzDQo+ID4gKG9yDQo+ID4gZGlk
IEkgbWlzcyB3aGVyZSB0aGF0IGlzIGJlaW5nIGdhdGVkPykuDQo+IA0KPiBUaGFua3MgZm9yIHRo
ZSByZXZpZXcuDQo+IA0KPiBTZWUgUlBDUkRNQV9JQV9SU1ZEX0NSRURJVCBpbiAxMS8xMi4gSXQn
cyBhIGhhY2sgSSdtIG5vdA0KPiB0ZXJyaWJseSBoYXBweSB3aXRoLg0KPiANCj4gU28sIEkgdGhp
bmsgeW91IGFyZSBzdWdnZXN0aW5nIHJlcGxhY2luZyB4cHJ0cmRtYSdzDQo+IC0+cmVzZXJ2ZV94
cHJ0IHdpdGggc29tZXRoaW5nIGxpa2U6DQo+IA0KPiBpbnQgeHBydF9yZG1hX3Jlc2VydmVfeHBy
dCh4cHJ0LCB0YXNrKQ0KPiB7DQo+IMKgwqDCoMKgwqDCoGlmIChSUENfU0tJUF9DT05HKHRhc2sp
KQ0KPiDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgcmV0dXJuIHhwcnRfcmVzZXJ2ZV94cHJ0KHhwcnQs
IHRhc2spOw0KPiDCoMKgwqDCoMKgwqByZXR1cm4geHBydF9yZXNlcnZlX3hwcnRfY29uZyh4cHJ0
LCB0YXNrKTsNCj4gfQ0KPiANCj4gYW5kIGxpa2V3aXNlIGZvciAtPnJlbGVhc2VfeHBydCA/DQoN
ClJpZ2h0Lg0KDQo+IFdoYXQgSSdkIHJlYWxseSBsaWtlIHRvIGRvIGlzIGhhdmUgdGhlIFJQQyBs
YXllcg0KPiBwcmV2ZW50IG1vcmUgdGhhbiBvbmUgUlBDIGF0IGEgdGltZSBmcm9tIHVzaW5nIHRo
ZQ0KPiBleHRyYSBjcmVkaXQsIGFuZCBzb21laG93IGVuc3VyZSB0aGF0IHRob3NlIFJQQ3MNCj4g
YXJlIGdvaW5nIHRvIGJlIHNob3J0LWxpdmVkIChTT0ZUIHwgU09GVENPTk4sDQo+IG1heWJlKS4N
Cg0KQ3JlZGl0cyBhcmUgYSB0cmFuc3BvcnQgbGF5ZXIgdGhpbmcsIHRob3VnaC4gVGhlcmUgaXMg
bm8gZXF1aXZhbGVudCBpbg0KdGhlIG5vbi1SRE1BIHdvcmxkLiBUQ1AgYW5kIFVEUCBzaG91bGQg
bm9ybWFsbHkgYm90aCBiZSBmaW5lIHdpdGgNCnRyYW5zbWl0dGluZyBhbiBleHRyYSBSUEMgY2Fs
bC4NCg0KRXZlbiB0aW1lb3V0cyBhcmUgYSB0cmFuc3BvcnQgbGF5ZXIgaXNzdWU7IHNlZSB0aGUg
cGF0Y2hlcyBJIHB1dCBvdXQNCnRoaXMgbW9ybmluZyBpbiBvcmRlciB0byByZWR1Y2UgdGhlIFRD
UCBjb25uZWN0aW9uIHRpbWVvdXRzIGFuZCBwdXQNCnRoZW0gbW9yZSBpbiBsaW5lIHdpdGggdGhl
IGxlYXNlIHBlcmlvZC4gU29tZXRoaW5nIGxpa2UgdGhhdCBtYWtlcyBubw0Kc2Vuc2UgaW4gdGhl
IFVEUCB3b3JsZCAobm8gY29ubmVjdGlvbnMpLCBvciBldmVuIGluIEFGX0xPQ0FMIChubw0Kcm91
dGluZyksIHdoaWNoIGlzIHdoeSBJIGFkZGVkIHRoZSBzZXRfY29ubmVjdGlvbl90aW1lb3V0KCkg
Y2FsbGJhY2suDQoNCj4gDQo+ID4gPiDCoAlpZiAocmVxLT5ycV9jb25nKQ0KPiA+ID4gwqAJCXJl
dHVybiAxOw0KPiA+ID4gwqAJZHByaW50aygiUlBDOiAlNXUgeHBydF9jd25kX2xpbWl0ZWQgY29u
ZyA9ICVsdSBjd25kID0NCj4gPiA+ICVsdVxuIiwNCj4gPiA+IGRpZmYgLS1naXQgYS9uZXQvc3Vu
cnBjL3hwcnRyZG1hL3RyYW5zcG9ydC5jDQo+ID4gPiBiL25ldC9zdW5ycGMveHBydHJkbWEvdHJh
bnNwb3J0LmMNCj4gPiA+IGluZGV4IDNhNWE4MDUuLjA3M2ZlY2QgMTAwNjQ0DQo+ID4gPiAtLS0g
YS9uZXQvc3VucnBjL3hwcnRyZG1hL3RyYW5zcG9ydC5jDQo+ID4gPiArKysgYi9uZXQvc3VucnBj
L3hwcnRyZG1hL3RyYW5zcG9ydC5jDQo+ID4gPiBAQCAtNTQ2LDcgKzU0Niw4IEBAIHN0YXRpYyB2
b2lkIHJwY3JkbWFfa2VlcGFsaXZlX3JlbGVhc2Uodm9pZA0KPiA+ID4gKmNhbGxkYXRhKQ0KPiA+
ID4gwqANCj4gPiA+IMKgCWRhdGEgPSB4cHJ0X2dldCh4cHJ0KTsNCj4gPiA+IMKgCW51bGxfdGFz
ayA9IHJwY19jYWxsX251bGxfaGVscGVyKHRhc2stPnRrX2NsaWVudCwgeHBydCwNCj4gPiA+IE5V
TEwsDQo+ID4gPiAtCQkJCQnCoFJQQ19UQVNLX1NPRlRQSU5HIHwNCj4gPiA+IFJQQ19UQVNLX0FT
WU5DLA0KPiA+ID4gKwkJCQkJwqBSUENfVEFTS19TT0ZUUElORyB8DQo+ID4gPiBSUENfVEFTS19B
U1lOQyB8DQo+ID4gPiArCQkJCQnCoFJQQ19UQVNLX05PX0NPTkcsDQo+ID4gPiDCoAkJCQkJwqAm
cnBjcmRtYV9rZWVwYWxpdmVfY2FsbA0KPiA+ID4gX29wcw0KPiA+ID4gLCBkYXRhKTsNCj4gPiA+
IMKgCWlmICghSVNfRVJSKG51bGxfdGFzaykpDQo+ID4gPiDCoAkJcnBjX3B1dF90YXNrKG51bGxf
dGFzayk7DQo+ID4gPiBkaWZmIC0tZ2l0IGEvbmV0L3N1bnJwYy94cHJ0cmRtYS92ZXJicy5jDQo+
ID4gPiBiL25ldC9zdW5ycGMveHBydHJkbWEvdmVyYnMuYw0KPiA+ID4gaW5kZXggODFjZDMxYS4u
ZDliNWZhNyAxMDA2NDQNCj4gPiA+IC0tLSBhL25ldC9zdW5ycGMveHBydHJkbWEvdmVyYnMuYw0K
PiA+ID4gKysrIGIvbmV0L3N1bnJwYy94cHJ0cmRtYS92ZXJicy5jDQo+ID4gPiBAQCAtMTM2LDE5
ICsxMzYsMjAgQEANCj4gPiA+IMKgc3RhdGljIHZvaWQNCj4gPiA+IMKgcnBjcmRtYV91cGRhdGVf
Z3JhbnRlZF9jcmVkaXRzKHN0cnVjdCBycGNyZG1hX3JlcCAqcmVwKQ0KPiA+ID4gwqB7DQo+ID4g
PiAtCXN0cnVjdCBycGNyZG1hX21zZyAqcm1zZ3AgPSByZG1hYl90b19tc2cocmVwLQ0KPiA+ID4g
PnJyX3JkbWFidWYpOw0KPiA+ID4gwqAJc3RydWN0IHJwY3JkbWFfYnVmZmVyICpidWZmZXIgPSAm
cmVwLT5ycl9yeHBydC0+cnhfYnVmOw0KPiA+ID4gKwlfX2JlMzIgKnAgPSByZXAtPnJyX3JkbWFi
dWYtPnJnX2Jhc2U7DQo+ID4gPiDCoAl1MzIgY3JlZGl0czsNCj4gPiA+IMKgDQo+ID4gPiDCoAlp
ZiAocmVwLT5ycl9sZW4gPCBSUENSRE1BX0hEUkxFTl9FUlIpDQo+ID4gPiDCoAkJcmV0dXJuOw0K
PiA+ID4gwqANCj4gPiA+IC0JY3JlZGl0cyA9IGJlMzJfdG9fY3B1KHJtc2dwLT5ybV9jcmVkaXQp
Ow0KPiA+ID4gKwljcmVkaXRzID0gYmUzMl90b19jcHVwKHAgKyAyKTsNCj4gPiA+ICsJaWYgKGNy
ZWRpdHMgPiBidWZmZXItPnJiX21heF9yZXF1ZXN0cykNCj4gPiA+ICsJCWNyZWRpdHMgPSBidWZm
ZXItPnJiX21heF9yZXF1ZXN0czsNCj4gPiA+ICsJLyogUmVzZXJ2ZSBvbmUgY3JlZGl0IGZvciBr
ZWVwYWxpdmUgcGluZyAqLw0KPiA+ID4gKwljcmVkaXRzLS07DQo+ID4gPiDCoAlpZiAoY3JlZGl0
cyA9PSAwKQ0KPiA+ID4gwqAJCWNyZWRpdHMgPSAxOwkvKiBkb24ndCBkZWFkbG9jayAqLw0KPiA+
ID4gLQllbHNlIGlmIChjcmVkaXRzID4gYnVmZmVyLT5yYl9tYXhfcmVxdWVzdHMpDQo+ID4gPiAt
CQljcmVkaXRzID0gYnVmZmVyLT5yYl9tYXhfcmVxdWVzdHM7DQo+ID4gPiAtDQo+ID4gPiDCoAlh
dG9taWNfc2V0KCZidWZmZXItPnJiX2NyZWRpdHMsIGNyZWRpdHMpOw0KPiA+ID4gwqB9DQo+ID4g
PiDCoA0KPiA+ID4gQEAgLTkxNSw2ICs5MTYsOCBAQCBzdHJ1Y3QgcnBjcmRtYV9yZXAgKg0KPiA+
ID4gwqAJc3RydWN0IHJwY3JkbWFfYnVmZmVyICpidWYgPSAmcl94cHJ0LT5yeF9idWY7DQo+ID4g
PiDCoAlpbnQgaSwgcmM7DQo+ID4gPiDCoA0KPiA+ID4gKwlpZiAocl94cHJ0LT5yeF9kYXRhLm1h
eF9yZXF1ZXN0cyA8IDIpDQo+ID4gPiArCQlyZXR1cm4gLUVJTlZBTDsNCj4gPiA+IMKgCWJ1Zi0+
cmJfbWF4X3JlcXVlc3RzID0gcl94cHJ0LT5yeF9kYXRhLm1heF9yZXF1ZXN0czsNCj4gPiA+IMKg
CWJ1Zi0+cmJfYmNfc3J2X21heF9yZXF1ZXN0cyA9IDA7DQo+ID4gPiDCoAlhdG9taWNfc2V0KCZi
dWYtPnJiX2NyZWRpdHMsIDEpOw0KPiA+ID4gDQo+ID4gPiAtLQ0KPiA+ID4gVG8gdW5zdWJzY3Jp
YmUgZnJvbSB0aGlzIGxpc3Q6IHNlbmQgdGhlIGxpbmUgInVuc3Vic2NyaWJlIGxpbnV4LQ0KPiA+
ID4gbmZzIg0KPiA+ID4gaW4NCj4gPiA+IHRoZSBib2R5IG9mIGEgbWVzc2FnZSB0byBtYWpvcmRv
bW9Admdlci5rZXJuZWwub3JnDQo+ID4gPiBNb3JlIG1ham9yZG9tbyBpbmZvIGF0wqDCoGh0dHA6
Ly92Z2VyLmtlcm5lbC5vcmcvbWFqb3Jkb21vLWluZm8uaHRtDQo+ID4gPiBsDQo+ID4gPiANCj4g
PiANCj4gPiAtLcKgDQo+ID4gDQo+ID4gDQo+ID4gCQ0KPiA+IAkNCj4gPiANCj4gPiANCj4gPiBU
cm9uZCBNeWtsZWJ1c3QNCj4gPiBQcmluY2lwYWwgU3lzdGVtIEFyY2hpdGVjdA0KPiA+IDQzMDAg
RWwgQ2FtaW5vIFJlYWwgfCBTdWl0ZSAxMDANCj4gPiBMb3MgQWx0b3MsIENBwqDCoDk0MDIyDQo+
ID4gVzogNjUwLTQyMi0zODAwDQo+ID4gQzogODAxLTkyMS00NTgzwqANCj4gPiB3d3cucHJpbWFy
eWRhdGEuY29tDQo+IA0KPiAtLQ0KPiBDaHVjayBMZXZlcg0KPiANCj4gDQo+IA0KLS0gDQoNCg0K
DQoJDQoJDQoNCg0KVHJvbmQgTXlrbGVidXN0DQpQcmluY2lwYWwgU3lzdGVtIEFyY2hpdGVjdA0K
NDMwMCBFbCBDYW1pbm8gUmVhbCB8IFN1aXRlIDEwMA0KTG9zIEFsdG9zLCBDQcKgwqA5NDAyMg0K
VzogNjUwLTQyMi0zODAwDQpDOiA4MDEtOTIxLTQ1ODPCoA0Kd3d3LnByaW1hcnlkYXRhLmNvbQ0K
DQoNCg0KDQo=
WARNING: multiple messages have this Message-ID (diff)
From: Trond Myklebust <trondmy-7I+n7zu2hftEKMMhf/gKZA@public.gmane.org>
To: "chuck.lever-QHcLZuEGTsvQT0dZR+AlfA@public.gmane.org"
<chuck.lever-QHcLZuEGTsvQT0dZR+AlfA@public.gmane.org>
Cc: "anna.schumaker-HgOvQuBEEgTQT0dZR+AlfA@public.gmane.org"
<anna.schumaker-HgOvQuBEEgTQT0dZR+AlfA@public.gmane.org>,
"linux-rdma-u79uwXL29TY76Z2rM5mHXA@public.gmane.org"
<linux-rdma-u79uwXL29TY76Z2rM5mHXA@public.gmane.org>,
"linux-nfs-u79uwXL29TY76Z2rM5mHXA@public.gmane.org"
<linux-nfs-u79uwXL29TY76Z2rM5mHXA@public.gmane.org>
Subject: Re: [PATCH v3 12/12] sunrpc: Allow keepalive ping on a credit-full transport
Date: Thu, 9 Feb 2017 00:48:54 +0000 [thread overview]
Message-ID: <1486601331.11028.5.camel@primarydata.com> (raw)
In-Reply-To: <9D6B8B44-9C23-427C-9E06-7C92302EB04D-QHcLZuEGTsvQT0dZR+AlfA@public.gmane.org>
On Wed, 2017-02-08 at 19:19 -0500, Chuck Lever wrote:
> > On Feb 8, 2017, at 7:05 PM, Trond Myklebust <trondmy@primarydata.co
> > m> wrote:
> >
> > On Wed, 2017-02-08 at 17:01 -0500, Chuck Lever wrote:
> > > Allow RPC-over-RDMA to send NULL pings even when the transport
> > > has
> > > hit its credit limit. One RPC-over-RDMA credit is reserved for
> > > operations like keepalive.
> > >
> > > For transports that convey NFSv4, it seems like lease renewal
> > > would
> > > also be a candidate for using a priority transport slot. I'd like
> > > to
> > > see a mechanism better than RPCRDMA_PRIORITY that can ensure only
> > > one priority operation is in use at a time.
> > >
> > > Signed-off-by: Chuck Lever <chuck.lever@oracle.com>
> > > ---
> > > include/linux/sunrpc/sched.h | 2 ++
> > > net/sunrpc/xprt.c | 4 ++++
> > > net/sunrpc/xprtrdma/transport.c | 3 ++-
> > > net/sunrpc/xprtrdma/verbs.c | 13 ++++++++-----
> > > 4 files changed, 16 insertions(+), 6 deletions(-)
> > >
> > > diff --git a/include/linux/sunrpc/sched.h
> > > b/include/linux/sunrpc/sched.h
> > > index 13822e6..fcea158 100644
> > > --- a/include/linux/sunrpc/sched.h
> > > +++ b/include/linux/sunrpc/sched.h
> > > @@ -127,6 +127,7 @@ struct rpc_task_setup {
> > > #define RPC_TASK_TIMEOUT 0x1000 /* fail
> > > with
> > > ETIMEDOUT on timeout */
> > > #define RPC_TASK_NOCONNECT 0x2000 /*
> > > return
> > > ENOTCONN if not connected */
> > > #define RPC_TASK_NO_RETRANS_TIMEOUT 0x4000
> > > /*
> > > wait forever for a reply */
> > > +#define RPC_TASK_NO_CONG 0x8000 /* skip
> > > congestion control */
> > >
> > > #define RPC_TASK_SOFTPING (RPC_TASK_SOFT |
> > > RPC_TASK_SOFTCONN)
> > >
> > > @@ -137,6 +138,7 @@ struct rpc_task_setup {
> > > #define RPC_IS_SOFT(t) ((t)->tk_flags &
> > > (RPC_TASK_SOFT|RPC_TASK_TIMEOUT))
> > > #define RPC_IS_SOFTCONN(t) ((t)->tk_flags &
> > > RPC_TASK_SOFTCONN)
> > > #define RPC_WAS_SENT(t) ((t)->tk_flags &
> > > RPC_TASK_SENT)
> > > +#define RPC_SKIP_CONG(t) ((t)->tk_flags &
> > > RPC_TASK_NO_CONG)
> > >
> > > #define RPC_TASK_RUNNING 0
> > > #define RPC_TASK_QUEUED 1
> > > diff --git a/net/sunrpc/xprt.c b/net/sunrpc/xprt.c
> > > index b530a28..a477ee6 100644
> > > --- a/net/sunrpc/xprt.c
> > > +++ b/net/sunrpc/xprt.c
> > > @@ -392,6 +392,10 @@ static inline void xprt_release_write(struct
> > > rpc_xprt *xprt, struct rpc_task *ta
> > > {
> > > struct rpc_rqst *req = task->tk_rqstp;
> > >
> > > + if (RPC_SKIP_CONG(task)) {
> > > + req->rq_cong = 0;
> > > + return 1;
> > > + }
> >
> > Why not just have the RDMA layer call xprt_reserve_xprt() (and
> > xprt_release_xprt()) if this flag is set? It seems to me that you
> > will
> > need some kind of extra congestion control in the RDMA layer anyway
> > since you only have one reserved credit for these privileged tasks
> > (or
> > did I miss where that is being gated?).
>
> Thanks for the review.
>
> See RPCRDMA_IA_RSVD_CREDIT in 11/12. It's a hack I'm not
> terribly happy with.
>
> So, I think you are suggesting replacing xprtrdma's
> ->reserve_xprt with something like:
>
> int xprt_rdma_reserve_xprt(xprt, task)
> {
> if (RPC_SKIP_CONG(task))
> return xprt_reserve_xprt(xprt, task);
> return xprt_reserve_xprt_cong(xprt, task);
> }
>
> and likewise for ->release_xprt ?
Right.
> What I'd really like to do is have the RPC layer
> prevent more than one RPC at a time from using the
> extra credit, and somehow ensure that those RPCs
> are going to be short-lived (SOFT | SOFTCONN,
> maybe).
Credits are a transport layer thing, though. There is no equivalent in
the non-RDMA world. TCP and UDP should normally both be fine with
transmitting an extra RPC call.
Even timeouts are a transport layer issue; see the patches I put out
this morning in order to reduce the TCP connection timeouts and put
them more in line with the lease period. Something like that makes no
sense in the UDP world (no connections), or even in AF_LOCAL (no
routing), which is why I added the set_connection_timeout() callback.
>
> > > if (req->rq_cong)
> > > return 1;
> > > dprintk("RPC: %5u xprt_cwnd_limited cong = %lu cwnd =
> > > %lu\n",
> > > diff --git a/net/sunrpc/xprtrdma/transport.c
> > > b/net/sunrpc/xprtrdma/transport.c
> > > index 3a5a805..073fecd 100644
> > > --- a/net/sunrpc/xprtrdma/transport.c
> > > +++ b/net/sunrpc/xprtrdma/transport.c
> > > @@ -546,7 +546,8 @@ static void rpcrdma_keepalive_release(void
> > > *calldata)
> > >
> > > data = xprt_get(xprt);
> > > null_task = rpc_call_null_helper(task->tk_client, xprt,
> > > NULL,
> > > - RPC_TASK_SOFTPING |
> > > RPC_TASK_ASYNC,
> > > + RPC_TASK_SOFTPING |
> > > RPC_TASK_ASYNC |
> > > + RPC_TASK_NO_CONG,
> > > &rpcrdma_keepalive_call
> > > _ops
> > > , data);
> > > if (!IS_ERR(null_task))
> > > rpc_put_task(null_task);
> > > diff --git a/net/sunrpc/xprtrdma/verbs.c
> > > b/net/sunrpc/xprtrdma/verbs.c
> > > index 81cd31a..d9b5fa7 100644
> > > --- a/net/sunrpc/xprtrdma/verbs.c
> > > +++ b/net/sunrpc/xprtrdma/verbs.c
> > > @@ -136,19 +136,20 @@
> > > static void
> > > rpcrdma_update_granted_credits(struct rpcrdma_rep *rep)
> > > {
> > > - struct rpcrdma_msg *rmsgp = rdmab_to_msg(rep-
> > > >rr_rdmabuf);
> > > struct rpcrdma_buffer *buffer = &rep->rr_rxprt->rx_buf;
> > > + __be32 *p = rep->rr_rdmabuf->rg_base;
> > > u32 credits;
> > >
> > > if (rep->rr_len < RPCRDMA_HDRLEN_ERR)
> > > return;
> > >
> > > - credits = be32_to_cpu(rmsgp->rm_credit);
> > > + credits = be32_to_cpup(p + 2);
> > > + if (credits > buffer->rb_max_requests)
> > > + credits = buffer->rb_max_requests;
> > > + /* Reserve one credit for keepalive ping */
> > > + credits--;
> > > if (credits == 0)
> > > credits = 1; /* don't deadlock */
> > > - else if (credits > buffer->rb_max_requests)
> > > - credits = buffer->rb_max_requests;
> > > -
> > > atomic_set(&buffer->rb_credits, credits);
> > > }
> > >
> > > @@ -915,6 +916,8 @@ struct rpcrdma_rep *
> > > struct rpcrdma_buffer *buf = &r_xprt->rx_buf;
> > > int i, rc;
> > >
> > > + if (r_xprt->rx_data.max_requests < 2)
> > > + return -EINVAL;
> > > buf->rb_max_requests = r_xprt->rx_data.max_requests;
> > > buf->rb_bc_srv_max_requests = 0;
> > > atomic_set(&buf->rb_credits, 1);
> > >
> > > --
> > > To unsubscribe from this list: send the line "unsubscribe linux-
> > > nfs"
> > > in
> > > the body of a message to majordomo@vger.kernel.org
> > > More majordomo info at http://vger.kernel.org/majordomo-info.htm
> > > l
> > >
> >
> > --
> >
> >
> >
> >
> >
> >
> > Trond Myklebust
> > Principal System Architect
> > 4300 El Camino Real | Suite 100
> > Los Altos, CA 94022
> > W: 650-422-3800
> > C: 801-921-4583
> > www.primarydata.com
>
> --
> Chuck Lever
>
>
>
--
Trond Myklebust
Principal System Architect
4300 El Camino Real | Suite 100
Los Altos, CA 94022
W: 650-422-3800
C: 801-921-4583
www.primarydata.com
next prev parent reply other threads:[~2017-02-09 0:52 UTC|newest]
Thread overview: 42+ messages / expand[flat|nested] mbox.gz Atom feed top
2017-02-08 21:59 [PATCH v3 00/12] NFS/RDMA client-side patches for 4.11 Chuck Lever
2017-02-08 21:59 ` Chuck Lever
2017-02-08 21:59 ` [PATCH v3 01/12] xprtrdma: Fix Read chunk padding Chuck Lever
2017-02-08 21:59 ` Chuck Lever
2017-02-08 21:59 ` [PATCH v3 02/12] xprtrdma: Per-connection pad optimization Chuck Lever
2017-02-08 21:59 ` Chuck Lever
2017-02-08 22:00 ` [PATCH v3 03/12] xprtrdma: Disable pad optimization by default Chuck Lever
2017-02-08 22:00 ` Chuck Lever
2017-02-08 22:00 ` [PATCH v3 04/12] xprtrdma: Reduce required number of send SGEs Chuck Lever
2017-02-08 22:00 ` Chuck Lever
2017-02-08 22:00 ` [PATCH v3 05/12] xprtrdma: Shrink send SGEs array Chuck Lever
2017-02-08 22:00 ` Chuck Lever
2017-02-08 22:00 ` [PATCH v3 06/12] xprtrdma: Properly recover FRWRs with in-flight FASTREG WRs Chuck Lever
2017-02-08 22:00 ` Chuck Lever
2017-02-08 22:00 ` [PATCH v3 07/12] xprtrdma: Handle stale connection rejection Chuck Lever
2017-02-08 22:00 ` Chuck Lever
2017-02-08 22:00 ` [PATCH v3 08/12] xprtrdma: Refactor management of mw_list field Chuck Lever
2017-02-08 22:00 ` Chuck Lever
2017-02-08 22:00 ` [PATCH v3 09/12] sunrpc: Allow xprt->ops->timer method to sleep Chuck Lever
2017-02-08 22:00 ` Chuck Lever
2017-02-08 23:48 ` Trond Myklebust
2017-02-08 23:48 ` Trond Myklebust
2017-02-08 22:00 ` [PATCH v3 10/12] sunrpc: Enable calls to rpc_call_null_helper() from other modules Chuck Lever
2017-02-08 22:00 ` Chuck Lever
2017-02-08 22:01 ` [PATCH v3 11/12] xprtrdma: Detect unreachable NFS/RDMA servers more reliably Chuck Lever
2017-02-08 22:01 ` Chuck Lever
2017-02-08 22:01 ` [PATCH v3 12/12] sunrpc: Allow keepalive ping on a credit-full transport Chuck Lever
2017-02-08 22:01 ` Chuck Lever
2017-02-09 0:05 ` Trond Myklebust
2017-02-09 0:05 ` Trond Myklebust
2017-02-09 0:19 ` Chuck Lever
2017-02-09 0:19 ` Chuck Lever
2017-02-09 0:48 ` Trond Myklebust [this message]
2017-02-09 0:48 ` Trond Myklebust
2017-02-09 15:37 ` Chuck Lever
2017-02-09 15:37 ` Chuck Lever
2017-02-09 19:42 ` Chuck Lever
2017-02-09 19:42 ` Chuck Lever
2017-02-09 20:13 ` Trond Myklebust
2017-02-09 20:13 ` Trond Myklebust
2017-02-09 20:39 ` Chuck Lever
2017-02-09 20:39 ` Chuck Lever
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=1486601331.11028.5.camel@primarydata.com \
--to=trondmy@primarydata.com \
--cc=anna.schumaker@netapp.com \
--cc=chuck.lever@oracle.com \
--cc=linux-nfs@vger.kernel.org \
--cc=linux-rdma@vger.kernel.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.