diff for duplicates of <1486671236.5570.4.camel@primarydata.com> diff --git a/a/1.txt b/N1/1.txt index b6e0494..9b4650b 100644 --- a/a/1.txt +++ b/N1/1.txt @@ -1,94 +1,153 @@ -T24gVGh1LCAyMDE3LTAyLTA5IGF0IDE0OjQyIC0wNTAwLCBDaHVjayBMZXZlciB3cm90ZToNCj4g -PiBPbiBGZWIgOSwgMjAxNywgYXQgMTA6MzcgQU0sIENodWNrIExldmVyIDxjaHVjay5sZXZlckBv -cmFjbGUuY29tPg0KPiA+IHdyb3RlOg0KPiA+IA0KPiA+ID4gDQo+ID4gPiBPbiBGZWIgOCwgMjAx -NywgYXQgNzo0OCBQTSwgVHJvbmQgTXlrbGVidXN0IDx0cm9uZG15QHByaW1hcnlkYXRhLg0KPiA+ -ID4gY29tPiB3cm90ZToNCj4gPiA+IA0KPiA+ID4gT24gV2VkLCAyMDE3LTAyLTA4IGF0IDE5OjE5 -IC0wNTAwLCBDaHVjayBMZXZlciB3cm90ZToNCj4gPiA+ID4gPiBPbiBGZWIgOCwgMjAxNywgYXQg -NzowNSBQTSwgVHJvbmQgTXlrbGVidXN0IDx0cm9uZG15QHByaW1hcnlkDQo+ID4gPiA+ID4gYXRh -LmNvDQo+ID4gPiA+ID4gbT4gd3JvdGU6DQo+ID4gPiA+ID4gDQo+ID4gPiA+ID4gT24gV2VkLCAy -MDE3LTAyLTA4IGF0IDE3OjAxIC0wNTAwLCBDaHVjayBMZXZlciB3cm90ZToNCj4gPiA+ID4gPiA+ -IEFsbG93IFJQQy1vdmVyLVJETUEgdG8gc2VuZCBOVUxMIHBpbmdzIGV2ZW4gd2hlbiB0aGUNCj4g -PiA+ID4gPiA+IHRyYW5zcG9ydA0KPiA+ID4gPiA+ID4gaGFzDQo+ID4gPiA+ID4gPiBoaXQgaXRz -IGNyZWRpdCBsaW1pdC4gT25lIFJQQy1vdmVyLVJETUEgY3JlZGl0IGlzIHJlc2VydmVkDQo+ID4g -PiA+ID4gPiBmb3INCj4gPiA+ID4gPiA+IG9wZXJhdGlvbnMgbGlrZSBrZWVwYWxpdmUuDQo+ID4g -PiA+ID4gPiANCj4gPiA+ID4gPiA+IEZvciB0cmFuc3BvcnRzIHRoYXQgY29udmV5IE5GU3Y0LCBp -dCBzZWVtcyBsaWtlIGxlYXNlDQo+ID4gPiA+ID4gPiByZW5ld2FsDQo+ID4gPiA+ID4gPiB3b3Vs -ZA0KPiA+ID4gPiA+ID4gYWxzbyBiZSBhIGNhbmRpZGF0ZSBmb3IgdXNpbmcgYSBwcmlvcml0eSB0 -cmFuc3BvcnQgc2xvdC4NCj4gPiA+ID4gPiA+IEknZCBsaWtlDQo+ID4gPiA+ID4gPiB0bw0KPiA+ -ID4gPiA+ID4gc2VlIGEgbWVjaGFuaXNtIGJldHRlciB0aGFuIFJQQ1JETUFfUFJJT1JJVFkgdGhh -dCBjYW4NCj4gPiA+ID4gPiA+IGVuc3VyZSBvbmx5DQo+ID4gPiA+ID4gPiBvbmUgcHJpb3JpdHkg -b3BlcmF0aW9uIGlzIGluIHVzZSBhdCBhIHRpbWUuDQo+ID4gPiA+ID4gPiANCj4gPiA+ID4gPiA+ -IFNpZ25lZC1vZmYtYnk6IENodWNrIExldmVyIDxjaHVjay5sZXZlckBvcmFjbGUuY29tPg0KPiA+ -ID4gPiA+ID4gLS0tDQo+ID4gPiA+ID4gPiBpbmNsdWRlL2xpbnV4L3N1bnJwYy9zY2hlZC5owqDC -oMKgwqB8wqDCoMKgwqAyICsrDQo+ID4gPiA+ID4gPiBuZXQvc3VucnBjL3hwcnQuY8KgwqDCoMKg -wqDCoMKgwqDCoMKgwqDCoMKgwqDCoHzCoMKgwqDCoDQgKysrKw0KPiA+ID4gPiA+ID4gbmV0L3N1 -bnJwYy94cHJ0cmRtYS90cmFuc3BvcnQuYyB8wqDCoMKgwqAzICsrLQ0KPiA+ID4gPiA+ID4gbmV0 -L3N1bnJwYy94cHJ0cmRtYS92ZXJicy5jwqDCoMKgwqDCoHzCoMKgwqAxMyArKysrKysrKy0tLS0t -DQo+ID4gPiA+ID4gPiA0IGZpbGVzIGNoYW5nZWQsIDE2IGluc2VydGlvbnMoKyksIDYgZGVsZXRp -b25zKC0pDQo+ID4gPiA+ID4gPiANCj4gPiA+ID4gPiA+IGRpZmYgLS1naXQgYS9pbmNsdWRlL2xp -bnV4L3N1bnJwYy9zY2hlZC5oDQo+ID4gPiA+ID4gPiBiL2luY2x1ZGUvbGludXgvc3VucnBjL3Nj -aGVkLmgNCj4gPiA+ID4gPiA+IGluZGV4IDEzODIyZTYuLmZjZWExNTggMTAwNjQ0DQo+ID4gPiA+ -ID4gPiAtLS0gYS9pbmNsdWRlL2xpbnV4L3N1bnJwYy9zY2hlZC5oDQo+ID4gPiA+ID4gPiArKysg -Yi9pbmNsdWRlL2xpbnV4L3N1bnJwYy9zY2hlZC5oDQo+ID4gPiA+ID4gPiBAQCAtMTI3LDYgKzEy -Nyw3IEBAIHN0cnVjdCBycGNfdGFza19zZXR1cCB7DQo+ID4gPiA+ID4gPiAjZGVmaW5lIFJQQ19U -QVNLX1RJTUVPVVQJMHgxMDAwCQkvKg0KPiA+ID4gPiA+ID4gZmFpbA0KPiA+ID4gPiA+ID4gd2l0 -aA0KPiA+ID4gPiA+ID4gRVRJTUVET1VUIG9uIHRpbWVvdXQgKi8NCj4gPiA+ID4gPiA+ICNkZWZp -bmUgUlBDX1RBU0tfTk9DT05ORUNUCTB4MjAwMAkJLyoNCj4gPiA+ID4gPiA+IHJldHVybg0KPiA+ -ID4gPiA+ID4gRU5PVENPTk4gaWYgbm90IGNvbm5lY3RlZCAqLw0KPiA+ID4gPiA+ID4gI2RlZmlu -ZSBSUENfVEFTS19OT19SRVRSQU5TX1RJTUVPVVQJMHg0MDAwCQkNCj4gPiA+ID4gPiA+IC8qDQo+ -ID4gPiA+ID4gPiB3YWl0IGZvcmV2ZXIgZm9yIGEgcmVwbHkgKi8NCj4gPiA+ID4gPiA+ICsjZGVm -aW5lIFJQQ19UQVNLX05PX0NPTkcJMHg4MDAwCQkvKg0KPiA+ID4gPiA+ID4gc2tpcA0KPiA+ID4g -PiA+ID4gY29uZ2VzdGlvbiBjb250cm9sICovDQo+ID4gPiA+ID4gPiANCj4gPiA+ID4gPiA+ICNk -ZWZpbmUgUlBDX1RBU0tfU09GVFBJTkcJKFJQQ19UQVNLX1NPRlQgfA0KPiA+ID4gPiA+ID4gUlBD -X1RBU0tfU09GVENPTk4pDQo+ID4gPiA+ID4gPiANCj4gPiA+ID4gPiA+IEBAIC0xMzcsNiArMTM4 -LDcgQEAgc3RydWN0IHJwY190YXNrX3NldHVwIHsNCj4gPiA+ID4gPiA+ICNkZWZpbmUgUlBDX0lT -X1NPRlQodCkJCSgodCktPnRrX2ZsYWdzICYNCj4gPiA+ID4gPiA+IChSUENfVEFTS19TT0ZUfFJQ -Q19UQVNLX1RJTUVPVVQpKQ0KPiA+ID4gPiA+ID4gI2RlZmluZSBSUENfSVNfU09GVENPTk4odCkJ -KCh0KS0+dGtfZmxhZ3MgJg0KPiA+ID4gPiA+ID4gUlBDX1RBU0tfU09GVENPTk4pDQo+ID4gPiA+ -ID4gPiAjZGVmaW5lIFJQQ19XQVNfU0VOVCh0KQkJKCh0KS0+dGtfZmxhZ3MgJg0KPiA+ID4gPiA+ -ID4gUlBDX1RBU0tfU0VOVCkNCj4gPiA+ID4gPiA+ICsjZGVmaW5lIFJQQ19TS0lQX0NPTkcodCkJ -KCh0KS0+dGtfZmxhZ3MgJg0KPiA+ID4gPiA+ID4gUlBDX1RBU0tfTk9fQ09ORykNCj4gPiA+ID4g -PiA+IA0KPiA+ID4gPiA+ID4gI2RlZmluZSBSUENfVEFTS19SVU5OSU5HCTANCj4gPiA+ID4gPiA+ -ICNkZWZpbmUgUlBDX1RBU0tfUVVFVUVECQkxDQo+ID4gPiA+ID4gPiBkaWZmIC0tZ2l0IGEvbmV0 -L3N1bnJwYy94cHJ0LmMgYi9uZXQvc3VucnBjL3hwcnQuYw0KPiA+ID4gPiA+ID4gaW5kZXggYjUz -MGEyOC4uYTQ3N2VlNiAxMDA2NDQNCj4gPiA+ID4gPiA+IC0tLSBhL25ldC9zdW5ycGMveHBydC5j -DQo+ID4gPiA+ID4gPiArKysgYi9uZXQvc3VucnBjL3hwcnQuYw0KPiA+ID4gPiA+ID4gQEAgLTM5 -Miw2ICszOTIsMTAgQEAgc3RhdGljIGlubGluZSB2b2lkDQo+ID4gPiA+ID4gPiB4cHJ0X3JlbGVh -c2Vfd3JpdGUoc3RydWN0DQo+ID4gPiA+ID4gPiBycGNfeHBydCAqeHBydCwgc3RydWN0IHJwY190 -YXNrICp0YQ0KPiA+ID4gPiA+ID4gew0KPiA+ID4gPiA+ID4gCXN0cnVjdCBycGNfcnFzdCAqcmVx -ID0gdGFzay0+dGtfcnFzdHA7DQo+ID4gPiA+ID4gPiANCj4gPiA+ID4gPiA+ICsJaWYgKFJQQ19T -S0lQX0NPTkcodGFzaykpIHsNCj4gPiA+ID4gPiA+ICsJCXJlcS0+cnFfY29uZyA9IDA7DQo+ID4g -PiA+ID4gPiArCQlyZXR1cm4gMTsNCj4gPiA+ID4gPiA+ICsJfQ0KPiA+ID4gPiA+IA0KPiA+ID4g -PiA+IFdoeSBub3QganVzdCBoYXZlIHRoZSBSRE1BIGxheWVyIGNhbGwgeHBydF9yZXNlcnZlX3hw -cnQoKQ0KPiA+ID4gPiA+IChhbmQNCj4gPiA+ID4gPiB4cHJ0X3JlbGVhc2VfeHBydCgpKSBpZiB0 -aGlzIGZsYWcgaXMgc2V0PyBJdCBzZWVtcyB0byBtZSB0aGF0DQo+ID4gPiA+ID4geW91DQo+ID4g -PiA+ID4gd2lsbA0KPiA+ID4gPiA+IG5lZWQgc29tZSBraW5kIG9mIGV4dHJhIGNvbmdlc3Rpb24g -Y29udHJvbCBpbiB0aGUgUkRNQSBsYXllcg0KPiA+ID4gPiA+IGFueXdheQ0KPiA+ID4gPiA+IHNp -bmNlIHlvdSBvbmx5IGhhdmUgb25lIHJlc2VydmVkIGNyZWRpdCBmb3IgdGhlc2UgcHJpdmlsZWdl -ZA0KPiA+ID4gPiA+IHRhc2tzDQo+ID4gPiA+ID4gKG9yDQo+ID4gPiA+ID4gZGlkIEkgbWlzcyB3 -aGVyZSB0aGF0IGlzIGJlaW5nIGdhdGVkPykuDQo+ID4gPiA+IA0KPiA+ID4gPiBUaGFua3MgZm9y -IHRoZSByZXZpZXcuDQo+ID4gPiA+IA0KPiA+ID4gPiBTZWUgUlBDUkRNQV9JQV9SU1ZEX0NSRURJ -VCBpbiAxMS8xMi4gSXQncyBhIGhhY2sgSSdtIG5vdA0KPiA+ID4gPiB0ZXJyaWJseSBoYXBweSB3 -aXRoLg0KPiA+ID4gPiANCj4gPiA+ID4gU28sIEkgdGhpbmsgeW91IGFyZSBzdWdnZXN0aW5nIHJl -cGxhY2luZyB4cHJ0cmRtYSdzDQo+ID4gPiA+IC0+cmVzZXJ2ZV94cHJ0IHdpdGggc29tZXRoaW5n -IGxpa2U6DQo+ID4gPiA+IA0KPiA+ID4gPiBpbnQgeHBydF9yZG1hX3Jlc2VydmVfeHBydCh4cHJ0 -LCB0YXNrKQ0KPiA+ID4gPiB7DQo+ID4gPiA+IMKgwqDCoMKgwqBpZiAoUlBDX1NLSVBfQ09ORyh0 -YXNrKSkNCj4gPiA+ID4gwqDCoMKgwqDCoMKgwqDCoMKgwqByZXR1cm4geHBydF9yZXNlcnZlX3hw -cnQoeHBydCwgdGFzayk7DQo+ID4gPiA+IMKgwqDCoMKgwqByZXR1cm4geHBydF9yZXNlcnZlX3hw -cnRfY29uZyh4cHJ0LCB0YXNrKTsNCj4gPiA+ID4gfQ0KPiA+ID4gPiANCj4gPiA+ID4gYW5kIGxp -a2V3aXNlIGZvciAtPnJlbGVhc2VfeHBydCA/DQo+ID4gPiANCj4gPiA+IFJpZ2h0Lg0KPiANCj4g -VGhpcyBzZWVtcyB0byB3b3JrIGZpbmUgZm9yIHRoZSBub3JtYWwgY2FzZXMuDQo+IA0KPiBJJ20g -Y29uZnVzZWQgYWJvdXQgaG93IHRvIGNvbnN0cnVjdCB4cHJ0X3JkbWFfcmVsZWFzZV94cHJ0KCkN -Cj4gc28gaXQgbmV2ZXIgcmVsZWFzZXMgYSBub3JtYWwgUlBDIHRhc2sgd2hlbiBhIFNLSVBfQ09O -Rw0KPiB0YXNrIGNvbXBsZXRlcyBhbmQgdGhlIGNyZWRpdCBsaW1pdCBpcyBzdGlsbCBmdWxsLg0K -PiANCj4gSWYgaXQgc2hvdWxkIHNlbmQgYSBub3JtYWwgdGFzayB1c2luZyB0aGUgcmVzZXJ2ZWQg -Y3JlZGl0DQo+IGFuZCB0aGF0IHRhc2sgaGFuZ3MgdG9vLCB3ZSdyZSBpbiBleGFjdGx5IHRoZSBw -b3NpdGlvbg0KPiB3ZSB3YW50ZWQgdG8gYXZvaWQuDQo+IA0KPiBNeSBvcmlnaW5hbCBzb2x1dGlv -biBtaWdodCBoYXZlIGhhZCBhIHNpbWlsYXIgcHJvYmxlbSwNCj4gY29tZSB0byB0aGluayBvZiBp -dC4NCj4gDQo+IA0KDQpUaGF0J3MgdHJ1ZS4uLiBZb3UgbWF5IG5lZWQgdG8gc2V0IHVwIGEgc2Vw -YXJhdGUgd2FpdHF1ZXVlIHRoYXQgaXMNCnJlc2VydmVkIGZvciBTS0lQX0NPTkcgdGFza3MuIEFn -YWluLCBpdCBtYWtlcyBzZW5zZSB0byBrZWVwIHRoYXQgaW4gdGhlDQpSRE1BIGNvZGUuDQoNCi0t -IA0KDQoNCgkNCgkNCg0KDQpUcm9uZCBNeWtsZWJ1c3QNClByaW5jaXBhbCBTeXN0ZW0gQXJjaGl0 -ZWN0DQo0MzAwIEVsIENhbWlubyBSZWFsIHwgU3VpdGUgMTAwDQpMb3MgQWx0b3MsIENBwqDCoDk0 -MDIyDQpXOiA2NTAtNDIyLTM4MDANCkM6IDgwMS05MjEtNDU4M8KgDQp3d3cucHJpbWFyeWRhdGEu -Y29tDQoNCg0KDQo= +On Thu, 2017-02-09 at 14:42 -0500, Chuck Lever wrote: +> > On Feb 9, 2017, at 10:37 AM, Chuck Lever <chuck.lever@oracle.com> +> > wrote: +> > +> > > +> > > On Feb 8, 2017, at 7:48 PM, Trond Myklebust <trondmy@primarydata. +> > > com> wrote: +> > > +> > > On Wed, 2017-02-08 at 19:19 -0500, Chuck Lever wrote: +> > > > > On Feb 8, 2017, at 7:05 PM, Trond Myklebust <trondmy@primaryd +> > > > > ata.co +> > > > > m> wrote: +> > > > > +> > > > > On Wed, 2017-02-08 at 17:01 -0500, Chuck Lever wrote: +> > > > > > Allow RPC-over-RDMA to send NULL pings even when the +> > > > > > transport +> > > > > > has +> > > > > > hit its credit limit. One RPC-over-RDMA credit is reserved +> > > > > > for +> > > > > > operations like keepalive. +> > > > > > +> > > > > > For transports that convey NFSv4, it seems like lease +> > > > > > renewal +> > > > > > would +> > > > > > also be a candidate for using a priority transport slot. +> > > > > > I'd like +> > > > > > to +> > > > > > see a mechanism better than RPCRDMA_PRIORITY that can +> > > > > > ensure only +> > > > > > one priority operation is in use at a time. +> > > > > > +> > > > > > Signed-off-by: Chuck Lever <chuck.lever@oracle.com> +> > > > > > --- +> > > > > > include/linux/sunrpc/sched.h | 2 ++ +> > > > > > net/sunrpc/xprt.c | 4 ++++ +> > > > > > net/sunrpc/xprtrdma/transport.c | 3 ++- +> > > > > > net/sunrpc/xprtrdma/verbs.c | 13 ++++++++----- +> > > > > > 4 files changed, 16 insertions(+), 6 deletions(-) +> > > > > > +> > > > > > diff --git a/include/linux/sunrpc/sched.h +> > > > > > b/include/linux/sunrpc/sched.h +> > > > > > index 13822e6..fcea158 100644 +> > > > > > --- a/include/linux/sunrpc/sched.h +> > > > > > +++ b/include/linux/sunrpc/sched.h +> > > > > > @@ -127,6 +127,7 @@ struct rpc_task_setup { +> > > > > > #define RPC_TASK_TIMEOUT 0x1000 /* +> > > > > > fail +> > > > > > with +> > > > > > ETIMEDOUT on timeout */ +> > > > > > #define RPC_TASK_NOCONNECT 0x2000 /* +> > > > > > return +> > > > > > ENOTCONN if not connected */ +> > > > > > #define RPC_TASK_NO_RETRANS_TIMEOUT 0x4000 +> > > > > > /* +> > > > > > wait forever for a reply */ +> > > > > > +#define RPC_TASK_NO_CONG 0x8000 /* +> > > > > > skip +> > > > > > congestion control */ +> > > > > > +> > > > > > #define RPC_TASK_SOFTPING (RPC_TASK_SOFT | +> > > > > > RPC_TASK_SOFTCONN) +> > > > > > +> > > > > > @@ -137,6 +138,7 @@ struct rpc_task_setup { +> > > > > > #define RPC_IS_SOFT(t) ((t)->tk_flags & +> > > > > > (RPC_TASK_SOFT|RPC_TASK_TIMEOUT)) +> > > > > > #define RPC_IS_SOFTCONN(t) ((t)->tk_flags & +> > > > > > RPC_TASK_SOFTCONN) +> > > > > > #define RPC_WAS_SENT(t) ((t)->tk_flags & +> > > > > > RPC_TASK_SENT) +> > > > > > +#define RPC_SKIP_CONG(t) ((t)->tk_flags & +> > > > > > RPC_TASK_NO_CONG) +> > > > > > +> > > > > > #define RPC_TASK_RUNNING 0 +> > > > > > #define RPC_TASK_QUEUED 1 +> > > > > > diff --git a/net/sunrpc/xprt.c b/net/sunrpc/xprt.c +> > > > > > index b530a28..a477ee6 100644 +> > > > > > --- a/net/sunrpc/xprt.c +> > > > > > +++ b/net/sunrpc/xprt.c +> > > > > > @@ -392,6 +392,10 @@ static inline void +> > > > > > xprt_release_write(struct +> > > > > > rpc_xprt *xprt, struct rpc_task *ta +> > > > > > { +> > > > > > struct rpc_rqst *req = task->tk_rqstp; +> > > > > > +> > > > > > + if (RPC_SKIP_CONG(task)) { +> > > > > > + req->rq_cong = 0; +> > > > > > + return 1; +> > > > > > + } +> > > > > +> > > > > Why not just have the RDMA layer call xprt_reserve_xprt() +> > > > > (and +> > > > > xprt_release_xprt()) if this flag is set? It seems to me that +> > > > > you +> > > > > will +> > > > > need some kind of extra congestion control in the RDMA layer +> > > > > anyway +> > > > > since you only have one reserved credit for these privileged +> > > > > tasks +> > > > > (or +> > > > > did I miss where that is being gated?). +> > > > +> > > > Thanks for the review. +> > > > +> > > > See RPCRDMA_IA_RSVD_CREDIT in 11/12. It's a hack I'm not +> > > > terribly happy with. +> > > > +> > > > So, I think you are suggesting replacing xprtrdma's +> > > > ->reserve_xprt with something like: +> > > > +> > > > int xprt_rdma_reserve_xprt(xprt, task) +> > > > { +> > > > if (RPC_SKIP_CONG(task)) +> > > > return xprt_reserve_xprt(xprt, task); +> > > > return xprt_reserve_xprt_cong(xprt, task); +> > > > } +> > > > +> > > > and likewise for ->release_xprt ? +> > > +> > > Right. +> +> This seems to work fine for the normal cases. +> +> I'm confused about how to construct xprt_rdma_release_xprt() +> so it never releases a normal RPC task when a SKIP_CONG +> task completes and the credit limit is still full. +> +> If it should send a normal task using the reserved credit +> and that task hangs too, we're in exactly the position +> we wanted to avoid. +> +> My original solution might have had a similar problem, +> come to think of it. +> +> + +That's true... You may need to set up a separate waitqueue that is +reserved for SKIP_CONG tasks. Again, it makes sense to keep that in the +RDMA code. + +-- + + + + + + +Trond Myklebust +Principal System Architect +4300 El Camino Real | Suite 100 +Los Altos, CA 94022 +W: 650-422-3800 +C: 801-921-4583 +www.primarydata.com diff --git a/a/content_digest b/N1/content_digest index e0d4cb1..09e32b8 100644 --- a/a/content_digest +++ b/N1/content_digest @@ -5,108 +5,168 @@ "ref\01486601331.11028.5.camel@primarydata.com\0" "ref\02AFD96A3-8D49-4E2E-B1F1-9F5C46D0C9C8@oracle.com\0" "ref\04E4245D4-8F9C-4CF3-8B2D-E4528B9E791F@oracle.com\0" - "From\0Trond Myklebust <trondmy@primarydata.com>\0" + "ref\04E4245D4-8F9C-4CF3-8B2D-E4528B9E791F-QHcLZuEGTsvQT0dZR+AlfA@public.gmane.org\0" + "From\0Trond Myklebust <trondmy-7I+n7zu2hftEKMMhf/gKZA@public.gmane.org>\0" "Subject\0Re: [PATCH v3 12/12] sunrpc: Allow keepalive ping on a credit-full transport\0" "Date\0Thu, 9 Feb 2017 20:13:58 +0000\0" - "To\0chuck.lever@oracle.com <chuck.lever@oracle.com>\0" - "Cc\0anna.schumaker@netapp.com <anna.schumaker@netapp.com>" - linux-rdma@vger.kernel.org <linux-rdma@vger.kernel.org> - " linux-nfs@vger.kernel.org <linux-nfs@vger.kernel.org>\0" + "To\0chuck.lever-QHcLZuEGTsvQT0dZR+AlfA@public.gmane.org <chuck.lever-QHcLZuEGTsvQT0dZR+AlfA@public.gmane.org>\0" + "Cc\0anna.schumaker-HgOvQuBEEgTQT0dZR+AlfA@public.gmane.org <anna.schumaker-HgOvQuBEEgTQT0dZR+AlfA@public.gmane.org>" + linux-rdma-u79uwXL29TY76Z2rM5mHXA@public.gmane.org <linux-rdma-u79uwXL29TY76Z2rM5mHXA@public.gmane.org> + " linux-nfs-u79uwXL29TY76Z2rM5mHXA@public.gmane.org <linux-nfs-u79uwXL29TY76Z2rM5mHXA@public.gmane.org>\0" "\00:1\0" "b\0" - "T24gVGh1LCAyMDE3LTAyLTA5IGF0IDE0OjQyIC0wNTAwLCBDaHVjayBMZXZlciB3cm90ZToNCj4g\n" - "PiBPbiBGZWIgOSwgMjAxNywgYXQgMTA6MzcgQU0sIENodWNrIExldmVyIDxjaHVjay5sZXZlckBv\n" - "cmFjbGUuY29tPg0KPiA+IHdyb3RlOg0KPiA+IA0KPiA+ID4gDQo+ID4gPiBPbiBGZWIgOCwgMjAx\n" - "NywgYXQgNzo0OCBQTSwgVHJvbmQgTXlrbGVidXN0IDx0cm9uZG15QHByaW1hcnlkYXRhLg0KPiA+\n" - "ID4gY29tPiB3cm90ZToNCj4gPiA+IA0KPiA+ID4gT24gV2VkLCAyMDE3LTAyLTA4IGF0IDE5OjE5\n" - "IC0wNTAwLCBDaHVjayBMZXZlciB3cm90ZToNCj4gPiA+ID4gPiBPbiBGZWIgOCwgMjAxNywgYXQg\n" - "NzowNSBQTSwgVHJvbmQgTXlrbGVidXN0IDx0cm9uZG15QHByaW1hcnlkDQo+ID4gPiA+ID4gYXRh\n" - "LmNvDQo+ID4gPiA+ID4gbT4gd3JvdGU6DQo+ID4gPiA+ID4gDQo+ID4gPiA+ID4gT24gV2VkLCAy\n" - "MDE3LTAyLTA4IGF0IDE3OjAxIC0wNTAwLCBDaHVjayBMZXZlciB3cm90ZToNCj4gPiA+ID4gPiA+\n" - "IEFsbG93IFJQQy1vdmVyLVJETUEgdG8gc2VuZCBOVUxMIHBpbmdzIGV2ZW4gd2hlbiB0aGUNCj4g\n" - "PiA+ID4gPiA+IHRyYW5zcG9ydA0KPiA+ID4gPiA+ID4gaGFzDQo+ID4gPiA+ID4gPiBoaXQgaXRz\n" - "IGNyZWRpdCBsaW1pdC4gT25lIFJQQy1vdmVyLVJETUEgY3JlZGl0IGlzIHJlc2VydmVkDQo+ID4g\n" - "PiA+ID4gPiBmb3INCj4gPiA+ID4gPiA+IG9wZXJhdGlvbnMgbGlrZSBrZWVwYWxpdmUuDQo+ID4g\n" - "PiA+ID4gPiANCj4gPiA+ID4gPiA+IEZvciB0cmFuc3BvcnRzIHRoYXQgY29udmV5IE5GU3Y0LCBp\n" - "dCBzZWVtcyBsaWtlIGxlYXNlDQo+ID4gPiA+ID4gPiByZW5ld2FsDQo+ID4gPiA+ID4gPiB3b3Vs\n" - "ZA0KPiA+ID4gPiA+ID4gYWxzbyBiZSBhIGNhbmRpZGF0ZSBmb3IgdXNpbmcgYSBwcmlvcml0eSB0\n" - "cmFuc3BvcnQgc2xvdC4NCj4gPiA+ID4gPiA+IEknZCBsaWtlDQo+ID4gPiA+ID4gPiB0bw0KPiA+\n" - "ID4gPiA+ID4gc2VlIGEgbWVjaGFuaXNtIGJldHRlciB0aGFuIFJQQ1JETUFfUFJJT1JJVFkgdGhh\n" - "dCBjYW4NCj4gPiA+ID4gPiA+IGVuc3VyZSBvbmx5DQo+ID4gPiA+ID4gPiBvbmUgcHJpb3JpdHkg\n" - "b3BlcmF0aW9uIGlzIGluIHVzZSBhdCBhIHRpbWUuDQo+ID4gPiA+ID4gPiANCj4gPiA+ID4gPiA+\n" - "IFNpZ25lZC1vZmYtYnk6IENodWNrIExldmVyIDxjaHVjay5sZXZlckBvcmFjbGUuY29tPg0KPiA+\n" - "ID4gPiA+ID4gLS0tDQo+ID4gPiA+ID4gPiBpbmNsdWRlL2xpbnV4L3N1bnJwYy9zY2hlZC5owqDC\n" - "oMKgwqB8wqDCoMKgwqAyICsrDQo+ID4gPiA+ID4gPiBuZXQvc3VucnBjL3hwcnQuY8KgwqDCoMKg\n" - "wqDCoMKgwqDCoMKgwqDCoMKgwqDCoHzCoMKgwqDCoDQgKysrKw0KPiA+ID4gPiA+ID4gbmV0L3N1\n" - "bnJwYy94cHJ0cmRtYS90cmFuc3BvcnQuYyB8wqDCoMKgwqAzICsrLQ0KPiA+ID4gPiA+ID4gbmV0\n" - "L3N1bnJwYy94cHJ0cmRtYS92ZXJicy5jwqDCoMKgwqDCoHzCoMKgwqAxMyArKysrKysrKy0tLS0t\n" - "DQo+ID4gPiA+ID4gPiA0IGZpbGVzIGNoYW5nZWQsIDE2IGluc2VydGlvbnMoKyksIDYgZGVsZXRp\n" - "b25zKC0pDQo+ID4gPiA+ID4gPiANCj4gPiA+ID4gPiA+IGRpZmYgLS1naXQgYS9pbmNsdWRlL2xp\n" - "bnV4L3N1bnJwYy9zY2hlZC5oDQo+ID4gPiA+ID4gPiBiL2luY2x1ZGUvbGludXgvc3VucnBjL3Nj\n" - "aGVkLmgNCj4gPiA+ID4gPiA+IGluZGV4IDEzODIyZTYuLmZjZWExNTggMTAwNjQ0DQo+ID4gPiA+\n" - "ID4gPiAtLS0gYS9pbmNsdWRlL2xpbnV4L3N1bnJwYy9zY2hlZC5oDQo+ID4gPiA+ID4gPiArKysg\n" - "Yi9pbmNsdWRlL2xpbnV4L3N1bnJwYy9zY2hlZC5oDQo+ID4gPiA+ID4gPiBAQCAtMTI3LDYgKzEy\n" - "Nyw3IEBAIHN0cnVjdCBycGNfdGFza19zZXR1cCB7DQo+ID4gPiA+ID4gPiAjZGVmaW5lIFJQQ19U\n" - "QVNLX1RJTUVPVVQJMHgxMDAwCQkvKg0KPiA+ID4gPiA+ID4gZmFpbA0KPiA+ID4gPiA+ID4gd2l0\n" - "aA0KPiA+ID4gPiA+ID4gRVRJTUVET1VUIG9uIHRpbWVvdXQgKi8NCj4gPiA+ID4gPiA+ICNkZWZp\n" - "bmUgUlBDX1RBU0tfTk9DT05ORUNUCTB4MjAwMAkJLyoNCj4gPiA+ID4gPiA+IHJldHVybg0KPiA+\n" - "ID4gPiA+ID4gRU5PVENPTk4gaWYgbm90IGNvbm5lY3RlZCAqLw0KPiA+ID4gPiA+ID4gI2RlZmlu\n" - "ZSBSUENfVEFTS19OT19SRVRSQU5TX1RJTUVPVVQJMHg0MDAwCQkNCj4gPiA+ID4gPiA+IC8qDQo+\n" - "ID4gPiA+ID4gPiB3YWl0IGZvcmV2ZXIgZm9yIGEgcmVwbHkgKi8NCj4gPiA+ID4gPiA+ICsjZGVm\n" - "aW5lIFJQQ19UQVNLX05PX0NPTkcJMHg4MDAwCQkvKg0KPiA+ID4gPiA+ID4gc2tpcA0KPiA+ID4g\n" - "PiA+ID4gY29uZ2VzdGlvbiBjb250cm9sICovDQo+ID4gPiA+ID4gPiANCj4gPiA+ID4gPiA+ICNk\n" - "ZWZpbmUgUlBDX1RBU0tfU09GVFBJTkcJKFJQQ19UQVNLX1NPRlQgfA0KPiA+ID4gPiA+ID4gUlBD\n" - "X1RBU0tfU09GVENPTk4pDQo+ID4gPiA+ID4gPiANCj4gPiA+ID4gPiA+IEBAIC0xMzcsNiArMTM4\n" - "LDcgQEAgc3RydWN0IHJwY190YXNrX3NldHVwIHsNCj4gPiA+ID4gPiA+ICNkZWZpbmUgUlBDX0lT\n" - "X1NPRlQodCkJCSgodCktPnRrX2ZsYWdzICYNCj4gPiA+ID4gPiA+IChSUENfVEFTS19TT0ZUfFJQ\n" - "Q19UQVNLX1RJTUVPVVQpKQ0KPiA+ID4gPiA+ID4gI2RlZmluZSBSUENfSVNfU09GVENPTk4odCkJ\n" - "KCh0KS0+dGtfZmxhZ3MgJg0KPiA+ID4gPiA+ID4gUlBDX1RBU0tfU09GVENPTk4pDQo+ID4gPiA+\n" - "ID4gPiAjZGVmaW5lIFJQQ19XQVNfU0VOVCh0KQkJKCh0KS0+dGtfZmxhZ3MgJg0KPiA+ID4gPiA+\n" - "ID4gUlBDX1RBU0tfU0VOVCkNCj4gPiA+ID4gPiA+ICsjZGVmaW5lIFJQQ19TS0lQX0NPTkcodCkJ\n" - "KCh0KS0+dGtfZmxhZ3MgJg0KPiA+ID4gPiA+ID4gUlBDX1RBU0tfTk9fQ09ORykNCj4gPiA+ID4g\n" - "PiA+IA0KPiA+ID4gPiA+ID4gI2RlZmluZSBSUENfVEFTS19SVU5OSU5HCTANCj4gPiA+ID4gPiA+\n" - "ICNkZWZpbmUgUlBDX1RBU0tfUVVFVUVECQkxDQo+ID4gPiA+ID4gPiBkaWZmIC0tZ2l0IGEvbmV0\n" - "L3N1bnJwYy94cHJ0LmMgYi9uZXQvc3VucnBjL3hwcnQuYw0KPiA+ID4gPiA+ID4gaW5kZXggYjUz\n" - "MGEyOC4uYTQ3N2VlNiAxMDA2NDQNCj4gPiA+ID4gPiA+IC0tLSBhL25ldC9zdW5ycGMveHBydC5j\n" - "DQo+ID4gPiA+ID4gPiArKysgYi9uZXQvc3VucnBjL3hwcnQuYw0KPiA+ID4gPiA+ID4gQEAgLTM5\n" - "Miw2ICszOTIsMTAgQEAgc3RhdGljIGlubGluZSB2b2lkDQo+ID4gPiA+ID4gPiB4cHJ0X3JlbGVh\n" - "c2Vfd3JpdGUoc3RydWN0DQo+ID4gPiA+ID4gPiBycGNfeHBydCAqeHBydCwgc3RydWN0IHJwY190\n" - "YXNrICp0YQ0KPiA+ID4gPiA+ID4gew0KPiA+ID4gPiA+ID4gCXN0cnVjdCBycGNfcnFzdCAqcmVx\n" - "ID0gdGFzay0+dGtfcnFzdHA7DQo+ID4gPiA+ID4gPiANCj4gPiA+ID4gPiA+ICsJaWYgKFJQQ19T\n" - "S0lQX0NPTkcodGFzaykpIHsNCj4gPiA+ID4gPiA+ICsJCXJlcS0+cnFfY29uZyA9IDA7DQo+ID4g\n" - "PiA+ID4gPiArCQlyZXR1cm4gMTsNCj4gPiA+ID4gPiA+ICsJfQ0KPiA+ID4gPiA+IA0KPiA+ID4g\n" - "PiA+IFdoeSBub3QganVzdCBoYXZlIHRoZSBSRE1BIGxheWVyIGNhbGwgeHBydF9yZXNlcnZlX3hw\n" - "cnQoKQ0KPiA+ID4gPiA+IChhbmQNCj4gPiA+ID4gPiB4cHJ0X3JlbGVhc2VfeHBydCgpKSBpZiB0\n" - "aGlzIGZsYWcgaXMgc2V0PyBJdCBzZWVtcyB0byBtZSB0aGF0DQo+ID4gPiA+ID4geW91DQo+ID4g\n" - "PiA+ID4gd2lsbA0KPiA+ID4gPiA+IG5lZWQgc29tZSBraW5kIG9mIGV4dHJhIGNvbmdlc3Rpb24g\n" - "Y29udHJvbCBpbiB0aGUgUkRNQSBsYXllcg0KPiA+ID4gPiA+IGFueXdheQ0KPiA+ID4gPiA+IHNp\n" - "bmNlIHlvdSBvbmx5IGhhdmUgb25lIHJlc2VydmVkIGNyZWRpdCBmb3IgdGhlc2UgcHJpdmlsZWdl\n" - "ZA0KPiA+ID4gPiA+IHRhc2tzDQo+ID4gPiA+ID4gKG9yDQo+ID4gPiA+ID4gZGlkIEkgbWlzcyB3\n" - "aGVyZSB0aGF0IGlzIGJlaW5nIGdhdGVkPykuDQo+ID4gPiA+IA0KPiA+ID4gPiBUaGFua3MgZm9y\n" - "IHRoZSByZXZpZXcuDQo+ID4gPiA+IA0KPiA+ID4gPiBTZWUgUlBDUkRNQV9JQV9SU1ZEX0NSRURJ\n" - "VCBpbiAxMS8xMi4gSXQncyBhIGhhY2sgSSdtIG5vdA0KPiA+ID4gPiB0ZXJyaWJseSBoYXBweSB3\n" - "aXRoLg0KPiA+ID4gPiANCj4gPiA+ID4gU28sIEkgdGhpbmsgeW91IGFyZSBzdWdnZXN0aW5nIHJl\n" - "cGxhY2luZyB4cHJ0cmRtYSdzDQo+ID4gPiA+IC0+cmVzZXJ2ZV94cHJ0IHdpdGggc29tZXRoaW5n\n" - "IGxpa2U6DQo+ID4gPiA+IA0KPiA+ID4gPiBpbnQgeHBydF9yZG1hX3Jlc2VydmVfeHBydCh4cHJ0\n" - "LCB0YXNrKQ0KPiA+ID4gPiB7DQo+ID4gPiA+IMKgwqDCoMKgwqBpZiAoUlBDX1NLSVBfQ09ORyh0\n" - "YXNrKSkNCj4gPiA+ID4gwqDCoMKgwqDCoMKgwqDCoMKgwqByZXR1cm4geHBydF9yZXNlcnZlX3hw\n" - "cnQoeHBydCwgdGFzayk7DQo+ID4gPiA+IMKgwqDCoMKgwqByZXR1cm4geHBydF9yZXNlcnZlX3hw\n" - "cnRfY29uZyh4cHJ0LCB0YXNrKTsNCj4gPiA+ID4gfQ0KPiA+ID4gPiANCj4gPiA+ID4gYW5kIGxp\n" - "a2V3aXNlIGZvciAtPnJlbGVhc2VfeHBydCA/DQo+ID4gPiANCj4gPiA+IFJpZ2h0Lg0KPiANCj4g\n" - "VGhpcyBzZWVtcyB0byB3b3JrIGZpbmUgZm9yIHRoZSBub3JtYWwgY2FzZXMuDQo+IA0KPiBJJ20g\n" - "Y29uZnVzZWQgYWJvdXQgaG93IHRvIGNvbnN0cnVjdCB4cHJ0X3JkbWFfcmVsZWFzZV94cHJ0KCkN\n" - "Cj4gc28gaXQgbmV2ZXIgcmVsZWFzZXMgYSBub3JtYWwgUlBDIHRhc2sgd2hlbiBhIFNLSVBfQ09O\n" - "Rw0KPiB0YXNrIGNvbXBsZXRlcyBhbmQgdGhlIGNyZWRpdCBsaW1pdCBpcyBzdGlsbCBmdWxsLg0K\n" - "PiANCj4gSWYgaXQgc2hvdWxkIHNlbmQgYSBub3JtYWwgdGFzayB1c2luZyB0aGUgcmVzZXJ2ZWQg\n" - "Y3JlZGl0DQo+IGFuZCB0aGF0IHRhc2sgaGFuZ3MgdG9vLCB3ZSdyZSBpbiBleGFjdGx5IHRoZSBw\n" - "b3NpdGlvbg0KPiB3ZSB3YW50ZWQgdG8gYXZvaWQuDQo+IA0KPiBNeSBvcmlnaW5hbCBzb2x1dGlv\n" - "biBtaWdodCBoYXZlIGhhZCBhIHNpbWlsYXIgcHJvYmxlbSwNCj4gY29tZSB0byB0aGluayBvZiBp\n" - "dC4NCj4gDQo+IA0KDQpUaGF0J3MgdHJ1ZS4uLiBZb3UgbWF5IG5lZWQgdG8gc2V0IHVwIGEgc2Vw\n" - "YXJhdGUgd2FpdHF1ZXVlIHRoYXQgaXMNCnJlc2VydmVkIGZvciBTS0lQX0NPTkcgdGFza3MuIEFn\n" - "YWluLCBpdCBtYWtlcyBzZW5zZSB0byBrZWVwIHRoYXQgaW4gdGhlDQpSRE1BIGNvZGUuDQoNCi0t\n" - "IA0KDQoNCgkNCgkNCg0KDQpUcm9uZCBNeWtsZWJ1c3QNClByaW5jaXBhbCBTeXN0ZW0gQXJjaGl0\n" - "ZWN0DQo0MzAwIEVsIENhbWlubyBSZWFsIHwgU3VpdGUgMTAwDQpMb3MgQWx0b3MsIENBwqDCoDk0\n" - "MDIyDQpXOiA2NTAtNDIyLTM4MDANCkM6IDgwMS05MjEtNDU4M8KgDQp3d3cucHJpbWFyeWRhdGEu\n" - Y29tDQoNCg0KDQo= + "On Thu, 2017-02-09 at 14:42 -0500, Chuck Lever wrote:\n" + "> > On Feb 9, 2017, at 10:37 AM, Chuck Lever <chuck.lever@oracle.com>\n" + "> > wrote:\n" + "> > \n" + "> > > \n" + "> > > On Feb 8, 2017, at 7:48 PM, Trond Myklebust <trondmy@primarydata.\n" + "> > > com> wrote:\n" + "> > > \n" + "> > > On Wed, 2017-02-08 at 19:19 -0500, Chuck Lever wrote:\n" + "> > > > > On Feb 8, 2017, at 7:05 PM, Trond Myklebust <trondmy@primaryd\n" + "> > > > > ata.co\n" + "> > > > > m> wrote:\n" + "> > > > > \n" + "> > > > > On Wed, 2017-02-08 at 17:01 -0500, Chuck Lever wrote:\n" + "> > > > > > Allow RPC-over-RDMA to send NULL pings even when the\n" + "> > > > > > transport\n" + "> > > > > > has\n" + "> > > > > > hit its credit limit. One RPC-over-RDMA credit is reserved\n" + "> > > > > > for\n" + "> > > > > > operations like keepalive.\n" + "> > > > > > \n" + "> > > > > > For transports that convey NFSv4, it seems like lease\n" + "> > > > > > renewal\n" + "> > > > > > would\n" + "> > > > > > also be a candidate for using a priority transport slot.\n" + "> > > > > > I'd like\n" + "> > > > > > to\n" + "> > > > > > see a mechanism better than RPCRDMA_PRIORITY that can\n" + "> > > > > > ensure only\n" + "> > > > > > one priority operation is in use at a time.\n" + "> > > > > > \n" + "> > > > > > Signed-off-by: Chuck Lever <chuck.lever@oracle.com>\n" + "> > > > > > ---\n" + "> > > > > > include/linux/sunrpc/sched.h\302\240\302\240\302\240\302\240|\302\240\302\240\302\240\302\2402 ++\n" + "> > > > > > net/sunrpc/xprt.c\302\240\302\240\302\240\302\240\302\240\302\240\302\240\302\240\302\240\302\240\302\240\302\240\302\240\302\240\302\240|\302\240\302\240\302\240\302\2404 ++++\n" + "> > > > > > net/sunrpc/xprtrdma/transport.c |\302\240\302\240\302\240\302\2403 ++-\n" + "> > > > > > net/sunrpc/xprtrdma/verbs.c\302\240\302\240\302\240\302\240\302\240|\302\240\302\240\302\24013 ++++++++-----\n" + "> > > > > > 4 files changed, 16 insertions(+), 6 deletions(-)\n" + "> > > > > > \n" + "> > > > > > diff --git a/include/linux/sunrpc/sched.h\n" + "> > > > > > b/include/linux/sunrpc/sched.h\n" + "> > > > > > index 13822e6..fcea158 100644\n" + "> > > > > > --- a/include/linux/sunrpc/sched.h\n" + "> > > > > > +++ b/include/linux/sunrpc/sched.h\n" + "> > > > > > @@ -127,6 +127,7 @@ struct rpc_task_setup {\n" + "> > > > > > #define RPC_TASK_TIMEOUT\t0x1000\t\t/*\n" + "> > > > > > fail\n" + "> > > > > > with\n" + "> > > > > > ETIMEDOUT on timeout */\n" + "> > > > > > #define RPC_TASK_NOCONNECT\t0x2000\t\t/*\n" + "> > > > > > return\n" + "> > > > > > ENOTCONN if not connected */\n" + "> > > > > > #define RPC_TASK_NO_RETRANS_TIMEOUT\t0x4000\t\t\n" + "> > > > > > /*\n" + "> > > > > > wait forever for a reply */\n" + "> > > > > > +#define RPC_TASK_NO_CONG\t0x8000\t\t/*\n" + "> > > > > > skip\n" + "> > > > > > congestion control */\n" + "> > > > > > \n" + "> > > > > > #define RPC_TASK_SOFTPING\t(RPC_TASK_SOFT |\n" + "> > > > > > RPC_TASK_SOFTCONN)\n" + "> > > > > > \n" + "> > > > > > @@ -137,6 +138,7 @@ struct rpc_task_setup {\n" + "> > > > > > #define RPC_IS_SOFT(t)\t\t((t)->tk_flags &\n" + "> > > > > > (RPC_TASK_SOFT|RPC_TASK_TIMEOUT))\n" + "> > > > > > #define RPC_IS_SOFTCONN(t)\t((t)->tk_flags &\n" + "> > > > > > RPC_TASK_SOFTCONN)\n" + "> > > > > > #define RPC_WAS_SENT(t)\t\t((t)->tk_flags &\n" + "> > > > > > RPC_TASK_SENT)\n" + "> > > > > > +#define RPC_SKIP_CONG(t)\t((t)->tk_flags &\n" + "> > > > > > RPC_TASK_NO_CONG)\n" + "> > > > > > \n" + "> > > > > > #define RPC_TASK_RUNNING\t0\n" + "> > > > > > #define RPC_TASK_QUEUED\t\t1\n" + "> > > > > > diff --git a/net/sunrpc/xprt.c b/net/sunrpc/xprt.c\n" + "> > > > > > index b530a28..a477ee6 100644\n" + "> > > > > > --- a/net/sunrpc/xprt.c\n" + "> > > > > > +++ b/net/sunrpc/xprt.c\n" + "> > > > > > @@ -392,6 +392,10 @@ static inline void\n" + "> > > > > > xprt_release_write(struct\n" + "> > > > > > rpc_xprt *xprt, struct rpc_task *ta\n" + "> > > > > > {\n" + "> > > > > > \tstruct rpc_rqst *req = task->tk_rqstp;\n" + "> > > > > > \n" + "> > > > > > +\tif (RPC_SKIP_CONG(task)) {\n" + "> > > > > > +\t\treq->rq_cong = 0;\n" + "> > > > > > +\t\treturn 1;\n" + "> > > > > > +\t}\n" + "> > > > > \n" + "> > > > > Why not just have the RDMA layer call xprt_reserve_xprt()\n" + "> > > > > (and\n" + "> > > > > xprt_release_xprt()) if this flag is set? It seems to me that\n" + "> > > > > you\n" + "> > > > > will\n" + "> > > > > need some kind of extra congestion control in the RDMA layer\n" + "> > > > > anyway\n" + "> > > > > since you only have one reserved credit for these privileged\n" + "> > > > > tasks\n" + "> > > > > (or\n" + "> > > > > did I miss where that is being gated?).\n" + "> > > > \n" + "> > > > Thanks for the review.\n" + "> > > > \n" + "> > > > See RPCRDMA_IA_RSVD_CREDIT in 11/12. It's a hack I'm not\n" + "> > > > terribly happy with.\n" + "> > > > \n" + "> > > > So, I think you are suggesting replacing xprtrdma's\n" + "> > > > ->reserve_xprt with something like:\n" + "> > > > \n" + "> > > > int xprt_rdma_reserve_xprt(xprt, task)\n" + "> > > > {\n" + "> > > > \302\240\302\240\302\240\302\240\302\240if (RPC_SKIP_CONG(task))\n" + "> > > > \302\240\302\240\302\240\302\240\302\240\302\240\302\240\302\240\302\240\302\240return xprt_reserve_xprt(xprt, task);\n" + "> > > > \302\240\302\240\302\240\302\240\302\240return xprt_reserve_xprt_cong(xprt, task);\n" + "> > > > }\n" + "> > > > \n" + "> > > > and likewise for ->release_xprt ?\n" + "> > > \n" + "> > > Right.\n" + "> \n" + "> This seems to work fine for the normal cases.\n" + "> \n" + "> I'm confused about how to construct xprt_rdma_release_xprt()\n" + "> so it never releases a normal RPC task when a SKIP_CONG\n" + "> task completes and the credit limit is still full.\n" + "> \n" + "> If it should send a normal task using the reserved credit\n" + "> and that task hangs too, we're in exactly the position\n" + "> we wanted to avoid.\n" + "> \n" + "> My original solution might have had a similar problem,\n" + "> come to think of it.\n" + "> \n" + "> \n" + "\n" + "That's true... You may need to set up a separate waitqueue that is\n" + "reserved for SKIP_CONG tasks. Again, it makes sense to keep that in the\n" + "RDMA code.\n" + "\n" + "-- \n" + "\n" + "\n" + "\t\n" + "\t\n" + "\n" + "\n" + "Trond Myklebust\n" + "Principal System Architect\n" + "4300 El Camino Real | Suite 100\n" + "Los Altos, CA\302\240\302\24094022\n" + "W: 650-422-3800\n" + "C: 801-921-4583\302\240\n" + www.primarydata.com -49c77e488dbed000adaf2e05b3edfedf778c7980ced7057d65eb14f371c22da2 +4daa8fd9280c7df0355ddc57dc8e23ed1376dfc0f906698ef795c928fe2f34ea
This is an external index of several public inboxes, see mirroring instructions on how to clone and mirror all data and code used by this external index.