* Re: RFC 5661 LAYOUTRETURN clarification. [not found] <CAHVgHyUpM0rQWqO5-id+FohPKm1Lk=kkekf7HqzpfKcfvxx23A@mail.gmail.com> @ 2012-06-11 18:40 ` Boaz Harrosh 2012-06-11 18:59 ` Myklebust, Trond 2012-06-11 19:02 ` [nfsv4] " david.noveck 0 siblings, 2 replies; 8+ messages in thread From: Boaz Harrosh @ 2012-06-11 18:40 UTC (permalink / raw) To: Andy Adamson; +Cc: NFSv4, Trond Myklebust, Andy Adamson, NFS list, Benny Halevy On 06/11/2012 07:01 PM, Andy Adamson wrote: > I'm coding file layout data server recovery for the Linux NFS client, > and came across an issue with LAYOUTRETURN that > could use some comment from the list. > > The error case I'm handling is an RPC layer dis-connection error > during heavy WRITE i/o to a file layout data server. Our response is > to internally mark the deviceid as invalid which prevents all pNFS > calls using the deviceid - e.g. no new I/O using any layout that uses > the invalid deviceid, and to redirect all I/O to the MDS (any queued > RPC request that has not been sent is redirected to the MDS). > > Plus - and here is where the clarification is needed - we immediately > send a LAYOUTRETURN for any layout with in-flight requests to the > dis-connected data server. By in-flight I mean transmitted WRT the > RPC layer. The purpose of this LAYOUTRETURN is to notify the file > layout MDS to fence the DS for the specified LAYOUTs, as the WRITEs > will also be sent to the MDS. > I do not disagree with this completely. The point here is very fine grained and should be specified explicitly. I would like to see text as of something like. There are 3 types of in-flght RPC/IO 1. Client has sent RPC header + all of associated data and is waiting for DS WRITE/READ_DONE reply. (For me this case can be, client may return LAYOUTRETURN as your suggestion) 2. Client has sent the RPC header but has got stuck sending the rest of the RPC message. Then received a network disconnect. This is the most common part. Putting aside the RPC that got the error for a second. The most important is what to do with parallel RPC/IO which are in this state. Are parallel RPCs allowed to continue sending network packets after the LAYOUTRETURN was sent? The specific RPC that got stuck is not interesting because it's kind of 1.5, We are not going to send any bytes on that channel. The interesting is these other DSs which are still streaming 3. Client has some internal RPC queue which do to some client parallelism will start sending RPC header + data after the LAYOUTRETURN was sent What my point was that with the code you submitted we are clearly violating 2. and even 3. Because I do not see anything avoiding this. And if the STD allows you 2 and 3. Then that's a big change to the concept. Not like you let it seem. > I contend that sending the LAYOUTRETURN in this error case does not > violate the two sections of RFC 5661 below, as the client has stopped > sending any I/O requests using the returned layout. > I would not mind if this was true. That is if the LAYOUTRETURN was a very clear barrier where our client would "magically" completely plug the network interface and will not continue to send a single byte on the wire to *any* DS involved with the layout. That's fine. That is only allow sate 1 and 1.5 RPCs above. Some/all bytes where presented on the wire, until the LAYOUTRETURN, from which point all RPCs are hard aborted and not a single byte is sent. > Others contend that since the in-flight RPCs reference the returned > layout, the client is still 'using' the layout with these in-flight > requests, and can not call LAYOUTRETURN until all in-flight RPCs > return, with or without an error. > With our client code I don't see how the guaranty of 2 and 3 above will happen without actually implementing this here. So in principal I agree with your principle, I only do not agree with your practice. In your new code you are violating 2 and 3 which are not to be allowed. And again, please explain why do you want it. What is wrong with the case we all agree with? ie: "Client can not call LAYOUTRETURN until all in-flight RPCs return, with or without an error" Thanks Boaz > > Section 18.44.3 - the description section of the LAYOUTRETURN operation: > > After this call, > the client MUST NOT use the returned layout(s) and the associated > storage protocol to access the file data. > > Section 13.6 Operations Sent to NFSv4.1 Data Servers > > As described in Section 12.5.1, a client > MUST NOT send an I/O to a data server for which it does not hold a > valid layout; the data server MUST reject such an I/O. > > > -->Andy ^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: RFC 5661 LAYOUTRETURN clarification. 2012-06-11 18:40 ` RFC 5661 LAYOUTRETURN clarification Boaz Harrosh @ 2012-06-11 18:59 ` Myklebust, Trond 2012-06-11 22:18 ` Boaz Harrosh 2012-06-12 16:34 ` Welch, Brent 2012-06-11 19:02 ` [nfsv4] " david.noveck 1 sibling, 2 replies; 8+ messages in thread From: Myklebust, Trond @ 2012-06-11 18:59 UTC (permalink / raw) To: Boaz Harrosh; +Cc: Andy Adamson, NFSv4, Adamson, Andy, NFS list, Benny Halevy T24gTW9uLCAyMDEyLTA2LTExIGF0IDIxOjQwICswMzAwLCBCb2F6IEhhcnJvc2ggd3JvdGU6DQoN Cj4gQW5kIGFnYWluLCBwbGVhc2UgZXhwbGFpbiB3aHkgZG8geW91IHdhbnQgaXQuIFdoYXQgaXMg d3Jvbmcgd2l0aCB0aGUNCj4gY2FzZSB3ZSBhbGwgYWdyZWUgd2l0aD8gaWU6ICJDbGllbnQgY2Fu IG5vdCBjYWxsIExBWU9VVFJFVFVSTiB1bnRpbA0KPiBhbGwgaW4tZmxpZ2h0IFJQQ3MgcmV0dXJu LCB3aXRoIG9yIHdpdGhvdXQgYW4gZXJyb3IiDQoNCldobyAiYWdyZWVkIiB0byB0aGlzPyBUaGlz IHdvdWxkIG1lYW4gdGhhdCBpZiB0aGUgRFMgZ29lcyBkb3duLCB3ZSBjYW4ndA0KZXZlciBzZW5k IExBWU9VVFJFVFVSTiB3aGljaCBpcyBwYXRlbnRseSB3cm9uZy4NCg0KLS0gDQpUcm9uZCBNeWts ZWJ1c3QNCkxpbnV4IE5GUyBjbGllbnQgbWFpbnRhaW5lcg0KDQpOZXRBcHANClRyb25kLk15a2xl YnVzdEBuZXRhcHAuY29tDQp3d3cubmV0YXBwLmNvbQ0KDQo= ^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: RFC 5661 LAYOUTRETURN clarification. 2012-06-11 18:59 ` Myklebust, Trond @ 2012-06-11 22:18 ` Boaz Harrosh 2012-06-11 22:31 ` Myklebust, Trond 2012-06-12 16:34 ` Welch, Brent 1 sibling, 1 reply; 8+ messages in thread From: Boaz Harrosh @ 2012-06-11 22:18 UTC (permalink / raw) To: Myklebust, Trond Cc: Andy Adamson, NFSv4, Adamson, Andy, NFS list, Benny Halevy On 06/11/2012 09:59 PM, Myklebust, Trond wrote: > On Mon, 2012-06-11 at 21:40 +0300, Boaz Harrosh wrote: > >> And again, please explain why do you want it. What is wrong with the >> case we all agree with? ie: "Client can not call LAYOUTRETURN until >> all in-flight RPCs return, with or without an error" > > Who "agreed" to this? This would mean that if the DS goes down, we can't > ever send LAYOUTRETURN which is patently wrong. > "DS goes down" is under the above "RPC return an error" the error condition of an RPC is well defined. >From what of my words did you understand that I said "we can't ever send a LAYOUTRETURN" If my English is wrongly worded. Which is perfectly possible. Please correct me so I can learn. Did you honestly think that's what I meant? I meant we all agree, that this case is covered by RFC. That is - no one would accuse a client who does that, as violating the RFC. And again my question. The motivation? Thanks Boaz ^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: RFC 5661 LAYOUTRETURN clarification. 2012-06-11 22:18 ` Boaz Harrosh @ 2012-06-11 22:31 ` Myklebust, Trond 0 siblings, 0 replies; 8+ messages in thread From: Myklebust, Trond @ 2012-06-11 22:31 UTC (permalink / raw) To: Boaz Harrosh; +Cc: Andy Adamson, NFSv4, Adamson, Andy, NFS list, Benny Halevy T24gVHVlLCAyMDEyLTA2LTEyIGF0IDAxOjE4ICswMzAwLCBCb2F6IEhhcnJvc2ggd3JvdGU6DQo+ IE9uIDA2LzExLzIwMTIgMDk6NTkgUE0sIE15a2xlYnVzdCwgVHJvbmQgd3JvdGU6DQo+IA0KPiA+ IE9uIE1vbiwgMjAxMi0wNi0xMSBhdCAyMTo0MCArMDMwMCwgQm9heiBIYXJyb3NoIHdyb3RlOg0K PiA+IA0KPiA+PiBBbmQgYWdhaW4sIHBsZWFzZSBleHBsYWluIHdoeSBkbyB5b3Ugd2FudCBpdC4g V2hhdCBpcyB3cm9uZyB3aXRoIHRoZQ0KPiA+PiBjYXNlIHdlIGFsbCBhZ3JlZSB3aXRoPyBpZTog IkNsaWVudCBjYW4gbm90IGNhbGwgTEFZT1VUUkVUVVJOIHVudGlsDQo+ID4+IGFsbCBpbi1mbGln aHQgUlBDcyByZXR1cm4sIHdpdGggb3Igd2l0aG91dCBhbiBlcnJvciINCj4gPiANCj4gPiBXaG8g ImFncmVlZCIgdG8gdGhpcz8gVGhpcyB3b3VsZCBtZWFuIHRoYXQgaWYgdGhlIERTIGdvZXMgZG93 biwgd2UgY2FuJ3QNCj4gPiBldmVyIHNlbmQgTEFZT1VUUkVUVVJOIHdoaWNoIGlzIHBhdGVudGx5 IHdyb25nLg0KPiA+IA0KPiANCj4gDQo+ICJEUyBnb2VzIGRvd24iIGlzIHVuZGVyIHRoZSBhYm92 ZSAiUlBDIHJldHVybiBhbiBlcnJvciIgdGhlIGVycm9yIGNvbmRpdGlvbg0KPiBvZiBhbiBSUEMg aXMgd2VsbCBkZWZpbmVkLg0KDQo/Pz8/IE5vdyB5b3UgaGF2ZSBtZSBleHRyZW1lbHkgY29uZnVz ZWQuIEhvdyBkb2VzIHRoZSBjbGllbnQgZGlzdGluZ3Vpc2gNCmJldHdlZW4gRVRJTUVET1VULWJl Y2F1c2UtRFMtd2VudC1kb3duIGFuZA0KRVRJTUVET1VULWJ1dC1pbi1mbGlnaHQtUlBDcy13aWxs LWV2ZW50dWFsbHktc3VjY2VlZC1zby1wbGVhc2UtaG9sZC10aGF0LUxBWU9VVFJFVFVSTj8NCg0K PiA+RnJvbSB3aGF0IG9mIG15IHdvcmRzIGRpZCB5b3UgdW5kZXJzdGFuZCB0aGF0IEkgc2FpZA0K PiAJIndlIGNhbid0IGV2ZXIgc2VuZCBhIExBWU9VVFJFVFVSTiINCj4gDQo+IElmIG15IEVuZ2xp c2ggaXMgd3JvbmdseSB3b3JkZWQuIFdoaWNoIGlzIHBlcmZlY3RseSBwb3NzaWJsZS4gUGxlYXNl IGNvcnJlY3QNCj4gbWUgc28gSSBjYW4gbGVhcm4uIERpZCB5b3UgaG9uZXN0bHkgdGhpbmsgdGhh dCdzIHdoYXQgSSBtZWFudD8gDQo+IA0KPiBJIG1lYW50IHdlIGFsbCBhZ3JlZSwgdGhhdCB0aGlz IGNhc2UgaXMgY292ZXJlZCBieSBSRkMuIFRoYXQgaXMgIC0gbm8gb25lIHdvdWxkDQo+IGFjY3Vz ZSBhIGNsaWVudCB3aG8gZG9lcyB0aGF0LCBhcyB2aW9sYXRpbmcgdGhlIFJGQy4NCj4gDQo+IEFu ZCBhZ2FpbiBteSBxdWVzdGlvbi4gVGhlIG1vdGl2YXRpb24/DQoNCk1ha2luZyBmYWxsYmFjay10 by1NRFMgd29yayB3aXRob3V0IGNvcnJ1cHRpbmcgeW91ciBkYXRhLiBQbGVhc2UgcmVhZCB1cA0K b24gbXVsdGlwYXRoLTEwMSBhbmQgdGhlIG5lZWQgZm9yIGZlbmNpbmcuLi4NCg0KLS0gDQpUcm9u ZCBNeWtsZWJ1c3QNCkxpbnV4IE5GUyBjbGllbnQgbWFpbnRhaW5lcg0KDQpOZXRBcHANClRyb25k Lk15a2xlYnVzdEBuZXRhcHAuY29tDQp3d3cubmV0YXBwLmNvbQ0KDQo= ^ permalink raw reply [flat|nested] 8+ messages in thread
* RE: RFC 5661 LAYOUTRETURN clarification. 2012-06-11 18:59 ` Myklebust, Trond 2012-06-11 22:18 ` Boaz Harrosh @ 2012-06-12 16:34 ` Welch, Brent 1 sibling, 0 replies; 8+ messages in thread From: Welch, Brent @ 2012-06-12 16:34 UTC (permalink / raw) To: Myklebust, Trond, Harrosh, Boaz Cc: NFS list, Adamson, Andy, Andy Adamson, NFSv4 This is why the objects layout communicates errors upon layout return. When you suffer write timeouts (or any errors) with a DS, then the client promptly returns the layout with an indication of what went wrong. This should be the intent of whatever words are in the RFC. -- Brent -----Original Message----- From: nfsv4-bounces@ietf.org [mailto:nfsv4-bounces@ietf.org] On Behalf Of Myklebust, Trond Sent: Monday, June 11, 2012 11:59 AM To: Harrosh, Boaz Cc: NFS list; Adamson, Andy; Andy Adamson; NFSv4 Subject: Re: [nfsv4] RFC 5661 LAYOUTRETURN clarification. On Mon, 2012-06-11 at 21:40 +0300, Boaz Harrosh wrote: > And again, please explain why do you want it. What is wrong with the > case we all agree with? ie: "Client can not call LAYOUTRETURN until > all in-flight RPCs return, with or without an error" Who "agreed" to this? This would mean that if the DS goes down, we can't ever send LAYOUTRETURN which is patently wrong. -- Trond Myklebust Linux NFS client maintainer NetApp Trond.Myklebust@netapp.com www.netapp.com _______________________________________________ nfsv4 mailing list nfsv4@ietf.org https://www.ietf.org/mailman/listinfo/nfsv4 ^ permalink raw reply [flat|nested] 8+ messages in thread
* RE: [nfsv4] RFC 5661 LAYOUTRETURN clarification. 2012-06-11 18:40 ` RFC 5661 LAYOUTRETURN clarification Boaz Harrosh 2012-06-11 18:59 ` Myklebust, Trond @ 2012-06-11 19:02 ` david.noveck 2012-06-11 19:43 ` Myklebust, Trond 1 sibling, 1 reply; 8+ messages in thread From: david.noveck @ 2012-06-11 19:02 UTC (permalink / raw) To: bharrosh, androsadamson; +Cc: andros, linux-nfs, Trond.Myklebust, nfsv4 > And again, please explain why do you want it. What is wrong with the > case we all agree with? ie: "Client can not call LAYOUTRETURN until > all in-flight RPCs return, with or without an error" It's a recipe for data corruption. If, as Andy explained, he starts doing IO's (let's suppose WRITEs) to the MDS any lingering WRITEs to the DS since they reflect an earlier state of affairs can cause data corruption. There are three ways to prevent those lingering DS writes from corrupting data: 1) Doing a LAYOUTRETURN 2) waiting until the IO's return. 3) "magically plugging the network interface". Since there is no way to do 3), saying that you only can do 1) until after 2) is done is essentially going to mean: a) that it may take a very long time: b) that you will only do it, when it is no longer useful. If you do 1) asap, then the lingering DS write problem is gone sooner, and that's a good thing. -----Original Message----- From: nfsv4-bounces@ietf.org [mailto:nfsv4-bounces@ietf.org] On Behalf Of Boaz Harrosh Sent: Monday, June 11, 2012 2:41 PM To: Andy Adamson Cc: Andy Adamson; NFS list; Trond Myklebust; NFSv4 Subject: Re: [nfsv4] RFC 5661 LAYOUTRETURN clarification. On 06/11/2012 07:01 PM, Andy Adamson wrote: > I'm coding file layout data server recovery for the Linux NFS client, > and came across an issue with LAYOUTRETURN that > could use some comment from the list. > > The error case I'm handling is an RPC layer dis-connection error > during heavy WRITE i/o to a file layout data server. Our response is > to internally mark the deviceid as invalid which prevents all pNFS > calls using the deviceid - e.g. no new I/O using any layout that uses > the invalid deviceid, and to redirect all I/O to the MDS (any queued > RPC request that has not been sent is redirected to the MDS). > > Plus - and here is where the clarification is needed - we immediately > send a LAYOUTRETURN for any layout with in-flight requests to the > dis-connected data server. By in-flight I mean transmitted WRT the > RPC layer. The purpose of this LAYOUTRETURN is to notify the file > layout MDS to fence the DS for the specified LAYOUTs, as the WRITEs > will also be sent to the MDS. > I do not disagree with this completely. The point here is very fine grained and should be specified explicitly. I would like to see text as of something like. There are 3 types of in-flght RPC/IO 1. Client has sent RPC header + all of associated data and is waiting for DS WRITE/READ_DONE reply. (For me this case can be, client may return LAYOUTRETURN as your suggestion) 2. Client has sent the RPC header but has got stuck sending the rest of the RPC message. Then received a network disconnect. This is the most common part. Putting aside the RPC that got the error for a second. The most important is what to do with parallel RPC/IO which are in this state. Are parallel RPCs allowed to continue sending network packets after the LAYOUTRETURN was sent? The specific RPC that got stuck is not interesting because it's kind of 1.5, We are not going to send any bytes on that channel. The interesting is these other DSs which are still streaming 3. Client has some internal RPC queue which do to some client parallelism will start sending RPC header + data after the LAYOUTRETURN was sent What my point was that with the code you submitted we are clearly violating 2. and even 3. Because I do not see anything avoiding this. And if the STD allows you 2 and 3. Then that's a big change to the concept. Not like you let it seem. > I contend that sending the LAYOUTRETURN in this error case does not > violate the two sections of RFC 5661 below, as the client has stopped > sending any I/O requests using the returned layout. > I would not mind if this was true. That is if the LAYOUTRETURN was a very clear barrier where our client would "magically" completely plug the network interface and will not continue to send a single byte on the wire to *any* DS involved with the layout. That's fine. That is only allow sate 1 and 1.5 RPCs above. Some/all bytes where presented on the wire, until the LAYOUTRETURN, from which point all RPCs are hard aborted and not a single byte is sent. > Others contend that since the in-flight RPCs reference the returned > layout, the client is still 'using' the layout with these in-flight > requests, and can not call LAYOUTRETURN until all in-flight RPCs > return, with or without an error. > With our client code I don't see how the guaranty of 2 and 3 above will happen without actually implementing this here. So in principal I agree with your principle, I only do not agree with your practice. In your new code you are violating 2 and 3 which are not to be allowed. And again, please explain why do you want it. What is wrong with the case we all agree with? ie: "Client can not call LAYOUTRETURN until all in-flight RPCs return, with or without an error" Thanks Boaz > > Section 18.44.3 - the description section of the LAYOUTRETURN operation: > > After this call, > the client MUST NOT use the returned layout(s) and the associated > storage protocol to access the file data. > > Section 13.6 Operations Sent to NFSv4.1 Data Servers > > As described in Section 12.5.1, a client > MUST NOT send an I/O to a data server for which it does not hold a > valid layout; the data server MUST reject such an I/O. > > > -->Andy _______________________________________________ nfsv4 mailing list nfsv4@ietf.org https://www.ietf.org/mailman/listinfo/nfsv4 ^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: [nfsv4] RFC 5661 LAYOUTRETURN clarification. 2012-06-11 19:02 ` [nfsv4] " david.noveck @ 2012-06-11 19:43 ` Myklebust, Trond 2012-06-11 20:11 ` [nfsv4] RFC 5661 write to DS clarification. (was: [nfsv4] RFC 5661 LAYOUTRETURN clarification) Myklebust, Trond 0 siblings, 1 reply; 8+ messages in thread From: Myklebust, Trond @ 2012-06-11 19:43 UTC (permalink / raw) To: david.noveck@emc.com Cc: bharrosh@panasas.com, androsadamson@gmail.com, Adamson, Andy, linux-nfs@vger.kernel.org, nfsv4@ietf.org DQpUaGUgX29ubHlfIHJlYXNvbiB3aHkgYSBwTkZTIGZpbGVzIGNsaWVudCB3b3VsZCBldmVyIHdh bnQgdG8gc2VuZCBhDQpMQVlPVVRSRVRVUk4gaXMgaW4gb3JkZXIgdG8gaGF2ZSB0aGUgTURTIHRh a2UgYWN0aW9uIHRvIGZlbmNlIG9mZiBhbnkNCm91dHN0YW5kaW5nIHdyaXRlcyB0byB0aGUgRFMu DQoNClRoZSBfb25seV8gY2FzZSB3aGVyZSB0aGF0IGlzIGFjdHVhbGx5IGFuIGltcG9ydGFudCBp c3N1ZSBpcyB3aGVuDQpzb21ldGhpbmcgaGFwcGVucyB0byB0aGUgRFMgd2hpY2ggZm9yY2VzIHRo ZSBjbGllbnQgdG8gZmFsbCBiYWNrIHRvDQp3cml0aW5nIHRocm91Z2ggdGhlIE1EUy4NCg0KX0FM TF8gb3RoZXIgY2FzZXMgYXJlIHRyaXZpYWxseSBjb3ZlcmVkIGJ5IHRoZSBleGlzdGluZyBORlN2 NCBzdGF0ZQ0KbW9kZWwgaW4gdGhhdCB3aGVuIHRoZSBjbGllbnQgdW5sb2NrcyBhbmQvb3IgY2xv c2VzIHRoZSBmaWxlLCB0aGVuIHRoZQ0KbG9jay9vcGVuIHN0YXRlaWRzIHRoYXQgYXJlIHVzZWQg aW4gdGhlIFJFQUQgYW5kIFdSSVRFIG9wZXJhdGlvbnMgd2lsbA0KYmUgdXBkYXRlZCwgYW5kIHdp bGwgY2F1c2UgdGhvc2Ugb3BlcmF0aW9ucyB0byBiZSByZWplY3RlZCB3aXRoIGENCkJBRF9TVEFU RUlEIGVycm9yLiBUaGlzIGZlbmNpbmcgbW9kZWwgaXMgaXJyZXNwZWN0aXZlIG9mIHdoZXRoZXIg b3Igbm90DQphIGxheW91dCBpcyBoZWxkLCBhbmQgaXMgaXJyZXNwZWN0aXZlIG9mIHdoZXRoZXIg dGhlIFJFQUQvV1JJVEUgd2FzIHNlbnQNCnRvIHRoZSBNRFMgb3IgdGhlIERTLg0KDQoNCklPVzog aWYgcE5GUyBmaWxlcyBzZXJ2ZXJzIGRvbid0IHdhbnQgdG8gZG8gdGhpcyBraW5kIG9mIGZlbmNp bmcsIHRoZW4gSQ0Kc3VnZ2VzdCB3ZSBmaWxlIGFuIGVycmF0YSB0aGF0IGxhYmVscyB0aGUgTEFZ T1VUUkVUVVJOIG9wZXJhdGlvbiBhcw0KbWFuZGF0b3J5IHRvIG5vdCBpbXBsZW1lbnQgZm9yIHRo b3NlIHNlcnZlcnMuDQoNCk9uIE1vbiwgMjAxMi0wNi0xMSBhdCAxNTowMiAtMDQwMCwgZGF2aWQu bm92ZWNrQGVtYy5jb20gd3JvdGU6DQo+ID4gQW5kIGFnYWluLCBwbGVhc2UgZXhwbGFpbiB3aHkg ZG8geW91IHdhbnQgaXQuIFdoYXQgaXMgd3Jvbmcgd2l0aCB0aGUNCj4gPiBjYXNlIHdlIGFsbCBh Z3JlZSB3aXRoPyBpZTogIkNsaWVudCBjYW4gbm90IGNhbGwgTEFZT1VUUkVUVVJOIHVudGlsDQo+ ID4gYWxsIGluLWZsaWdodCBSUENzIHJldHVybiwgd2l0aCBvciB3aXRob3V0IGFuIGVycm9yIg0K PiANCj4gSXQncyBhIHJlY2lwZSBmb3IgZGF0YSBjb3JydXB0aW9uLiAgSWYsIGFzIEFuZHkgZXhw bGFpbmVkLCBoZSBzdGFydHMgZG9pbmcNCj4gSU8ncyAobGV0J3Mgc3VwcG9zZSBXUklURXMpIHRv IHRoZSBNRFMgYW55IGxpbmdlcmluZyBXUklURXMgdG8gdGhlIERTDQo+IHNpbmNlIHRoZXkgcmVm bGVjdCBhbiBlYXJsaWVyIHN0YXRlIG9mIGFmZmFpcnMgY2FuIGNhdXNlIGRhdGEgY29ycnVwdGlv bi4NCj4gDQo+IFRoZXJlIGFyZSB0aHJlZSB3YXlzIHRvIHByZXZlbnQgdGhvc2UgbGluZ2VyaW5n IERTIHdyaXRlcyBmcm9tIGNvcnJ1cHRpbmcgDQo+IGRhdGE6DQo+IA0KPiAxKSBEb2luZyBhIExB WU9VVFJFVFVSTg0KPiAyKSB3YWl0aW5nIHVudGlsIHRoZSBJTydzIHJldHVybi4NCj4gMykgIm1h Z2ljYWxseSBwbHVnZ2luZyB0aGUgbmV0d29yayBpbnRlcmZhY2UiLg0KPiANCj4gDQo+IFNpbmNl IHRoZXJlIGlzIG5vIHdheSB0byBkbyAzKSwgc2F5aW5nIHRoYXQgeW91IG9ubHkgY2FuIGRvIDEp IHVudGlsIGFmdGVyDQo+IDIpIGlzIGRvbmUgaXMgZXNzZW50aWFsbHkgZ29pbmcgdG8gbWVhbjoN Cj4gDQo+IGEpIHRoYXQgaXQgbWF5IHRha2UgYSB2ZXJ5IGxvbmcgdGltZToNCj4gYikgdGhhdCB5 b3Ugd2lsbCBvbmx5IGRvIGl0LCB3aGVuIGl0IGlzIG5vIGxvbmdlciB1c2VmdWwuDQo+IA0KPiBJ ZiB5b3UgZG8gMSkgYXNhcCwgdGhlbiB0aGUgbGluZ2VyaW5nIERTIHdyaXRlIHByb2JsZW0gaXMg Z29uZSBzb29uZXIsDQo+IGFuZCB0aGF0J3MgYSBnb29kIHRoaW5nLiANCj4gDQo+IC0tLS0tT3Jp Z2luYWwgTWVzc2FnZS0tLS0tDQo+IEZyb206IG5mc3Y0LWJvdW5jZXNAaWV0Zi5vcmcgW21haWx0 bzpuZnN2NC1ib3VuY2VzQGlldGYub3JnXSBPbiBCZWhhbGYgT2YgQm9heiBIYXJyb3NoDQo+IFNl bnQ6IE1vbmRheSwgSnVuZSAxMSwgMjAxMiAyOjQxIFBNDQo+IFRvOiBBbmR5IEFkYW1zb24NCj4g Q2M6IEFuZHkgQWRhbXNvbjsgTkZTIGxpc3Q7IFRyb25kIE15a2xlYnVzdDsgTkZTdjQNCj4gU3Vi amVjdDogUmU6IFtuZnN2NF0gUkZDIDU2NjEgTEFZT1VUUkVUVVJOIGNsYXJpZmljYXRpb24uDQo+ IA0KPiBPbiAwNi8xMS8yMDEyIDA3OjAxIFBNLCBBbmR5IEFkYW1zb24gd3JvdGU6DQo+IA0KPiA+ IEknbSBjb2RpbmcgZmlsZSBsYXlvdXQgZGF0YSBzZXJ2ZXIgcmVjb3ZlcnkgZm9yIHRoZSBMaW51 eCBORlMgY2xpZW50LA0KPiA+IGFuZCBjYW1lIGFjcm9zcyBhbiBpc3N1ZSB3aXRoIExBWU9VVFJF VFVSTiB0aGF0DQo+ID4gY291bGQgdXNlIHNvbWUgY29tbWVudCBmcm9tIHRoZSBsaXN0Lg0KPiA+ IA0KPiA+IFRoZSBlcnJvciBjYXNlIEknbSBoYW5kbGluZyBpcyBhbiBSUEMgbGF5ZXIgZGlzLWNv bm5lY3Rpb24gZXJyb3INCj4gPiBkdXJpbmcgaGVhdnkgV1JJVEUgaS9vIHRvIGEgZmlsZSBsYXlv dXQgZGF0YSBzZXJ2ZXIuIE91ciByZXNwb25zZSBpcw0KPiA+IHRvIGludGVybmFsbHkgbWFyayB0 aGUgZGV2aWNlaWQgYXMgaW52YWxpZCB3aGljaCBwcmV2ZW50cyBhbGwgcE5GUw0KPiA+IGNhbGxz IHVzaW5nIHRoZSBkZXZpY2VpZCAtIGUuZy4gbm8gbmV3IEkvTyB1c2luZyBhbnkgbGF5b3V0IHRo YXQgdXNlcw0KPiA+IHRoZSBpbnZhbGlkIGRldmljZWlkLCBhbmQgdG8gcmVkaXJlY3QgYWxsIEkv TyB0byB0aGUgTURTIChhbnkgcXVldWVkDQo+ID4gUlBDIHJlcXVlc3QgdGhhdCBoYXMgbm90IGJl ZW4gc2VudCBpcyByZWRpcmVjdGVkIHRvIHRoZSBNRFMpLg0KPiA+IA0KPiA+IFBsdXMgLSBhbmQg aGVyZSBpcyB3aGVyZSB0aGUgY2xhcmlmaWNhdGlvbiBpcyBuZWVkZWQgLSB3ZSBpbW1lZGlhdGVs eQ0KPiA+IHNlbmQgYSBMQVlPVVRSRVRVUk4gZm9yIGFueSBsYXlvdXQgd2l0aCBpbi1mbGlnaHQg cmVxdWVzdHMgdG8gdGhlDQo+ID4gZGlzLWNvbm5lY3RlZCBkYXRhIHNlcnZlci4gIEJ5IGluLWZs aWdodCBJIG1lYW4gdHJhbnNtaXR0ZWQgV1JUIHRoZQ0KPiA+IFJQQyBsYXllci4gIFRoZSBwdXJw b3NlIG9mIHRoaXMgTEFZT1VUUkVUVVJOIGlzIHRvIG5vdGlmeSB0aGUgZmlsZQ0KPiA+IGxheW91 dCBNRFMgdG8gZmVuY2UgdGhlIERTIGZvciB0aGUgc3BlY2lmaWVkIExBWU9VVHMsIGFzIHRoZSBX UklURXMNCj4gPiB3aWxsIGFsc28gYmUgc2VudCB0byB0aGUgTURTLg0KPiA+IA0KPiANCj4gDQo+ IEkgZG8gbm90IGRpc2FncmVlIHdpdGggdGhpcyBjb21wbGV0ZWx5LiBUaGUgcG9pbnQgaGVyZSBp cyB2ZXJ5IGZpbmUNCj4gZ3JhaW5lZCBhbmQgc2hvdWxkIGJlIHNwZWNpZmllZCBleHBsaWNpdGx5 LiBJIHdvdWxkIGxpa2UgdG8gc2VlIHRleHQNCj4gYXMgb2Ygc29tZXRoaW5nIGxpa2UuDQo+IA0K PiBUaGVyZSBhcmUgMyB0eXBlcyBvZiBpbi1mbGdodCBSUEMvSU8NCj4gMS4gQ2xpZW50IGhhcyBz ZW50IFJQQyBoZWFkZXIgKyBhbGwgb2YgYXNzb2NpYXRlZCBkYXRhIGFuZCBpcyB3YWl0aW5nDQo+ ICAgIGZvciBEUyBXUklURS9SRUFEX0RPTkUgcmVwbHkuDQo+IA0KPiAgICAoRm9yIG1lIHRoaXMg Y2FzZSBjYW4gYmUsIGNsaWVudCBtYXkgcmV0dXJuIExBWU9VVFJFVFVSTiBhcyB5b3VyDQo+ICAg ICBzdWdnZXN0aW9uKQ0KPiANCj4gMi4gQ2xpZW50IGhhcyBzZW50IHRoZSBSUEMgaGVhZGVyIGJ1 dCBoYXMgZ290IHN0dWNrIHNlbmRpbmcgdGhlIHJlc3QNCj4gICAgb2YgdGhlIFJQQyBtZXNzYWdl LiBUaGVuIHJlY2VpdmVkIGEgbmV0d29yayBkaXNjb25uZWN0LiBUaGlzIGlzIHRoZQ0KPiAgICBt b3N0IGNvbW1vbiBwYXJ0LiBQdXR0aW5nIGFzaWRlIHRoZSBSUEMgdGhhdCBnb3QgdGhlIGVycm9y IGZvciBhIHNlY29uZC4NCj4gICAgVGhlIG1vc3QgaW1wb3J0YW50IGlzIHdoYXQgdG8gZG8gd2l0 aCBwYXJhbGxlbCBSUEMvSU8gd2hpY2ggYXJlIGluIHRoaXMNCj4gICAgc3RhdGUuIEFyZSBwYXJh bGxlbCBSUENzIGFsbG93ZWQgdG8gY29udGludWUgc2VuZGluZyBuZXR3b3JrIHBhY2tldHMNCj4g ICAgYWZ0ZXIgdGhlIExBWU9VVFJFVFVSTiB3YXMgc2VudD8NCj4gDQo+ICAgIFRoZSBzcGVjaWZp YyBSUEMgdGhhdCBnb3Qgc3R1Y2sgaXMgbm90IGludGVyZXN0aW5nIGJlY2F1c2UgaXQncyBraW5k IG9mDQo+ICAgIDEuNSwgV2UgYXJlIG5vdCBnb2luZyB0byBzZW5kIGFueSBieXRlcyBvbiB0aGF0 IGNoYW5uZWwuIFRoZSBpbnRlcmVzdGluZw0KPiAgICBpcyB0aGVzZSBvdGhlciBEU3Mgd2hpY2gg YXJlIHN0aWxsIHN0cmVhbWluZw0KPiANCj4gMy4gQ2xpZW50IGhhcyBzb21lIGludGVybmFsIFJQ QyBxdWV1ZSB3aGljaCBkbyB0byBzb21lIGNsaWVudCBwYXJhbGxlbGlzbQ0KPiAgICB3aWxsIHN0 YXJ0IHNlbmRpbmcgUlBDIGhlYWRlciArIGRhdGEgYWZ0ZXIgdGhlIExBWU9VVFJFVFVSTiB3YXMg c2VudA0KPiAgICANCj4gV2hhdCBteSBwb2ludCB3YXMgdGhhdCB3aXRoIHRoZSBjb2RlIHlvdSBz dWJtaXR0ZWQgd2UgYXJlIGNsZWFybHkgdmlvbGF0aW5nDQo+IDIuIGFuZCBldmVuIDMuIEJlY2F1 c2UgSSBkbyBub3Qgc2VlIGFueXRoaW5nIGF2b2lkaW5nIHRoaXMuDQo+IA0KPiBBbmQgaWYgdGhl IFNURCBhbGxvd3MgeW91IDIgYW5kIDMuIFRoZW4gdGhhdCdzIGEgYmlnIGNoYW5nZSB0byB0aGUg Y29uY2VwdC4NCj4gTm90IGxpa2UgeW91IGxldCBpdCBzZWVtLg0KPiANCj4gPiBJIGNvbnRlbmQg dGhhdCBzZW5kaW5nIHRoZSBMQVlPVVRSRVRVUk4gaW4gdGhpcyBlcnJvciBjYXNlIGRvZXMgbm90 DQo+ID4gdmlvbGF0ZSB0aGUgdHdvIHNlY3Rpb25zIG9mIFJGQyA1NjYxIGJlbG93LCBhcyB0aGUg Y2xpZW50IGhhcyBzdG9wcGVkDQo+ID4gc2VuZGluZyBhbnkgSS9PIHJlcXVlc3RzIHVzaW5nIHRo ZSByZXR1cm5lZCBsYXlvdXQuDQo+ID4gDQo+IA0KPiANCj4gSSB3b3VsZCBub3QgbWluZCBpZiB0 aGlzIHdhcyB0cnVlLiBUaGF0IGlzIGlmIHRoZSBMQVlPVVRSRVRVUk4gd2FzDQo+IGEgdmVyeSBj bGVhciBiYXJyaWVyIHdoZXJlIG91ciBjbGllbnQgd291bGQgIm1hZ2ljYWxseSIgY29tcGxldGVs eQ0KPiBwbHVnIHRoZSBuZXR3b3JrIGludGVyZmFjZSBhbmQgd2lsbCBub3QgY29udGludWUgdG8g c2VuZCBhIHNpbmdsZQ0KPiBieXRlIG9uIHRoZSB3aXJlIHRvICphbnkqIERTIGludm9sdmVkIHdp dGggdGhlIGxheW91dC4gVGhhdCdzIGZpbmUuDQo+IA0KPiBUaGF0IGlzIG9ubHkgYWxsb3cgc2F0 ZSAxIGFuZCAxLjUgUlBDcyBhYm92ZS4gU29tZS9hbGwgYnl0ZXMgd2hlcmUNCj4gcHJlc2VudGVk IG9uIHRoZSB3aXJlLCB1bnRpbCB0aGUgTEFZT1VUUkVUVVJOLCBmcm9tIHdoaWNoIHBvaW50IGFs bA0KPiBSUENzIGFyZSBoYXJkIGFib3J0ZWQgYW5kIG5vdCBhIHNpbmdsZSBieXRlIGlzIHNlbnQu DQo+IA0KPiANCj4gPiBPdGhlcnMgY29udGVuZCB0aGF0IHNpbmNlIHRoZSBpbi1mbGlnaHQgUlBD cyByZWZlcmVuY2UgdGhlIHJldHVybmVkDQo+ID4gbGF5b3V0LCB0aGUgY2xpZW50IGlzIHN0aWxs ICd1c2luZycgdGhlIGxheW91dCB3aXRoIHRoZXNlIGluLWZsaWdodA0KPiA+IHJlcXVlc3RzLCBh bmQgY2FuIG5vdCBjYWxsIExBWU9VVFJFVFVSTiB1bnRpbCBhbGwgaW4tZmxpZ2h0IFJQQ3MNCj4g PiByZXR1cm4sIHdpdGggb3Igd2l0aG91dCBhbiBlcnJvci4NCj4gPiANCj4gDQo+IA0KPiBXaXRo IG91ciBjbGllbnQgY29kZSBJIGRvbid0IHNlZSBob3cgdGhlIGd1YXJhbnR5IG9mIDIgYW5kIDMg YWJvdmUNCj4gd2lsbCBoYXBwZW4gd2l0aG91dCBhY3R1YWxseSBpbXBsZW1lbnRpbmcgdGhpcyBo ZXJlLg0KPiANCj4gU28gaW4gcHJpbmNpcGFsIEkgYWdyZWUgd2l0aCB5b3VyIHByaW5jaXBsZSwg SSBvbmx5IGRvIG5vdCBhZ3JlZQ0KPiB3aXRoIHlvdXIgcHJhY3RpY2UuIEluIHlvdXIgbmV3IGNv ZGUgeW91IGFyZSB2aW9sYXRpbmcgMiBhbmQgMw0KPiB3aGljaCBhcmUgbm90IHRvIGJlIGFsbG93 ZWQuDQo+IA0KPiBBbmQgYWdhaW4sIHBsZWFzZSBleHBsYWluIHdoeSBkbyB5b3Ugd2FudCBpdC4g V2hhdCBpcyB3cm9uZyB3aXRoIHRoZQ0KPiBjYXNlIHdlIGFsbCBhZ3JlZSB3aXRoPyBpZTogIkNs aWVudCBjYW4gbm90IGNhbGwgTEFZT1VUUkVUVVJOIHVudGlsDQo+IGFsbCBpbi1mbGlnaHQgUlBD cyByZXR1cm4sIHdpdGggb3Igd2l0aG91dCBhbiBlcnJvciINCj4gDQo+IFRoYW5rcw0KPiBCb2F6 DQo+IA0KPiA+IA0KPiA+IFNlY3Rpb24gMTguNDQuMyAtIHRoZSBkZXNjcmlwdGlvbiBzZWN0aW9u IG9mIHRoZSBMQVlPVVRSRVRVUk4gb3BlcmF0aW9uOg0KPiA+IA0KPiA+ICAgIEFmdGVyIHRoaXMg Y2FsbCwNCj4gPiAgICB0aGUgY2xpZW50IE1VU1QgTk9UIHVzZSB0aGUgcmV0dXJuZWQgbGF5b3V0 KHMpIGFuZCB0aGUgYXNzb2NpYXRlZA0KPiA+ICAgIHN0b3JhZ2UgcHJvdG9jb2wgdG8gYWNjZXNz IHRoZSBmaWxlIGRhdGEuDQo+ID4gDQo+ID4gU2VjdGlvbiAxMy42IE9wZXJhdGlvbnMgU2VudCB0 byBORlN2NC4xIERhdGEgU2VydmVycw0KPiA+IA0KPiA+ICAgQXMgZGVzY3JpYmVkIGluIFNlY3Rp b24gMTIuNS4xLCBhIGNsaWVudA0KPiA+ICAgTVVTVCBOT1Qgc2VuZCBhbiBJL08gdG8gYSBkYXRh IHNlcnZlciBmb3Igd2hpY2ggaXQgZG9lcyBub3QgaG9sZCBhDQo+ID4gICB2YWxpZCBsYXlvdXQ7 IHRoZSBkYXRhIHNlcnZlciBNVVNUIHJlamVjdCBzdWNoIGFuIEkvTy4NCj4gPiANCj4gPiANCj4g PiAtLT5BbmR5DQo+IA0KPiANCj4gX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19f X19fX19fX19fX18NCj4gbmZzdjQgbWFpbGluZyBsaXN0DQo+IG5mc3Y0QGlldGYub3JnDQo+IGh0 dHBzOi8vd3d3LmlldGYub3JnL21haWxtYW4vbGlzdGluZm8vbmZzdjQNCj4gDQoNCi0tIA0KVHJv bmQgTXlrbGVidXN0DQpMaW51eCBORlMgY2xpZW50IG1haW50YWluZXINCg0KTmV0QXBwDQpUcm9u ZC5NeWtsZWJ1c3RAbmV0YXBwLmNvbQ0Kd3d3Lm5ldGFwcC5jb20NCg0K ^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: [nfsv4] RFC 5661 write to DS clarification. (was: [nfsv4] RFC 5661 LAYOUTRETURN clarification) 2012-06-11 19:43 ` Myklebust, Trond @ 2012-06-11 20:11 ` Myklebust, Trond 0 siblings, 0 replies; 8+ messages in thread From: Myklebust, Trond @ 2012-06-11 20:11 UTC (permalink / raw) To: linux-nfs@vger.kernel.org, nfsv4@ietf.org Cc: bharrosh@panasas.com, androsadamson@gmail.com, Adamson, Andy, david.noveck@emc.com QlRXOiANCg0KSSBmb3Jnb3QgdG8gYWRkIHRoYXQgdGhlIGZlbmNpbmcgaXNzdWVzIGFyZSBhbHNv IHRoZSByZWFzb24gd2h5IHRoZQ0KTGludXggY2xpZW50IGlzIHVubGlrZWx5IHRvIGNvbXBseSBh bnkgdGltZSBzb29uIHdpdGggUkZDNTY2MSBTZWN0aW9uDQoxMy45LjEuJ3MgcmVxdWVzdCB0aGF0 IHdlIHByZWZlciB1c2Ugb2YgdGhlIE9QRU4gc3RhdGVpZCBvdmVyIHRoZSBMT0NLDQpzdGF0ZWlk IHdoZW4gdGFsa2luZyB0byB0aGUgRFMuDQpJZiB0aGUgc2VydmVyIHJldm9rZXMgdGhlIGxvY2ss IG9yIGlmIHRoZSBjbGllbnQgY2FsbHMgTE9DS1UsIGFsbCBXUklURXMNCnRoYXQgd2VyZSBtYWRl IHVuZGVyIHRoYXQgbG9jayBuZWVkIHRvIGJlIGZlbmNlZCBvZmYuIFVubGVzcyBtYW5kYXRvcnkN CmxvY2tpbmcgaXMgaW4gZWZmZWN0LCB0aGF0IHdvbid0IGhhcHBlbiBpZiB0aGUgV1JJVEUgb3Bz IHdlcmUgc2VudCB1c2luZw0KdGhlIE9QRU4gc3RhdGVpZC4NCg0KVGhpcyBpcyBhbHNvIHdoeSBJ IGJlbGlldmUgd2Ugc2hvdWxkIHJldmlzaXQgdGhlIHJ1bGUgdGhhdCB0aGUgY2xpZW50DQpzaG91 bGQgb25seSBzZW5kIHN0YXRlaWRzIHdpdGggYSB6ZXJvIHNlcWlkIHRvIHRoZSBEUy4NCg0KQ2hl ZXJzDQogIFRyb25kDQoNCk9uIE1vbiwgMjAxMi0wNi0xMSBhdCAxNTo0MyAtMDQwMCwgVHJvbmQg TXlrbGVidXN0IHdyb3RlOg0KPiBUaGUgX29ubHlfIHJlYXNvbiB3aHkgYSBwTkZTIGZpbGVzIGNs aWVudCB3b3VsZCBldmVyIHdhbnQgdG8gc2VuZCBhDQo+IExBWU9VVFJFVFVSTiBpcyBpbiBvcmRl ciB0byBoYXZlIHRoZSBNRFMgdGFrZSBhY3Rpb24gdG8gZmVuY2Ugb2ZmIGFueQ0KPiBvdXRzdGFu ZGluZyB3cml0ZXMgdG8gdGhlIERTLg0KPiANCj4gVGhlIF9vbmx5XyBjYXNlIHdoZXJlIHRoYXQg aXMgYWN0dWFsbHkgYW4gaW1wb3J0YW50IGlzc3VlIGlzIHdoZW4NCj4gc29tZXRoaW5nIGhhcHBl bnMgdG8gdGhlIERTIHdoaWNoIGZvcmNlcyB0aGUgY2xpZW50IHRvIGZhbGwgYmFjayB0bw0KPiB3 cml0aW5nIHRocm91Z2ggdGhlIE1EUy4NCj4gDQo+IF9BTExfIG90aGVyIGNhc2VzIGFyZSB0cml2 aWFsbHkgY292ZXJlZCBieSB0aGUgZXhpc3RpbmcgTkZTdjQgc3RhdGUNCj4gbW9kZWwgaW4gdGhh dCB3aGVuIHRoZSBjbGllbnQgdW5sb2NrcyBhbmQvb3IgY2xvc2VzIHRoZSBmaWxlLCB0aGVuIHRo ZQ0KPiBsb2NrL29wZW4gc3RhdGVpZHMgdGhhdCBhcmUgdXNlZCBpbiB0aGUgUkVBRCBhbmQgV1JJ VEUgb3BlcmF0aW9ucyB3aWxsDQo+IGJlIHVwZGF0ZWQsIGFuZCB3aWxsIGNhdXNlIHRob3NlIG9w ZXJhdGlvbnMgdG8gYmUgcmVqZWN0ZWQgd2l0aCBhDQo+IEJBRF9TVEFURUlEIGVycm9yLiBUaGlz IGZlbmNpbmcgbW9kZWwgaXMgaXJyZXNwZWN0aXZlIG9mIHdoZXRoZXIgb3Igbm90DQo+IGEgbGF5 b3V0IGlzIGhlbGQsIGFuZCBpcyBpcnJlc3BlY3RpdmUgb2Ygd2hldGhlciB0aGUgUkVBRC9XUklU RSB3YXMgc2VudA0KPiB0byB0aGUgTURTIG9yIHRoZSBEUy4NCj4gDQo+IA0KPiBJT1c6IGlmIHBO RlMgZmlsZXMgc2VydmVycyBkb24ndCB3YW50IHRvIGRvIHRoaXMga2luZCBvZiBmZW5jaW5nLCB0 aGVuIEkNCj4gc3VnZ2VzdCB3ZSBmaWxlIGFuIGVycmF0YSB0aGF0IGxhYmVscyB0aGUgTEFZT1VU UkVUVVJOIG9wZXJhdGlvbiBhcw0KPiBtYW5kYXRvcnkgdG8gbm90IGltcGxlbWVudCBmb3IgdGhv c2Ugc2VydmVycy4NCj4gDQo+IE9uIE1vbiwgMjAxMi0wNi0xMSBhdCAxNTowMiAtMDQwMCwgZGF2 aWQubm92ZWNrQGVtYy5jb20gd3JvdGU6DQo+ID4gPiBBbmQgYWdhaW4sIHBsZWFzZSBleHBsYWlu IHdoeSBkbyB5b3Ugd2FudCBpdC4gV2hhdCBpcyB3cm9uZyB3aXRoIHRoZQ0KPiA+ID4gY2FzZSB3 ZSBhbGwgYWdyZWUgd2l0aD8gaWU6ICJDbGllbnQgY2FuIG5vdCBjYWxsIExBWU9VVFJFVFVSTiB1 bnRpbA0KPiA+ID4gYWxsIGluLWZsaWdodCBSUENzIHJldHVybiwgd2l0aCBvciB3aXRob3V0IGFu IGVycm9yIg0KPiA+IA0KPiA+IEl0J3MgYSByZWNpcGUgZm9yIGRhdGEgY29ycnVwdGlvbi4gIElm LCBhcyBBbmR5IGV4cGxhaW5lZCwgaGUgc3RhcnRzIGRvaW5nDQo+ID4gSU8ncyAobGV0J3Mgc3Vw cG9zZSBXUklURXMpIHRvIHRoZSBNRFMgYW55IGxpbmdlcmluZyBXUklURXMgdG8gdGhlIERTDQo+ ID4gc2luY2UgdGhleSByZWZsZWN0IGFuIGVhcmxpZXIgc3RhdGUgb2YgYWZmYWlycyBjYW4gY2F1 c2UgZGF0YSBjb3JydXB0aW9uLg0KPiA+IA0KPiA+IFRoZXJlIGFyZSB0aHJlZSB3YXlzIHRvIHBy ZXZlbnQgdGhvc2UgbGluZ2VyaW5nIERTIHdyaXRlcyBmcm9tIGNvcnJ1cHRpbmcgDQo+ID4gZGF0 YToNCj4gPiANCj4gPiAxKSBEb2luZyBhIExBWU9VVFJFVFVSTg0KPiA+IDIpIHdhaXRpbmcgdW50 aWwgdGhlIElPJ3MgcmV0dXJuLg0KPiA+IDMpICJtYWdpY2FsbHkgcGx1Z2dpbmcgdGhlIG5ldHdv cmsgaW50ZXJmYWNlIi4NCj4gPiANCj4gPiANCj4gPiBTaW5jZSB0aGVyZSBpcyBubyB3YXkgdG8g ZG8gMyksIHNheWluZyB0aGF0IHlvdSBvbmx5IGNhbiBkbyAxKSB1bnRpbCBhZnRlcg0KPiA+IDIp IGlzIGRvbmUgaXMgZXNzZW50aWFsbHkgZ29pbmcgdG8gbWVhbjoNCj4gPiANCj4gPiBhKSB0aGF0 IGl0IG1heSB0YWtlIGEgdmVyeSBsb25nIHRpbWU6DQo+ID4gYikgdGhhdCB5b3Ugd2lsbCBvbmx5 IGRvIGl0LCB3aGVuIGl0IGlzIG5vIGxvbmdlciB1c2VmdWwuDQo+ID4gDQo+ID4gSWYgeW91IGRv IDEpIGFzYXAsIHRoZW4gdGhlIGxpbmdlcmluZyBEUyB3cml0ZSBwcm9ibGVtIGlzIGdvbmUgc29v bmVyLA0KPiA+IGFuZCB0aGF0J3MgYSBnb29kIHRoaW5nLiANCj4gPiANCj4gPiAtLS0tLU9yaWdp bmFsIE1lc3NhZ2UtLS0tLQ0KPiA+IEZyb206IG5mc3Y0LWJvdW5jZXNAaWV0Zi5vcmcgW21haWx0 bzpuZnN2NC1ib3VuY2VzQGlldGYub3JnXSBPbiBCZWhhbGYgT2YgQm9heiBIYXJyb3NoDQo+ID4g U2VudDogTW9uZGF5LCBKdW5lIDExLCAyMDEyIDI6NDEgUE0NCj4gPiBUbzogQW5keSBBZGFtc29u DQo+ID4gQ2M6IEFuZHkgQWRhbXNvbjsgTkZTIGxpc3Q7IFRyb25kIE15a2xlYnVzdDsgTkZTdjQN Cj4gPiBTdWJqZWN0OiBSZTogW25mc3Y0XSBSRkMgNTY2MSBMQVlPVVRSRVRVUk4gY2xhcmlmaWNh dGlvbi4NCj4gPiANCj4gPiBPbiAwNi8xMS8yMDEyIDA3OjAxIFBNLCBBbmR5IEFkYW1zb24gd3Jv dGU6DQo+ID4gDQo+ID4gPiBJJ20gY29kaW5nIGZpbGUgbGF5b3V0IGRhdGEgc2VydmVyIHJlY292 ZXJ5IGZvciB0aGUgTGludXggTkZTIGNsaWVudCwNCj4gPiA+IGFuZCBjYW1lIGFjcm9zcyBhbiBp c3N1ZSB3aXRoIExBWU9VVFJFVFVSTiB0aGF0DQo+ID4gPiBjb3VsZCB1c2Ugc29tZSBjb21tZW50 IGZyb20gdGhlIGxpc3QuDQo+ID4gPiANCj4gPiA+IFRoZSBlcnJvciBjYXNlIEknbSBoYW5kbGlu ZyBpcyBhbiBSUEMgbGF5ZXIgZGlzLWNvbm5lY3Rpb24gZXJyb3INCj4gPiA+IGR1cmluZyBoZWF2 eSBXUklURSBpL28gdG8gYSBmaWxlIGxheW91dCBkYXRhIHNlcnZlci4gT3VyIHJlc3BvbnNlIGlz DQo+ID4gPiB0byBpbnRlcm5hbGx5IG1hcmsgdGhlIGRldmljZWlkIGFzIGludmFsaWQgd2hpY2gg cHJldmVudHMgYWxsIHBORlMNCj4gPiA+IGNhbGxzIHVzaW5nIHRoZSBkZXZpY2VpZCAtIGUuZy4g bm8gbmV3IEkvTyB1c2luZyBhbnkgbGF5b3V0IHRoYXQgdXNlcw0KPiA+ID4gdGhlIGludmFsaWQg ZGV2aWNlaWQsIGFuZCB0byByZWRpcmVjdCBhbGwgSS9PIHRvIHRoZSBNRFMgKGFueSBxdWV1ZWQN Cj4gPiA+IFJQQyByZXF1ZXN0IHRoYXQgaGFzIG5vdCBiZWVuIHNlbnQgaXMgcmVkaXJlY3RlZCB0 byB0aGUgTURTKS4NCj4gPiA+IA0KPiA+ID4gUGx1cyAtIGFuZCBoZXJlIGlzIHdoZXJlIHRoZSBj bGFyaWZpY2F0aW9uIGlzIG5lZWRlZCAtIHdlIGltbWVkaWF0ZWx5DQo+ID4gPiBzZW5kIGEgTEFZ T1VUUkVUVVJOIGZvciBhbnkgbGF5b3V0IHdpdGggaW4tZmxpZ2h0IHJlcXVlc3RzIHRvIHRoZQ0K PiA+ID4gZGlzLWNvbm5lY3RlZCBkYXRhIHNlcnZlci4gIEJ5IGluLWZsaWdodCBJIG1lYW4gdHJh bnNtaXR0ZWQgV1JUIHRoZQ0KPiA+ID4gUlBDIGxheWVyLiAgVGhlIHB1cnBvc2Ugb2YgdGhpcyBM QVlPVVRSRVRVUk4gaXMgdG8gbm90aWZ5IHRoZSBmaWxlDQo+ID4gPiBsYXlvdXQgTURTIHRvIGZl bmNlIHRoZSBEUyBmb3IgdGhlIHNwZWNpZmllZCBMQVlPVVRzLCBhcyB0aGUgV1JJVEVzDQo+ID4g PiB3aWxsIGFsc28gYmUgc2VudCB0byB0aGUgTURTLg0KPiA+ID4gDQo+ID4gDQo+ID4gDQo+ID4g SSBkbyBub3QgZGlzYWdyZWUgd2l0aCB0aGlzIGNvbXBsZXRlbHkuIFRoZSBwb2ludCBoZXJlIGlz IHZlcnkgZmluZQ0KPiA+IGdyYWluZWQgYW5kIHNob3VsZCBiZSBzcGVjaWZpZWQgZXhwbGljaXRs eS4gSSB3b3VsZCBsaWtlIHRvIHNlZSB0ZXh0DQo+ID4gYXMgb2Ygc29tZXRoaW5nIGxpa2UuDQo+ ID4gDQo+ID4gVGhlcmUgYXJlIDMgdHlwZXMgb2YgaW4tZmxnaHQgUlBDL0lPDQo+ID4gMS4gQ2xp ZW50IGhhcyBzZW50IFJQQyBoZWFkZXIgKyBhbGwgb2YgYXNzb2NpYXRlZCBkYXRhIGFuZCBpcyB3 YWl0aW5nDQo+ID4gICAgZm9yIERTIFdSSVRFL1JFQURfRE9ORSByZXBseS4NCj4gPiANCj4gPiAg ICAoRm9yIG1lIHRoaXMgY2FzZSBjYW4gYmUsIGNsaWVudCBtYXkgcmV0dXJuIExBWU9VVFJFVFVS TiBhcyB5b3VyDQo+ID4gICAgIHN1Z2dlc3Rpb24pDQo+ID4gDQo+ID4gMi4gQ2xpZW50IGhhcyBz ZW50IHRoZSBSUEMgaGVhZGVyIGJ1dCBoYXMgZ290IHN0dWNrIHNlbmRpbmcgdGhlIHJlc3QNCj4g PiAgICBvZiB0aGUgUlBDIG1lc3NhZ2UuIFRoZW4gcmVjZWl2ZWQgYSBuZXR3b3JrIGRpc2Nvbm5l Y3QuIFRoaXMgaXMgdGhlDQo+ID4gICAgbW9zdCBjb21tb24gcGFydC4gUHV0dGluZyBhc2lkZSB0 aGUgUlBDIHRoYXQgZ290IHRoZSBlcnJvciBmb3IgYSBzZWNvbmQuDQo+ID4gICAgVGhlIG1vc3Qg aW1wb3J0YW50IGlzIHdoYXQgdG8gZG8gd2l0aCBwYXJhbGxlbCBSUEMvSU8gd2hpY2ggYXJlIGlu IHRoaXMNCj4gPiAgICBzdGF0ZS4gQXJlIHBhcmFsbGVsIFJQQ3MgYWxsb3dlZCB0byBjb250aW51 ZSBzZW5kaW5nIG5ldHdvcmsgcGFja2V0cw0KPiA+ICAgIGFmdGVyIHRoZSBMQVlPVVRSRVRVUk4g d2FzIHNlbnQ/DQo+ID4gDQo+ID4gICAgVGhlIHNwZWNpZmljIFJQQyB0aGF0IGdvdCBzdHVjayBp cyBub3QgaW50ZXJlc3RpbmcgYmVjYXVzZSBpdCdzIGtpbmQgb2YNCj4gPiAgICAxLjUsIFdlIGFy ZSBub3QgZ29pbmcgdG8gc2VuZCBhbnkgYnl0ZXMgb24gdGhhdCBjaGFubmVsLiBUaGUgaW50ZXJl c3RpbmcNCj4gPiAgICBpcyB0aGVzZSBvdGhlciBEU3Mgd2hpY2ggYXJlIHN0aWxsIHN0cmVhbWlu Zw0KPiA+IA0KPiA+IDMuIENsaWVudCBoYXMgc29tZSBpbnRlcm5hbCBSUEMgcXVldWUgd2hpY2gg ZG8gdG8gc29tZSBjbGllbnQgcGFyYWxsZWxpc20NCj4gPiAgICB3aWxsIHN0YXJ0IHNlbmRpbmcg UlBDIGhlYWRlciArIGRhdGEgYWZ0ZXIgdGhlIExBWU9VVFJFVFVSTiB3YXMgc2VudA0KPiA+ICAg IA0KPiA+IFdoYXQgbXkgcG9pbnQgd2FzIHRoYXQgd2l0aCB0aGUgY29kZSB5b3Ugc3VibWl0dGVk IHdlIGFyZSBjbGVhcmx5IHZpb2xhdGluZw0KPiA+IDIuIGFuZCBldmVuIDMuIEJlY2F1c2UgSSBk byBub3Qgc2VlIGFueXRoaW5nIGF2b2lkaW5nIHRoaXMuDQo+ID4gDQo+ID4gQW5kIGlmIHRoZSBT VEQgYWxsb3dzIHlvdSAyIGFuZCAzLiBUaGVuIHRoYXQncyBhIGJpZyBjaGFuZ2UgdG8gdGhlIGNv bmNlcHQuDQo+ID4gTm90IGxpa2UgeW91IGxldCBpdCBzZWVtLg0KPiA+IA0KPiA+ID4gSSBjb250 ZW5kIHRoYXQgc2VuZGluZyB0aGUgTEFZT1VUUkVUVVJOIGluIHRoaXMgZXJyb3IgY2FzZSBkb2Vz IG5vdA0KPiA+ID4gdmlvbGF0ZSB0aGUgdHdvIHNlY3Rpb25zIG9mIFJGQyA1NjYxIGJlbG93LCBh cyB0aGUgY2xpZW50IGhhcyBzdG9wcGVkDQo+ID4gPiBzZW5kaW5nIGFueSBJL08gcmVxdWVzdHMg dXNpbmcgdGhlIHJldHVybmVkIGxheW91dC4NCj4gPiA+IA0KPiA+IA0KPiA+IA0KPiA+IEkgd291 bGQgbm90IG1pbmQgaWYgdGhpcyB3YXMgdHJ1ZS4gVGhhdCBpcyBpZiB0aGUgTEFZT1VUUkVUVVJO IHdhcw0KPiA+IGEgdmVyeSBjbGVhciBiYXJyaWVyIHdoZXJlIG91ciBjbGllbnQgd291bGQgIm1h Z2ljYWxseSIgY29tcGxldGVseQ0KPiA+IHBsdWcgdGhlIG5ldHdvcmsgaW50ZXJmYWNlIGFuZCB3 aWxsIG5vdCBjb250aW51ZSB0byBzZW5kIGEgc2luZ2xlDQo+ID4gYnl0ZSBvbiB0aGUgd2lyZSB0 byAqYW55KiBEUyBpbnZvbHZlZCB3aXRoIHRoZSBsYXlvdXQuIFRoYXQncyBmaW5lLg0KPiA+IA0K PiA+IFRoYXQgaXMgb25seSBhbGxvdyBzYXRlIDEgYW5kIDEuNSBSUENzIGFib3ZlLiBTb21lL2Fs bCBieXRlcyB3aGVyZQ0KPiA+IHByZXNlbnRlZCBvbiB0aGUgd2lyZSwgdW50aWwgdGhlIExBWU9V VFJFVFVSTiwgZnJvbSB3aGljaCBwb2ludCBhbGwNCj4gPiBSUENzIGFyZSBoYXJkIGFib3J0ZWQg YW5kIG5vdCBhIHNpbmdsZSBieXRlIGlzIHNlbnQuDQo+ID4gDQo+ID4gDQo+ID4gPiBPdGhlcnMg Y29udGVuZCB0aGF0IHNpbmNlIHRoZSBpbi1mbGlnaHQgUlBDcyByZWZlcmVuY2UgdGhlIHJldHVy bmVkDQo+ID4gPiBsYXlvdXQsIHRoZSBjbGllbnQgaXMgc3RpbGwgJ3VzaW5nJyB0aGUgbGF5b3V0 IHdpdGggdGhlc2UgaW4tZmxpZ2h0DQo+ID4gPiByZXF1ZXN0cywgYW5kIGNhbiBub3QgY2FsbCBM QVlPVVRSRVRVUk4gdW50aWwgYWxsIGluLWZsaWdodCBSUENzDQo+ID4gPiByZXR1cm4sIHdpdGgg b3Igd2l0aG91dCBhbiBlcnJvci4NCj4gPiA+IA0KPiA+IA0KPiA+IA0KPiA+IFdpdGggb3VyIGNs aWVudCBjb2RlIEkgZG9uJ3Qgc2VlIGhvdyB0aGUgZ3VhcmFudHkgb2YgMiBhbmQgMyBhYm92ZQ0K PiA+IHdpbGwgaGFwcGVuIHdpdGhvdXQgYWN0dWFsbHkgaW1wbGVtZW50aW5nIHRoaXMgaGVyZS4N Cj4gPiANCj4gPiBTbyBpbiBwcmluY2lwYWwgSSBhZ3JlZSB3aXRoIHlvdXIgcHJpbmNpcGxlLCBJ IG9ubHkgZG8gbm90IGFncmVlDQo+ID4gd2l0aCB5b3VyIHByYWN0aWNlLiBJbiB5b3VyIG5ldyBj b2RlIHlvdSBhcmUgdmlvbGF0aW5nIDIgYW5kIDMNCj4gPiB3aGljaCBhcmUgbm90IHRvIGJlIGFs bG93ZWQuDQo+ID4gDQo+ID4gQW5kIGFnYWluLCBwbGVhc2UgZXhwbGFpbiB3aHkgZG8geW91IHdh bnQgaXQuIFdoYXQgaXMgd3Jvbmcgd2l0aCB0aGUNCj4gPiBjYXNlIHdlIGFsbCBhZ3JlZSB3aXRo PyBpZTogIkNsaWVudCBjYW4gbm90IGNhbGwgTEFZT1VUUkVUVVJOIHVudGlsDQo+ID4gYWxsIGlu LWZsaWdodCBSUENzIHJldHVybiwgd2l0aCBvciB3aXRob3V0IGFuIGVycm9yIg0KPiA+IA0KPiA+ IFRoYW5rcw0KPiA+IEJvYXoNCj4gPiANCj4gPiA+IA0KPiA+ID4gU2VjdGlvbiAxOC40NC4zIC0g dGhlIGRlc2NyaXB0aW9uIHNlY3Rpb24gb2YgdGhlIExBWU9VVFJFVFVSTiBvcGVyYXRpb246DQo+ ID4gPiANCj4gPiA+ICAgIEFmdGVyIHRoaXMgY2FsbCwNCj4gPiA+ICAgIHRoZSBjbGllbnQgTVVT VCBOT1QgdXNlIHRoZSByZXR1cm5lZCBsYXlvdXQocykgYW5kIHRoZSBhc3NvY2lhdGVkDQo+ID4g PiAgICBzdG9yYWdlIHByb3RvY29sIHRvIGFjY2VzcyB0aGUgZmlsZSBkYXRhLg0KPiA+ID4gDQo+ ID4gPiBTZWN0aW9uIDEzLjYgT3BlcmF0aW9ucyBTZW50IHRvIE5GU3Y0LjEgRGF0YSBTZXJ2ZXJz DQo+ID4gPiANCj4gPiA+ICAgQXMgZGVzY3JpYmVkIGluIFNlY3Rpb24gMTIuNS4xLCBhIGNsaWVu dA0KPiA+ID4gICBNVVNUIE5PVCBzZW5kIGFuIEkvTyB0byBhIGRhdGEgc2VydmVyIGZvciB3aGlj aCBpdCBkb2VzIG5vdCBob2xkIGENCj4gPiA+ICAgdmFsaWQgbGF5b3V0OyB0aGUgZGF0YSBzZXJ2 ZXIgTVVTVCByZWplY3Qgc3VjaCBhbiBJL08uDQo+ID4gPiANCj4gPiA+IA0KPiA+ID4gLS0+QW5k eQ0KPiA+IA0KPiA+IA0KPiA+IF9fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19f X19fX19fX19fDQo+ID4gbmZzdjQgbWFpbGluZyBsaXN0DQo+ID4gbmZzdjRAaWV0Zi5vcmcNCj4g PiBodHRwczovL3d3dy5pZXRmLm9yZy9tYWlsbWFuL2xpc3RpbmZvL25mc3Y0DQo+ID4gDQo+IA0K DQotLSANClRyb25kIE15a2xlYnVzdA0KTGludXggTkZTIGNsaWVudCBtYWludGFpbmVyDQoNCk5l dEFwcA0KVHJvbmQuTXlrbGVidXN0QG5ldGFwcC5jb20NCnd3dy5uZXRhcHAuY29tDQoNCg== ^ permalink raw reply [flat|nested] 8+ messages in thread
end of thread, other threads:[~2012-06-12 16:34 UTC | newest]
Thread overview: 8+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
[not found] <CAHVgHyUpM0rQWqO5-id+FohPKm1Lk=kkekf7HqzpfKcfvxx23A@mail.gmail.com>
2012-06-11 18:40 ` RFC 5661 LAYOUTRETURN clarification Boaz Harrosh
2012-06-11 18:59 ` Myklebust, Trond
2012-06-11 22:18 ` Boaz Harrosh
2012-06-11 22:31 ` Myklebust, Trond
2012-06-12 16:34 ` Welch, Brent
2012-06-11 19:02 ` [nfsv4] " david.noveck
2012-06-11 19:43 ` Myklebust, Trond
2012-06-11 20:11 ` [nfsv4] RFC 5661 write to DS clarification. (was: [nfsv4] RFC 5661 LAYOUTRETURN clarification) Myklebust, Trond
This is an external index of several public inboxes, see mirroring instructions on how to clone and mirror all data and code used by this external index.