NFSv3 caching vs NFSv4

NFSv3 caching vs NFSv4

[Diego Moreno]

Hello everyone,

Performing some attribute caching tests I think I found some 
inconsistencies between nfsv3 and nfsv4. Running a kernel based on 
rhel6.3 (2.6.32-279.el6.x86_64) I found nfsv4 doesn't manage a file 
modified on the server side (even with the noac mount option) while 
nfsv3 does.

My test will be modifying a file every second on the server mount point:

[root@nfs ~]#  FILE=/tmp/nfs_server/myfile ;  while true ; do sleep 1 ; 
sed -i 's/text1/text2/g' $FILE ; cat $FILE ; sleep 1; sed -i 
's/text2/text1/g' $FILE ; cat $FILE ; done
text2
text1
text2
text1
text2
(and so on...)

Meanwhile, the client (which BTW is the same machine) will read the file 
every second :

With NFSv4:

[root@nfs ~]# mount -o noac,vers=4 nfs:/tmp/nfs_server/ /tmp/nfs_client/
[root@nfs ~]# for i in 1 2 3 4 5 ; do cat /tmp/nfs_client/myfile; sleep 
1 ; done
text2
text1
text1
text1
text1

With NFSv3:
[root@nfs ~]# mount -o noac,vers=3 nfs:/tmp/nfs_server/ /tmp/nfs_client/
[root@nfs ~]# for i in 1 2 3 4 5 ; do cat /tmp/nfs_client/myfile; sleep 
1 ; done
text1
text2
text1
text2
text1

I can see in the tcpdump traces how the new change attribute and the 
FATTR4_TIME_MODIFY attributes are well received but it's as if the nfs 
client were ignoring these changes. I've been searching in the NFS list 
but I didn't find anything similar. Is this a bug or just a normal 
behavior? Sorry if it has been already pointed out.

Regards,

Diego Moreno

Re: NFSv3 caching vs NFSv4

[Myklebust, Trond]

T24gV2VkLCAyMDEyLTA5LTA1IGF0IDE0OjQwICswMjAwLCBEaWVnbyBNb3Jlbm8gd3JvdGU6DQo+
IEhlbGxvIGV2ZXJ5b25lLA0KPiANCj4gUGVyZm9ybWluZyBzb21lIGF0dHJpYnV0ZSBjYWNoaW5n
IHRlc3RzIEkgdGhpbmsgSSBmb3VuZCBzb21lIA0KPiBpbmNvbnNpc3RlbmNpZXMgYmV0d2VlbiBu
ZnN2MyBhbmQgbmZzdjQuIFJ1bm5pbmcgYSBrZXJuZWwgYmFzZWQgb24gDQo+IHJoZWw2LjMgKDIu
Ni4zMi0yNzkuZWw2Lng4Nl82NCkgSSBmb3VuZCBuZnN2NCBkb2Vzbid0IG1hbmFnZSBhIGZpbGUg
DQo+IG1vZGlmaWVkIG9uIHRoZSBzZXJ2ZXIgc2lkZSAoZXZlbiB3aXRoIHRoZSBub2FjIG1vdW50
IG9wdGlvbikgd2hpbGUgDQo+IG5mc3YzIGRvZXMuDQo+IA0KPiBNeSB0ZXN0IHdpbGwgYmUgbW9k
aWZ5aW5nIGEgZmlsZSBldmVyeSBzZWNvbmQgb24gdGhlIHNlcnZlciBtb3VudCBwb2ludDoNCj4g
DQo+IFtyb290QG5mcyB+XSMgIEZJTEU9L3RtcC9uZnNfc2VydmVyL215ZmlsZSA7ICB3aGlsZSB0
cnVlIDsgZG8gc2xlZXAgMSA7IA0KPiBzZWQgLWkgJ3MvdGV4dDEvdGV4dDIvZycgJEZJTEUgOyBj
YXQgJEZJTEUgOyBzbGVlcCAxOyBzZWQgLWkgDQo+ICdzL3RleHQyL3RleHQxL2cnICRGSUxFIDsg
Y2F0ICRGSUxFIDsgZG9uZQ0KPiB0ZXh0Mg0KPiB0ZXh0MQ0KPiB0ZXh0Mg0KPiB0ZXh0MQ0KPiB0
ZXh0Mg0KPiAoYW5kIHNvIG9uLi4uKQ0KPiANCj4gTWVhbndoaWxlLCB0aGUgY2xpZW50ICh3aGlj
aCBCVFcgaXMgdGhlIHNhbWUgbWFjaGluZSkgd2lsbCByZWFkIHRoZSBmaWxlIA0KPiBldmVyeSBz
ZWNvbmQgOg0KPiANCj4gV2l0aCBORlN2NDoNCj4gDQo+IFtyb290QG5mcyB+XSMgbW91bnQgLW8g
bm9hYyx2ZXJzPTQgbmZzOi90bXAvbmZzX3NlcnZlci8gL3RtcC9uZnNfY2xpZW50Lw0KPiBbcm9v
dEBuZnMgfl0jIGZvciBpIGluIDEgMiAzIDQgNSA7IGRvIGNhdCAvdG1wL25mc19jbGllbnQvbXlm
aWxlOyBzbGVlcCANCj4gMSA7IGRvbmUNCj4gdGV4dDINCj4gdGV4dDENCj4gdGV4dDENCj4gdGV4
dDENCj4gdGV4dDENCj4gDQo+IFdpdGggTkZTdjM6DQo+IFtyb290QG5mcyB+XSMgbW91bnQgLW8g
bm9hYyx2ZXJzPTMgbmZzOi90bXAvbmZzX3NlcnZlci8gL3RtcC9uZnNfY2xpZW50Lw0KPiBbcm9v
dEBuZnMgfl0jIGZvciBpIGluIDEgMiAzIDQgNSA7IGRvIGNhdCAvdG1wL25mc19jbGllbnQvbXlm
aWxlOyBzbGVlcCANCj4gMSA7IGRvbmUNCj4gdGV4dDENCj4gdGV4dDINCj4gdGV4dDENCj4gdGV4
dDINCj4gdGV4dDENCj4gDQo+IEkgY2FuIHNlZSBpbiB0aGUgdGNwZHVtcCB0cmFjZXMgaG93IHRo
ZSBuZXcgY2hhbmdlIGF0dHJpYnV0ZSBhbmQgdGhlIA0KPiBGQVRUUjRfVElNRV9NT0RJRlkgYXR0
cmlidXRlcyBhcmUgd2VsbCByZWNlaXZlZCBidXQgaXQncyBhcyBpZiB0aGUgbmZzIA0KPiBjbGll
bnQgd2VyZSBpZ25vcmluZyB0aGVzZSBjaGFuZ2VzLiBJJ3ZlIGJlZW4gc2VhcmNoaW5nIGluIHRo
ZSBORlMgbGlzdCANCj4gYnV0IEkgZGlkbid0IGZpbmQgYW55dGhpbmcgc2ltaWxhci4gSXMgdGhp
cyBhIGJ1ZyBvciBqdXN0IGEgbm9ybWFsIA0KPiBiZWhhdmlvcj8gU29ycnkgaWYgaXQgaGFzIGJl
ZW4gYWxyZWFkeSBwb2ludGVkIG91dC4NCg0KXUlmIHlvdSBjYW4gcmVwcm9kdWNlIHRoZSBzYW1l
IGlzc3VlcyBvbiB0aGUgdXBzdHJlYW0ga2VybmVsLCB0aGVuIGl0IGlzDQpvbiB0b3BpYyBmb3Ig
dGhpcyBsaXN0LCBvdGhlcndpc2UgaXQgaXMgYSBxdWVzdGlvbiBmb3IgUmVkIEhhdC4NCg0KQ2hl
ZXJzDQogIFRyb25kDQotLSANClRyb25kIE15a2xlYnVzdA0KTGludXggTkZTIGNsaWVudCBtYWlu
dGFpbmVyDQoNCk5ldEFwcA0KVHJvbmQuTXlrbGVidXN0QG5ldGFwcC5jb20NCnd3dy5uZXRhcHAu
Y29tDQoNCg==

Re: NFSv3 caching vs NFSv4

[J. Bruce Fields]

On Wed, Sep 05, 2012 at 02:49:18PM +0000, Myklebust, Trond wrote:
> On Wed, 2012-09-05 at 14:40 +0200, Diego Moreno wrote:
> > Hello everyone,
> > 
> > Performing some attribute caching tests I think I found some 
> > inconsistencies between nfsv3 and nfsv4. Running a kernel based on 
> > rhel6.3 (2.6.32-279.el6.x86_64) I found nfsv4 doesn't manage a file 
> > modified on the server side (even with the noac mount option) while 
> > nfsv3 does.
> > 
> > My test will be modifying a file every second on the server mount point:
> > 
> > [root@nfs ~]#  FILE=/tmp/nfs_server/myfile ;  while true ; do sleep 1 ; 
> > sed -i 's/text1/text2/g' $FILE ; cat $FILE ; sleep 1; sed -i 
> > 's/text2/text1/g' $FILE ; cat $FILE ; done
> > text2
> > text1
> > text2
> > text1
> > text2
> > (and so on...)
> > 
> > Meanwhile, the client (which BTW is the same machine) will read the file 
> > every second :
> > 
> > With NFSv4:
> > 
> > [root@nfs ~]# mount -o noac,vers=4 nfs:/tmp/nfs_server/ /tmp/nfs_client/
> > [root@nfs ~]# for i in 1 2 3 4 5 ; do cat /tmp/nfs_client/myfile; sleep 
> > 1 ; done
> > text2
> > text1
> > text1
> > text1
> > text1
> > 
> > With NFSv3:
> > [root@nfs ~]# mount -o noac,vers=3 nfs:/tmp/nfs_server/ /tmp/nfs_client/
> > [root@nfs ~]# for i in 1 2 3 4 5 ; do cat /tmp/nfs_client/myfile; sleep 
> > 1 ; done
> > text1
> > text2
> > text1
> > text2
> > text1
> > 
> > I can see in the tcpdump traces how the new change attribute and the 
> > FATTR4_TIME_MODIFY attributes are well received but it's as if the nfs 
> > client were ignoring these changes. I've been searching in the NFS list 
> > but I didn't find anything similar. Is this a bug or just a normal 
> > behavior? Sorry if it has been already pointed out.
> 
> ]If you can reproduce the same issues on the upstream kernel, then it is
> on topic for this list, otherwise it is a question for Red Hat.

I believe sed -i is actually doing a rename not a file modification.  So
the problem is likely another symptom of the bug that delegations aren't
revoked on rename/link/unlink.  Hoping to post the latest draft of
patches to fix that later today.

--b.

Re: NFSv3 caching vs NFSv4

[Diego Moreno]

Ok, so this is a delegation bug. I think you're rigth Bruce, I cannot 
see the bug if I just modify the file with vi.

I'll try your delegation patch ASAP.

Thanks,

Diego

On Wed 05 Sep 2012 07:21:58 PM CEST, J. Bruce Fields wrote:
> On Wed, Sep 05, 2012 at 02:49:18PM +0000, Myklebust, Trond wrote:
>> On Wed, 2012-09-05 at 14:40 +0200, Diego Moreno wrote:
>>> Hello everyone,
>>>
>>> Performing some attribute caching tests I think I found some
>>> inconsistencies between nfsv3 and nfsv4. Running a kernel based on
>>> rhel6.3 (2.6.32-279.el6.x86_64) I found nfsv4 doesn't manage a file
>>> modified on the server side (even with the noac mount option) while
>>> nfsv3 does.
>>>
>>> My test will be modifying a file every second on the server mount point:
>>>
>>> [root@nfs ~]#  FILE=/tmp/nfs_server/myfile ;  while true ; do sleep 1 ;
>>> sed -i 's/text1/text2/g' $FILE ; cat $FILE ; sleep 1; sed -i
>>> 's/text2/text1/g' $FILE ; cat $FILE ; done
>>> text2
>>> text1
>>> text2
>>> text1
>>> text2
>>> (and so on...)
>>>
>>> Meanwhile, the client (which BTW is the same machine) will read the file
>>> every second :
>>>
>>> With NFSv4:
>>>
>>> [root@nfs ~]# mount -o noac,vers=4 nfs:/tmp/nfs_server/ /tmp/nfs_client/
>>> [root@nfs ~]# for i in 1 2 3 4 5 ; do cat /tmp/nfs_client/myfile; sleep
>>> 1 ; done
>>> text2
>>> text1
>>> text1
>>> text1
>>> text1
>>>
>>> With NFSv3:
>>> [root@nfs ~]# mount -o noac,vers=3 nfs:/tmp/nfs_server/ /tmp/nfs_client/
>>> [root@nfs ~]# for i in 1 2 3 4 5 ; do cat /tmp/nfs_client/myfile; sleep
>>> 1 ; done
>>> text1
>>> text2
>>> text1
>>> text2
>>> text1
>>>
>>> I can see in the tcpdump traces how the new change attribute and the
>>> FATTR4_TIME_MODIFY attributes are well received but it's as if the nfs
>>> client were ignoring these changes. I've been searching in the NFS list
>>> but I didn't find anything similar. Is this a bug or just a normal
>>> behavior? Sorry if it has been already pointed out.
>>
>> ]If you can reproduce the same issues on the upstream kernel, then it is
>> on topic for this list, otherwise it is a question for Red Hat.
>
> I believe sed -i is actually doing a rename not a file modification.  So
> the problem is likely another symptom of the bug that delegations aren't
> revoked on rename/link/unlink.  Hoping to post the latest draft of
> patches to fix that later today.
>
> --b.
> --
> To unsubscribe from this list: send the line "unsubscribe linux-nfs" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html