disabling caching

disabling caching

[Stefan Palme]

Hello,

I have a ext3-directory exported via NFS to a second linux (2.6.x) box.
This first (exporting) host is a linux-2.6.x host, too.

An application on that second box tries to open a file in that
directory. When this fails, it causes the exporting linux box to 
create the missing file (via an application-internal RPC call). 
After that RPC call the second linux box tries again to read 
this file. Although the file now exists, the open() call still 
returns "no such file or directory". This second try is much 
less than one second after the first try. 

After a few seconds, the open() call on the second box succeeds.
So I guess that caching is the reason for the second open()-call
to fail.

Is there a way to make the second call to open() succeed? 

Thanks i.a.
regards

-stefan-





-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642
_______________________________________________
NFS maillist  -  NFS@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/nfs

Re: disabling caching

[Frank van Maarseveen]

On Fri, Jul 14, 2006 at 10:12:32AM +0200, Stefan Palme wrote:
> 
> After a few seconds, the open() call on the second box succeeds.
> So I guess that caching is the reason for the second open()-call
> to fail.

You might want to try (1) and/or ((2) or (3)) on the NFS client:

(1)	On the NFS client, put this before the open():

		close(open("whatever", O_WRONLY|O_CREAT, 0666));
		unlink("whatever");

(2)	Try the "noac" nfs mount option, it'll cost performance though.

(3)	Another solution might be to use the patch at www.frankvm.com/nfs-noac
	to selectively disable attribute caching (see the README). It's a trick
	I use for some distributed work.


-- 
Frank


-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642
_______________________________________________
NFS maillist  -  NFS@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/nfs

Re: disabling caching

[Stefan Palme]

Thanks. Variant (1) is not possible for me, because I can't modify
the application in question. Variant (2) may be exactly what I have 
searched for (but did not read "man nfs" down to the end :-)

regards
-stefan-

> On Fri, Jul 14, 2006 at 10:12:32AM +0200, Stefan Palme wrote:
> > 
> > After a few seconds, the open() call on the second box succeeds.
> > So I guess that caching is the reason for the second open()-call
> > to fail.
> 
> You might want to try (1) and/or ((2) or (3)) on the NFS client:
> 
> (1)	On the NFS client, put this before the open():
> 
> 		close(open("whatever", O_WRONLY|O_CREAT, 0666));
> 		unlink("whatever");
> 
> (2)	Try the "noac" nfs mount option, it'll cost performance though.
> 
> (3)	Another solution might be to use the patch at www.frankvm.com/nfs-noac
> 	to selectively disable attribute caching (see the README). It's a trick
> 	I use for some distributed work.
> 
> 
-- 
-------------------------------------------------------------------
Dipl. Inf. (FH) Stefan Palme
 
email: kleiner@hora-obscura.de
www:   http://hbci4java.kapott.org
icq:   36376278
phon:  +49 341 3910484
fax:   +49 1212 517956219
mobil: +49 178 3227887
 
key fingerprint: 1BA7 D217 36A1 534C A5AD  F18A E2D1 488A E904 F9EC
-------------------------------------------------------------------



-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642
_______________________________________________
NFS maillist  -  NFS@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/nfs

Re: disabling caching

[Stefan Palme]

Hmpf. Using variant (2) did not fix the problem. Is it enough to

  remount -o remount /my/imported/dir,

or do I have to completely unmount and remount the directory? I ask
before doing this, because doing this would require to completely
shutdown our server process first, because this process has open 
files in the mounted dir...

I have taken a look at patch (3), but I guess, this would not make
much difference to solution (2), because this patch does the same
as the noac-option, but more "locally" (in time and place) - am I right?

Are there any other hints?

Best regards
-stefan-


> Thanks. Variant (1) is not possible for me, because I can't modify
> the application in question. Variant (2) may be exactly what I have 
> searched for (but did not read "man nfs" down to the end :-)
> 
> regards
> -stefan-
> 
> > On Fri, Jul 14, 2006 at 10:12:32AM +0200, Stefan Palme wrote:
> > > 
> > > After a few seconds, the open() call on the second box succeeds.
> > > So I guess that caching is the reason for the second open()-call
> > > to fail.
> > 
> > You might want to try (1) and/or ((2) or (3)) on the NFS client:
> > 
> > (1)	On the NFS client, put this before the open():
> > 
> > 		close(open("whatever", O_WRONLY|O_CREAT, 0666));
> > 		unlink("whatever");
> > 
> > (2)	Try the "noac" nfs mount option, it'll cost performance though.
> > 
> > (3)	Another solution might be to use the patch at www.frankvm.com/nfs-noac
> > 	to selectively disable attribute caching (see the README). It's a trick
> > 	I use for some distributed work.
> > 
> > 
-- 
-------------------------------------------------------------------
Dipl. Inf. (FH) Stefan Palme
 
email: kleiner@hora-obscura.de
www:   http://hbci4java.kapott.org
icq:   36376278
phon:  +49 341 3910484
fax:   +49 1212 517956219
mobil: +49 178 3227887
 
key fingerprint: 1BA7 D217 36A1 534C A5AD  F18A E2D1 488A E904 F9EC
-------------------------------------------------------------------



-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642
_______________________________________________
NFS maillist  -  NFS@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/nfs

Re: disabling caching

[Stefan Palme]

> Hmpf. Using variant (2) did not fix the problem. Is it enough to
> 
>   remount -o remount /my/imported/dir,
> 
> or do I have to completely unmount and remount the directory? 

... after adding the noac-option to /etc/fstab




-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642
_______________________________________________
NFS maillist  -  NFS@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/nfs

Re: disabling caching

[Stefan Palme]

> However, there is another alternative for your application.  NFS
> implements "close-to-open" caching.  This means an "open()" will force
> the client to revalidate its cache.  Since the *directory* is updated
> during a create, you need to open() the directory to force a cache
> refresh.  An opendir() on the directory containing the new file should
> enable your application to see the freshly created file immediately.

Because my application is written in Python, I have no access to the
opendir()-syscall directly. But the Python-Function "listdir()" from 
the os module uses this syscall (according to the Python source code).

But after using listdir(), the problem still persists:


  callToRemoteApplicationToCreateFile('file1')
  os.listdir('/my/imported/directory')
  f = file('/my/imported/directory/file1')

--> "No such file or directory"


An "ls /my/imported/directory" shows the file. Are you sure, that
opendir() causes the cache refresh? If yes, the problem must be
on the Python-side, so I have to look there...

thanks and regards
-stefan-





-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642
_______________________________________________
NFS maillist  -  NFS@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/nfs

Re: disabling caching

[Daniel Forrest]

Stefan,

> Because my application is written in Python, I have no access to the
> opendir()-syscall directly. But the Python-Function "listdir()" from 
> the os module uses this syscall (according to the Python source code).
> 
> But after using listdir(), the problem still persists:
> 
> 
>   callToRemoteApplicationToCreateFile('file1')
>   os.listdir('/my/imported/directory')
>   f = file('/my/imported/directory/file1')
> 
> --> "No such file or directory"
> 
> 
> An "ls /my/imported/directory" shows the file. Are you sure, that
> opendir() causes the cache refresh? If yes, the problem must be
> on the Python-side, so I have to look there...

What I have done in similar cases is the following:

1.) Remote site creates file1
2.) Local site creates foo
3.) Local site tries to link foo to file1 (this should fail)
4.) Local site removes foo
5.) Local site opens file1

In bash the last 4 steps are:

$ touch foo
$ ln foo file1
ln: `file1': File exists
$ rm foo
$ cat file1

Perhaps you could try something similar in Python?

-- 
Dan


-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642
_______________________________________________
NFS maillist  -  NFS@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/nfs

Re: disabling caching

[Stefan Palme]

> What I have done in similar cases is the following:
> 
> 1.) Remote site creates file1
> 2.) Local site creates foo
> 3.) Local site tries to link foo to file1 (this should fail)
> 4.) Local site removes foo
> 5.) Local site opens file1
> 
> In bash the last 4 steps are:
> 
> $ touch foo
> $ ln foo file1
> ln: `file1': File exists
> $ rm foo
> $ cat file1
> 
> Perhaps you could try something similar in Python?

Thank you for that tip, maybe I will use this as a last resort,
if I don't find the reason for this "misbehaviour", because it 
looks a little big "ugly" and creates a lot of overhead...

Regards
-stefan-




-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642
_______________________________________________
NFS maillist  -  NFS@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/nfs

Re: disabling caching

[Daniel Forrest]

Stefan,

> > What I have done in similar cases is the following:
> > 
> > 1.) Remote site creates file1
> > 2.) Local site creates foo
> > 3.) Local site tries to link foo to file1 (this should fail)
> > 4.) Local site removes foo
> > 5.) Local site opens file1
> > 
> > In bash the last 4 steps are:
> > 
> > $ touch foo
> > $ ln foo file1
> > ln: `file1': File exists
> > $ rm foo
> > $ cat file1
> > 
> > Perhaps you could try something similar in Python?
> 
> Thank you for that tip, maybe I will use this as a last resort,
> if I don't find the reason for this "misbehaviour", because it 
> looks a little big "ugly" and creates a lot of overhead...

Actually there isn't much overhead, since anything you do that ensures
coherency is going to require a round trip to the NFS server.  If there
is an already existing file that you know of in the directory you can
eliminate "foo" entirely and just use:

$ ln alreadyexists file1
ln: `file1': File exists


And get the same effect.

-- 
Dan


-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642
_______________________________________________
NFS maillist  -  NFS@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/nfs