RE: mtime not updated in client cache?

RE: mtime not updated in client cache?

[Lever, Charles]

> On Fri, 2005-12-09 at 09:24 -0500, Peter Staubach wrote:
>=20
> > >One alternative would be to flush dirty data to the server=20
> on every call
> > >to stat(), but that would slow it down considerably.

only in certain cases.  i can think of one:  'ls -l' in a directory full
of data files for an active Oracle instance.

> > Yes, to this last, but this is how most NFS implementations=20
> that I have seen
> > conform to the requirment that write(2) cause the mtime to=20
> be updated.  The
> > use of UNSTABLE NFSv3/NFSv4 writes can help as well as=20
> limiting the amount
> > of dirty data that the client allows to be cached.
>=20
> My main concern is that it will hit commands like 'ls -l' _hard_. The
> latter will, for instance, be forced to wait until all=20
> pending writes on
> all files in the directory have been flushed out. I'm not sure that
> strict POSIX compatibility is worth that kind of hit.
>=20
> Anyhow, the attached patch implements it. Lets try it out, and see how
> it does. Definitely _not_ a 2.6.15 candidate, though.

i like the idea.

do you need to prevent writes *during* the stat() call?  or do you just
want to make sure that any writes that happened *before* the stat() call
have the intended side-effects on the file size and mtime?  that might
be worth a comment or two.

are you flushing dirty mmap'd data too?  or can this be skipped until
msync() time?

and now that you have added a "flush but don't commit" flag to
nfs_sync_file, maybe we might consider using that (optionally) at file
close() time, like some other NFS client implementations do.  :^)


-------------------------------------------------------
This SF.net email is sponsored by: Splunk Inc. Do you grep through log files
for problems?  Stop!  Download the new AJAX search engine that makes
searching your log files as easy as surfing the  web.  DOWNLOAD SPLUNK!
http://ads.osdn.com/?ad_id=7637&alloc_id=16865&op=click
_______________________________________________
NFS maillist  -  NFS@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/nfs

RE: mtime not updated in client cache?

[Trond Myklebust]

On Fri, 2005-12-09 at 07:56 -0800, Lever, Charles wrote:
> do you need to prevent writes *during* the stat() call?  or do you just
> want to make sure that any writes that happened *before* the stat() call
> have the intended side-effects on the file size and mtime?  that might
> be worth a comment or two.

I cannot see that we would want strict serialisation of stat() and
write(). That would give rise to an even worse performance hit.
If someone calls write() while we're in stat() then the outcome should
be indeterminate, just as it is for local filesystems.

> are you flushing dirty mmap'd data too?  or can this be skipped until
> msync() time?

There appears to be no SuS requirement to do so: the description of
msync() says that you are only required to update m/ctime if writes are
actually initiated.

  http://www.opengroup.org/onlinepubs/009695399/toc.htm

> and now that you have added a "flush but don't commit" flag to
> nfs_sync_file, maybe we might consider using that (optionally) at file
> close() time, like some other NFS client implementations do.  :^)

Nope.

Cheers,
  Trond



-------------------------------------------------------
This SF.net email is sponsored by: Splunk Inc. Do you grep through log files
for problems?  Stop!  Download the new AJAX search engine that makes
searching your log files as easy as surfing the  web.  DOWNLOAD SPLUNK!
http://ads.osdn.com/?ad_id=7637&alloc_id=16865&op=click
_______________________________________________
NFS maillist  -  NFS@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/nfs

Re: mtime not updated in client cache?

[Peter Staubach]

Lever, Charles wrote:

>
>i like the idea.
>
>do you need to prevent writes *during* the stat() call?  or do you just
>want to make sure that any writes that happened *before* the stat() call
>have the intended side-effects on the file size and mtime?  that might
>be worth a comment or two.
>
>  
>

The semantic is generally that any write(2) calls made before the 
stat(2) call
should be reflected in the update mtime.

>are you flushing dirty mmap'd data too?  or can this be skipped until
>msync() time?
>
>and now that you have added a "flush but don't commit" flag to
>nfs_sync_file, maybe we might consider using that (optionally) at file
>close() time, like some other NFS client implementations do.  :^)
>  
>

This seems like a pretty dangerous thing to do.  I've thought about it
because it would seem to make close-to-open less expensive, but it 
introduces
the possibility of data loss without the application being aware.  If the
data is not committed at close(2), then if it can not be committed when it
needs to be committed, then there is no way to return an error to the
application.  A somewat contrived but still possible scenario is that the
server crashes, comes back up, and the file system fills up before the 
client
can retransmit all of the data.  In this case, the data will be lost, but
the application thought that it was all stored because no system calls
returned any errors.

    Thanx...

       ps


-------------------------------------------------------
This SF.net email is sponsored by: Splunk Inc. Do you grep through log files
for problems?  Stop!  Download the new AJAX search engine that makes
searching your log files as easy as surfing the  web.  DOWNLOAD SPLUNK!
http://ads.osdn.com/?ad_id=7637&alloc_id=16865&op=click
_______________________________________________
NFS maillist  -  NFS@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/nfs

RE: mtime not updated in client cache?

[Lever, Charles]

hi peter-

> >and now that you have added a "flush but don't commit" flag to
> >nfs_sync_file, maybe we might consider using that=20
> (optionally) at file
> >close() time, like some other NFS client implementations do.  :^)
>=20
> This seems like a pretty dangerous thing to do.  I've thought about it
> because it would seem to make close-to-open less expensive, but it=20
> introduces
> the possibility of data loss without the application being=20
> aware.  If the
> data is not committed at close(2), then if it can not be=20
> committed when it
> needs to be committed, then there is no way to return an error to the
> application.  A somewat contrived but still possible scenario=20
> is that the
> server crashes, comes back up, and the file system fills up=20
> before the client
> can retransmit all of the data.  In this case, the data will=20
> be lost, but
> the application thought that it was all stored because no system calls
> returned any errors.

risks understood.

however, how is this different from ext3?  a close() on an ext3 file
doesn't even attempt to schedule a flush of the data to disk.  if the
system crashes or the disk dies or the file system fills up, the data is
lost, just like in the NFS case.

in addition, mapped dirty data is never flushed on close() on NFS.  what
happens if a server failure pins dirty mapped pages in the client's
memory?

if an application really wants the guarantee that the data is
permanently stored or have the opportunity to recover if an error
occurs, it will use O_SYNC or msync() or fflush() before closing the
file.

to make a more philosophical argument, UNIX has always compromised data
integrity for performance in the common case.  for example, why aren't
there more file systems around that provide ACID semantics?  because in
99.9% of cases, ACID semantics are simply not required, and just slow
everything else down.

i think this is reasonable behavior to enable with either a mount option
or a sysctl.  we have "nocto," after all.

i'm not trying to start an argument.  just wondering aloud.


-------------------------------------------------------
This SF.net email is sponsored by: Splunk Inc. Do you grep through log files
for problems?  Stop!  Download the new AJAX search engine that makes
searching your log files as easy as surfing the  web.  DOWNLOAD SPLUNK!
http://ads.osdn.com/?ad_id=7637&alloc_id=16865&op=click
_______________________________________________
NFS maillist  -  NFS@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/nfs

Re: mtime not updated in client cache?

[Peter Staubach]

Lever, Charles wrote:

>
>risks understood.
>
>however, how is this different from ext3?  a close() on an ext3 file
>doesn't even attempt to schedule a flush of the data to disk.  if the
>system crashes or the disk dies or the file system fills up, the data is
>lost, just like in the NFS case.
>  
>

I think that the difference is that a system crash also terminates the
application and there is just more visibility into what happened.

>in addition, mapped dirty data is never flushed on close() on NFS.  what
>happens if a server failure pins dirty mapped pages in the client's
>memory?
>
>  
>

Actually, I think that part of close-to-open processing should be write
out dirty pages on munmap.

>if an application really wants the guarantee that the data is
>permanently stored or have the opportunity to recover if an error
>occurs, it will use O_SYNC or msync() or fflush() before closing the
>file.
>
>  
>

True.  An intermediate position has been to pay attention to system call
return values, including that from close(2), and pass them along to the
user to figure out what to do.

>to make a more philosophical argument, UNIX has always compromised data
>integrity for performance in the common case.  for example, why aren't
>there more file systems around that provide ACID semantics?  because in
>99.9% of cases, ACID semantics are simply not required, and just slow
>everything else down.
>
>i think this is reasonable behavior to enable with either a mount option
>or a sysctl.  we have "nocto," after all.
>
>  
>

I think that "nocto" was invented to be able to optimize situations where it
was known that a specific client was the _only_ client accessing those 
files.
Thus, close-to-open consistency was not required.

I've wondered about the real performance advantage though.  I've never
seen it measured in a believeable way.

>i'm not trying to start an argument.  just wondering aloud.
>  
>

I am happy to have the discussion, but it feels to me like a very dangerous
area to venture into.  Several times, I thought about trying to do this,
but each time stopped because I wasn't comfortable with the possibility
of silent data loss.

    Thanx...

       ps


-------------------------------------------------------
This SF.net email is sponsored by: Splunk Inc. Do you grep through log files
for problems?  Stop!  Download the new AJAX search engine that makes
searching your log files as easy as surfing the  web.  DOWNLOAD SPLUNK!
http://ads.osdn.com/?ad_id=7637&alloc_id=16865&op=click
_______________________________________________
NFS maillist  -  NFS@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/nfs

RE: mtime not updated in client cache?

[Roger Heflin]

 

> -----Original Message-----
> From: nfs-admin@lists.sourceforge.net 
> [mailto:nfs-admin@lists.sourceforge.net] On Behalf Of Peter Staubach
> Sent: Friday, December 09, 2005 1:20 PM
> To: Lever, Charles
> Cc: nfs@lists.sourceforge.net
> Subject: Re: [NFS] mtime not updated in client cache?
> 
> Lever, Charles wrote:
> 
> >
> >risks understood.
> >
> >however, how is this different from ext3?  a close() on an ext3 file 
> >doesn't even attempt to schedule a flush of the data to 
> disk.  if the 
> >system crashes or the disk dies or the file system fills up, 
> the data 
> >is lost, just like in the NFS case.
> >  
> >
> 
> I think that the difference is that a system crash also 
> terminates the application and there is just more visibility 
> into what happened.

On a local machine, the application can complete successfully, 
and the data won't be written out for up to 30 seconds, and this
can be higher on machines that have huge amounts of ram, and can
hold a lot more in its write cache, so data can be
lost even after the program completes properly if the machine crashes
before the writes are finished.

If you get a hard scsi/disk error when writing data from local cache, there
is not a process to return that error to the program that generated
the data assuming that the program is even still running.

                           Roger



-------------------------------------------------------
This SF.net email is sponsored by: Splunk Inc. Do you grep through log files
for problems?  Stop!  Download the new AJAX search engine that makes
searching your log files as easy as surfing the  web.  DOWNLOAD SPLUNK!
http://ads.osdn.com/?ad_id=7637&alloc_id=16865&op=click
_______________________________________________
NFS maillist  -  NFS@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/nfs

Re: mtime not updated in client cache?

[Peter Staubach]

Roger Heflin wrote:

> 
>
>  
>
>>-----Original Message-----
>>From: nfs-admin@lists.sourceforge.net 
>>[mailto:nfs-admin@lists.sourceforge.net] On Behalf Of Peter Staubach
>>Sent: Friday, December 09, 2005 1:20 PM
>>To: Lever, Charles
>>Cc: nfs@lists.sourceforge.net
>>Subject: Re: [NFS] mtime not updated in client cache?
>>
>>Lever, Charles wrote:
>>
>>    
>>
>>>risks understood.
>>>
>>>however, how is this different from ext3?  a close() on an ext3 file 
>>>doesn't even attempt to schedule a flush of the data to 
>>>      
>>>
>>disk.  if the 
>>    
>>
>>>system crashes or the disk dies or the file system fills up, 
>>>      
>>>
>>the data 
>>    
>>
>>>is lost, just like in the NFS case.
>>> 
>>>
>>>      
>>>
>>I think that the difference is that a system crash also 
>>terminates the application and there is just more visibility 
>>into what happened.
>>    
>>
>
>On a local machine, the application can complete successfully, 
>and the data won't be written out for up to 30 seconds, and this
>can be higher on machines that have huge amounts of ram, and can
>hold a lot more in its write cache, so data can be
>lost even after the program completes properly if the machine crashes
>before the writes are finished.
>
>If you get a hard scsi/disk error when writing data from local cache, there
>is not a process to return that error to the program that generated
>the data assuming that the program is even still running.
>

Right.  But the difference is that these things are visible on the system.
The user can know that there may be a problem.  If such a thing happens on
the NFS server, without the client's knowledge, then there is no way of
telling why the data is not there, just that it is not.

    Thanx...

       ps


-------------------------------------------------------
This SF.net email is sponsored by: Splunk Inc. Do you grep through log files
for problems?  Stop!  Download the new AJAX search engine that makes
searching your log files as easy as surfing the  web.  DOWNLOAD SPLUNK!
http://ads.osdn.com/?ad_id=7637&alloc_id=16865&op=click
_______________________________________________
NFS maillist  -  NFS@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/nfs

mtime not updated in client cache?

[Vince Busam]

The following code snippet run on a 2.6.13.4 (+CITI patch) client shows that mtime doesn't 
get updated unless fclose() is called before stat, or O_DIRECT is used.  In 2.4, the stat 
would return an updated mtime.  Is this a bug, or expected cache behavior?

Thanks,
Vince

#include <stdio.h>
#include <time.h>
#include <sys/types.h>
#include <unistd.h>
#include <sys/stat.h>

int main(int argc, char **argv) {
   time_t tm;
   struct stat statinfo;
   FILE *fp = fopen("testfile", "w");
   sleep(2);
   tm = time(NULL);
   fwrite("X", 1, 1, fp);
   fflush(fp);

   stat("testfile", &statinfo);

   if (statinfo.st_mtime < tm) {
     printf("Bug: write time = %u, mtime = %u\n", tm, statinfo.st_mtime);
   }

   fclose(fp);
   return 0;
}


-------------------------------------------------------
This SF.net email is sponsored by: Splunk Inc. Do you grep through log files
for problems?  Stop!  Download the new AJAX search engine that makes
searching your log files as easy as surfing the  web.  DOWNLOAD SPLUNK!
http://ads.osdn.com/?ad_id=7637&alloc_id=16865&op=click
_______________________________________________
NFS maillist  -  NFS@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/nfs

Re: mtime not updated in client cache?

[Trond Myklebust]

On Mon, 2005-11-28 at 16:09 -0800, Vince Busam wrote:
> The following code snippet run on a 2.6.13.4 (+CITI patch) client shows that mtime doesn't 
> get updated unless fclose() is called before stat, or O_DIRECT is used.  In 2.4, the stat 
> would return an updated mtime.  Is this a bug, or expected cache behavior?

That is a known issue that should be fixed in 2.6.15-rcX

Cheers,
  Trond

> Thanks,
> Vince
> 
> #include <stdio.h>
> #include <time.h>
> #include <sys/types.h>
> #include <unistd.h>
> #include <sys/stat.h>
> 
> int main(int argc, char **argv) {
>    time_t tm;
>    struct stat statinfo;
>    FILE *fp = fopen("testfile", "w");
>    sleep(2);
>    tm = time(NULL);
>    fwrite("X", 1, 1, fp);
>    fflush(fp);
> 
>    stat("testfile", &statinfo);
> 
>    if (statinfo.st_mtime < tm) {
>      printf("Bug: write time = %u, mtime = %u\n", tm, statinfo.st_mtime);
>    }
> 
>    fclose(fp);
>    return 0;
> }
> 
> 
> -------------------------------------------------------
> This SF.net email is sponsored by: Splunk Inc. Do you grep through log files
> for problems?  Stop!  Download the new AJAX search engine that makes
> searching your log files as easy as surfing the  web.  DOWNLOAD SPLUNK!
> http://ads.osdn.com/?ad_id=7637&alloc_id=16865&op=click
> _______________________________________________
> NFS maillist  -  NFS@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/nfs



-------------------------------------------------------
This SF.net email is sponsored by: Splunk Inc. Do you grep through log files
for problems?  Stop!  Download the new AJAX search engine that makes
searching your log files as easy as surfing the  web.  DOWNLOAD SPLUNK!
http://ads.osdn.com/?ad_id=7637&alloc_id=16865&op=click
_______________________________________________
NFS maillist  -  NFS@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/nfs

Re: mtime not updated in client cache?

[Vince Busam]

Trond Myklebust wrote:
> On Mon, 2005-11-28 at 16:09 -0800, Vince Busam wrote:
> 
>>The following code snippet run on a 2.6.13.4 (+CITI patch) client shows that mtime doesn't 
>>get updated unless fclose() is called before stat, or O_DIRECT is used.  In 2.4, the stat 
>>would return an updated mtime.  Is this a bug, or expected cache behavior?
> 
> 
> That is a known issue that should be fixed in 2.6.15-rcX
> 

Awesome.  Is there a patch I can help test out?

Vince


-------------------------------------------------------
This SF.net email is sponsored by: Splunk Inc. Do you grep through log files
for problems?  Stop!  Download the new AJAX search engine that makes
searching your log files as easy as surfing the  web.  DOWNLOAD SPLUNK!
http://ads.osdn.com/?ad_id=7637&alloc_id=16865&op=click
_______________________________________________
NFS maillist  -  NFS@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/nfs

Re: mtime not updated in client cache?

[Trond Myklebust]

On Wed, 2005-11-30 at 17:06 -0800, Vince Busam wrote:
> Trond Myklebust wrote:
> > On Mon, 2005-11-28 at 16:09 -0800, Vince Busam wrote:
> > 
> >>The following code snippet run on a 2.6.13.4 (+CITI patch) client shows that mtime doesn't 
> >>get updated unless fclose() is called before stat, or O_DIRECT is used.  In 2.4, the stat 
> >>would return an updated mtime.  Is this a bug, or expected cache behavior?
> > 
> > 
> > That is a known issue that should be fixed in 2.6.15-rcX
> > 
> 
> Awesome.  Is there a patch I can help test out?

The standard 2.6.15-rc3 patch from ftp.kernel.org should suffice. There
is both an NFS_ALL and a CITI_ALL that you can apply on top of it if you
like, but the straight 2.6.15-rc3 should already fix the mtime issue you
reported.

Cheers,
  Trond



-------------------------------------------------------
This SF.net email is sponsored by: Splunk Inc. Do you grep through log files
for problems?  Stop!  Download the new AJAX search engine that makes
searching your log files as easy as surfing the  web.  DOWNLOAD SPLUNK!
http://ads.osdn.com/?ad_id=7637&alloc_id=16865&op=click
_______________________________________________
NFS maillist  -  NFS@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/nfs

Re: mtime not updated in client cache?

[Vince Busam]

Trond Myklebust wrote:
> On Wed, 2005-11-30 at 17:06 -0800, Vince Busam wrote:
> 
>>Trond Myklebust wrote:
>>
>>>On Mon, 2005-11-28 at 16:09 -0800, Vince Busam wrote:
>>>
>>>
>>>>The following code snippet run on a 2.6.13.4 (+CITI patch) client shows that mtime doesn't 
>>>>get updated unless fclose() is called before stat, or O_DIRECT is used.  In 2.4, the stat 
>>>>would return an updated mtime.  Is this a bug, or expected cache behavior?
>>>
>>>
>>>That is a known issue that should be fixed in 2.6.15-rcX
>>>
>>
>>Awesome.  Is there a patch I can help test out?
> 
> 
> The standard 2.6.15-rc3 patch from ftp.kernel.org should suffice. There
> is both an NFS_ALL and a CITI_ALL that you can apply on top of it if you
> like, but the straight 2.6.15-rc3 should already fix the mtime issue you
> reported.
> 

I tried 2.6.15-rc5 (withoug NFS_ALL and CITI_ALL), and it exhibits the same issue as 
demonstrated by the above referenced code.

Vince


-------------------------------------------------------
This SF.net email is sponsored by: Splunk Inc. Do you grep through log files
for problems?  Stop!  Download the new AJAX search engine that makes
searching your log files as easy as surfing the  web.  DOWNLOAD SPLUNK!
http://ads.osdn.com/?ad_id=7637&alloc_id=16865&op=click
_______________________________________________
NFS maillist  -  NFS@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/nfs

Re: mtime not updated in client cache?

[Trond Myklebust]

On Wed, 2005-12-07 at 17:22 -0800, Vince Busam wrote:

> I tried 2.6.15-rc5 (withoug NFS_ALL and CITI_ALL), and it exhibits the same issue as 
> demonstrated by the above referenced code.

<looks at code>

Yep. Your code contains a wrong assumption: fflush() is not guaranteed
to immediately update mtime/ctime since it does not sync data to disk.
The same is true of the posix "write()" function.

If you add the line fsync(fileno(fp)) after the call to fflush(), then
the mtime will be correctly updated.

Cheers,
 Trond



-------------------------------------------------------
This SF.net email is sponsored by: Splunk Inc. Do you grep through log files
for problems?  Stop!  Download the new AJAX search engine that makes
searching your log files as easy as surfing the  web.  DOWNLOAD SPLUNK!
http://ads.osdn.com/?ad_id=7637&alloc_id=16865&op=click
_______________________________________________
NFS maillist  -  NFS@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/nfs

Re: mtime not updated in client cache?

[Peter Staubach]

Trond Myklebust wrote:

><looks at code>
>
>Yep. Your code contains a wrong assumption: fflush() is not guaranteed
>to immediately update mtime/ctime since it does not sync data to disk.
>The same is true of the posix "write()" function.
>
>If you add the line fsync(fileno(fp)) after the call to fflush(), then
>the mtime will be correctly updated.
>

Perhaps I am missing something, but fsync(2) should not cause the 
mtime/ctime
to change.  The write(2) call should.  Sequences such as 
stat(&a)/write()/stat(&b)
should result b.st_mtime != a.st_mtime.

    Thanx...

       ps


-------------------------------------------------------
This SF.net email is sponsored by: Splunk Inc. Do you grep through log files
for problems?  Stop!  Download the new AJAX search engine that makes
searching your log files as easy as surfing the  web.  DOWNLOAD SPLUNK!
http://ads.osdn.com/?ad_id=7637&alloc_id=16865&op=click
_______________________________________________
NFS maillist  -  NFS@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/nfs

Re: mtime not updated in client cache?

[Trond Myklebust]

On Fri, 2005-12-09 at 09:06 -0500, Peter Staubach wrote:
> Trond Myklebust wrote:
> 
> ><looks at code>
> >
> >Yep. Your code contains a wrong assumption: fflush() is not guaranteed
> >to immediately update mtime/ctime since it does not sync data to disk.
> >The same is true of the posix "write()" function.
> >
> >If you add the line fsync(fileno(fp)) after the call to fflush(), then
> >the mtime will be correctly updated.
> >
> 
> Perhaps I am missing something, but fsync(2) should not cause the 
> mtime/ctime
> to change.  The write(2) call should.  Sequences such as 
> stat(&a)/write()/stat(&b)
> should result b.st_mtime != a.st_mtime.

If you are caching the write on the client, then the server will not
update the mtime on the file.

OTOH, I do not like the idea of updating mtime on the client, since
there is no guarantee that the client and server are synchronised (you
can easily end up with the mtime jumping forwards and backwards).

One alternative would be to flush dirty data to the server on every call
to stat(), but that would slow it down considerably.

Cheers,
 Trond



-------------------------------------------------------
This SF.net email is sponsored by: Splunk Inc. Do you grep through log files
for problems?  Stop!  Download the new AJAX search engine that makes
searching your log files as easy as surfing the  web.  DOWNLOAD SPLUNK!
http://ads.osdn.com/?ad_id=7637&alloc_id=16865&op=click
_______________________________________________
NFS maillist  -  NFS@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/nfs

Re: mtime not updated in client cache?

[Peter Staubach]

Trond Myklebust wrote:

>On Fri, 2005-12-09 at 09:06 -0500, Peter Staubach wrote:
>  
>
>>Trond Myklebust wrote:
>>
>>    
>>
>>><looks at code>
>>>
>>>Yep. Your code contains a wrong assumption: fflush() is not guaranteed
>>>to immediately update mtime/ctime since it does not sync data to disk.
>>>The same is true of the posix "write()" function.
>>>
>>>If you add the line fsync(fileno(fp)) after the call to fflush(), then
>>>the mtime will be correctly updated.
>>>
>>>      
>>>
>>Perhaps I am missing something, but fsync(2) should not cause the 
>>mtime/ctime
>>to change.  The write(2) call should.  Sequences such as 
>>stat(&a)/write()/stat(&b)
>>should result b.st_mtime != a.st_mtime.
>>    
>>
>
>If you are caching the write on the client, then the server will not
>update the mtime on the file.
>
>OTOH, I do not like the idea of updating mtime on the client, since
>there is no guarantee that the client and server are synchronised (you
>can easily end up with the mtime jumping forwards and backwards).
>
>One alternative would be to flush dirty data to the server on every call
>to stat(), but that would slow it down considerably.
>

Yes, to this last, but this is how most NFS implementations that I have seen
conform to the requirment that write(2) cause the mtime to be updated.  The
use of UNSTABLE NFSv3/NFSv4 writes can help as well as limiting the amount
of dirty data that the client allows to be cached.

The client should _definitely_ not be setting the mtime itself.  The 
file times
are always managed by the server.

    Thanx...

       ps


-------------------------------------------------------
This SF.net email is sponsored by: Splunk Inc. Do you grep through log files
for problems?  Stop!  Download the new AJAX search engine that makes
searching your log files as easy as surfing the  web.  DOWNLOAD SPLUNK!
http://ads.osdn.com/?ad_id=7637&alloc_id=16865&op=click
_______________________________________________
NFS maillist  -  NFS@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/nfs

Re: mtime not updated in client cache?

[Trond Myklebust]

[-- Attachment #1: Type: text/plain, Size: 1024 bytes --]

On Fri, 2005-12-09 at 09:24 -0500, Peter Staubach wrote:

> >One alternative would be to flush dirty data to the server on every call
> >to stat(), but that would slow it down considerably.
> >
> 
> Yes, to this last, but this is how most NFS implementations that I have seen
> conform to the requirment that write(2) cause the mtime to be updated.  The
> use of UNSTABLE NFSv3/NFSv4 writes can help as well as limiting the amount
> of dirty data that the client allows to be cached.

My main concern is that it will hit commands like 'ls -l' _hard_. The
latter will, for instance, be forced to wait until all pending writes on
all files in the directory have been flushed out. I'm not sure that
strict POSIX compatibility is worth that kind of hit.

Anyhow, the attached patch implements it. Lets try it out, and see how
it does. Definitely _not_ a 2.6.15 candidate, though.

> The client should _definitely_ not be setting the mtime itself.  The 
> file times
> are always managed by the server.

Agreed.

Cheers,
  Trond

[-- Attachment #2: linux-2.6.15-35-write_updates_mtime.dif --]
[-- Type: text/plain, Size: 2471 bytes --]

NFS: Make stat() return updated mtimes after a write()

 The SuS states that a call to write() will cause mtime to be updated on
 the file. In order to satisfy that requirement, we need to flush out
 any cached writes in nfs_getattr().
 Speed things up slightly by not committing the writes.

 Signed-off-by: Trond Myklebust <Trond.Myklebust@netapp.com>
---

 fs/nfs/inode.c         |    2 ++
 fs/nfs/write.c         |   23 ++++++++++++-----------
 include/linux/nfs_fs.h |    1 +
 3 files changed, 15 insertions(+), 11 deletions(-)

diff --git a/fs/nfs/inode.c b/fs/nfs/inode.c
index acde2c5..2c7f8aa 100644
--- a/fs/nfs/inode.c
+++ b/fs/nfs/inode.c
@@ -952,6 +952,8 @@ int nfs_getattr(struct vfsmount *mnt, st
 	int need_atime = NFS_I(inode)->cache_validity & NFS_INO_INVALID_ATIME;
 	int err;
 
+	/* Flush out writes to the server in order to update c/mtime */
+	nfs_sync_inode(inode, 0, 0, FLUSH_WAIT|FLUSH_NOCOMMIT);
 	if (__IS_FLG(inode, MS_NOATIME))
 		need_atime = 0;
 	else if (__IS_FLG(inode, MS_NODIRATIME) && S_ISDIR(inode->i_mode))
diff --git a/fs/nfs/write.c b/fs/nfs/write.c
index 1ce0c20..9449b68 100644
--- a/fs/nfs/write.c
+++ b/fs/nfs/write.c
@@ -1377,22 +1377,23 @@ int nfs_commit_inode(struct inode *inode
 int nfs_sync_inode(struct inode *inode, unsigned long idx_start,
 		  unsigned int npages, int how)
 {
-	int	error,
-		wait;
+	int nocommit = how & FLUSH_NOCOMMIT;
+	int wait = how & FLUSH_WAIT;
+	int error;
 
-	wait = how & FLUSH_WAIT;
-	how &= ~FLUSH_WAIT;
+	how &= ~(FLUSH_WAIT|FLUSH_NOCOMMIT);
 
 	do {
-		error = 0;
-		if (wait)
+		if (wait) {
 			error = nfs_wait_on_requests(inode, idx_start, npages);
-		if (error == 0)
-			error = nfs_flush_inode(inode, idx_start, npages, how);
-#if defined(CONFIG_NFS_V3) || defined(CONFIG_NFS_V4)
-		if (error == 0)
+			if (error != 0)
+				continue;
+		}
+		error = nfs_flush_inode(inode, idx_start, npages, how);
+		if (error != 0)
+			continue;
+		if (!nocommit)
 			error = nfs_commit_inode(inode, how);
-#endif
 	} while (error > 0);
 	return error;
 }
diff --git a/include/linux/nfs_fs.h b/include/linux/nfs_fs.h
index d38010b..408d82d 100644
--- a/include/linux/nfs_fs.h
+++ b/include/linux/nfs_fs.h
@@ -62,6 +62,7 @@
 #define FLUSH_STABLE		4	/* commit to stable storage */
 #define FLUSH_LOWPRI		8	/* low priority background flush */
 #define FLUSH_HIGHPRI		16	/* high priority memory reclaim flush */
+#define FLUSH_NOCOMMIT		32	/* Don't send the NFSv3/v4 COMMIT */
 
 #ifdef __KERNEL__
 

Re: mtime not updated in client cache?

[Vince Busam]

> Anyhow, the attached patch implements it. Lets try it out, and see how
> it does. Definitely _not_ a 2.6.15 candidate, though.

Without getting into the argument of whether this is the right thing to do or not, I will 
verify that this patch resolved the "problem" and that the example code acts as it does on 
2.4 and Solaris.

Thanks,
Vince


-------------------------------------------------------
This SF.net email is sponsored by: Splunk Inc. Do you grep through log files
for problems?  Stop!  Download the new AJAX search engine that makes
searching your log files as easy as surfing the  web.  DOWNLOAD SPLUNK!
http://ads.osdn.com/?ad_id=7637&alloc_id=16865&op=click
_______________________________________________
NFS maillist  -  NFS@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/nfs