Re: knsfd - files don't sync to disk

["Jörgen Karlsson" <publius@chello.se>]

[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #1: Type: text/plain; charset=us-ascii; format=flowed, Size: 2930 bytes --]

We are using NFSv2.

We have found a temporary workaround by reducing the number of nfs threads
to 1. 

We have a 30 processor cluster that gave zero size files everytime we made
a backup. By reducing the number of threads from 8 to 1 we were able to make
25 backups without any problem. Switching back to 8 threads we immediately
got the corrupted files back.

Maybe some down() and up() are missing.......

If you think the the patch may fix it for v2 as well as v3 we can try the
patch next week.

Can running with only one thread cause any performance problems (with
sync the bottleneck probably is disk io....)?

/Jorgen Karlsson

Neil Brown wrote:

>Are you using NFSv3?
>
>Does this patch:
>http://www.cse.unsw.edu.au/~neilb/patches/linux-stable/2.4.19-pre5/patch-I-NfsdVfsTidyup
>
>make a difference?
>
>NeilBrown
>
>On Thursday May 16, jorgen.karlsson@chello.se wrote:
>
>>Hi,
>>
>>We have a serious problem with knfsd and kernel 2.4.17.
>>
>>When doing a database backup in our linux cluster we have noticed that
>>a few files are not written to disk properly. The files have zero
>>file size when checked with the 'ls' command
>>
>>Several thousands of files are written to the nfs server during a
>>short period of time.  Average file size is 300-400 bytes.
>>
>>We have noticed that usually 1-2 out of 3000 files will have their sizes
>>truncated to 0.
>>
>>File  system is exported with (rw,sync)
>>
>>The disk filesystem is ext2.
>>
>>E.g. this is what is happening:
>>          - client writes 272 bytes to the server
>>          - mm/file.c:generic_file_write() returns 272 bytes written
>>          - fs/nfsd/vfs.c:nfs_write() increments nfsdstats.io_write +=272
>>             and returns that the write was sucessful.
>>          - knfsd returns to client that 272 bytes was written.
>>          - when checking with 'ls' command on server the file size is zero.
>>
>>Apparently the nfs server lies to the client and the files are not
>>properly synced/written to disk.
>>
>>Setting no_wdelay makes no difference.
>>
>>When running with async set (or sync removed) the problem
>>disappears (no files have their sizes truncated to zero).
>>
>>The nfs server is a PIII-700/ 1GB RAM
>>
>>Any ideas what is going on ?
>>
>
>_______________________________________________________________
>
>Have big pipes? SourceForge.net is looking for download mirrors. We supply
>the hardware. You get the recognition. Email Us: bandwidth@sourceforge.net
>_______________________________________________
>NFS maillist  -  NFS@lists.sourceforge.net
>https://lists.sourceforge.net/lists/listinfo/nfs
>




_______________________________________________________________

Hundreds of nodes, one monster rendering program.
Now that’s a super model! Visit http://clustering.foundries.sf.net/
_______________________________________________
NFS maillist  -  NFS@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/nfs