How to change the FSINFO for nfsd?

How to change the FSINFO for nfsd?

[P.V.Anthony]

Hi,

Currently a gentoo linux server is used as a nfs server for some Mac OS
X clients. These Mac OS X clients are doing video editing.

Currently the video are captured directly to the NFS mounts.

Here is the problem. The video files are broken into 2GB files. This
because the video editing software(Final Cut Pro) breaks the files into
2GB sizes. It does not seem that the Mac OS X is not doing the breaking
up of files. It is the video editing software.

Some commercial company found an interesting behaviour. All the Mac OS X
NFS server FSINFO response in which the max file size is set to 'FFFF
FFFF FFFF FFFF' (all '1's).

The link about the above information can be found at this link below.
Read towards the bottom of the link.
http://forums.creativecow.net/thread/8/975362

I really would like to know where to change in the NFS source code so
that the FSINFO response from the server in which the max file size is
set to 'FFFF FFFF FFFF FFFF' (all '1's).

P.V.Anthony

Re: How to change the FSINFO for nfsd?

[J. Bruce Fields]

On Fri, May 09, 2008 at 01:05:17AM +0800, P.V.Anthony wrote:
> Hi,
> 
> Currently a gentoo linux server is used as a nfs server for some Mac OS
> X clients. These Mac OS X clients are doing video editing.
> 
> Currently the video are captured directly to the NFS mounts.
> 
> Here is the problem. The video files are broken into 2GB files. This
> because the video editing software(Final Cut Pro) breaks the files into
> 2GB sizes. It does not seem that the Mac OS X is not doing the breaking
> up of files. It is the video editing software.
> 
> Some commercial company found an interesting behaviour. All the Mac OS X
> NFS server FSINFO response in which the max file size is set to 'FFFF
> FFFF FFFF FFFF' (all '1's).
> 
> The link about the above information can be found at this link below.
> Read towards the bottom of the link.
> http://forums.creativecow.net/thread/8/975362
> 
> I really would like to know where to change in the NFS source code so
> that the FSINFO response from the server in which the max file size is
> set to 'FFFF FFFF FFFF FFFF' (all '1's).

I think you're probably barking up the wrong tree, but the max file size
as returned by fsinfo is set in fs/nfsd/nfs3proc.c:nfsd3_proc_fsinfo().

--b.

Re: How to change the FSINFO for nfsd?

[P.V.Anthony]

J. Bruce Fields wrote:

> I think you're probably barking up the wrong tree, but the max file size
> as returned by fsinfo is set in fs/nfsd/nfs3proc.c:nfsd3_proc_fsinfo().

Thank you very much for information on which file to change.

Need to ask one question. Please bear with me as I am not a programmer.

1. What program should be used to see the FSINFO of a NFS server?
This program is needed to see if the changes done to the source is
returning with the required FSINFO information.

Just for the record, the change done is as follows.

/usr/src/linux/fs/nfsd # diff nfs3proc.c.org nfs3proc.c
558c558
<       resp->f_maxfilesize = ~(u32) 0;
---
>       resp->f_maxfilesize = 1111111111111111;

P.V.Anthony

Re: How to change the FSINFO for nfsd?

[Bernd Eckenfels]

In article <4827227D.6040903@singnet.com.sg> you wrote:
> 1. What program should be used to see the FSINFO of a NFS server?

I think pathconf is supposed to do that, however I dont remember if it can
query filesize. I have seen only PATH_MAX und NAME_MAX on NFS>=3.

However you can strace that:

# strace getconf NAME_MAX /nfs/home
...
readlink("/usr/lib/getconf/default", 0xbf846ea6, 21) = -1 ENOENT (No such file or directory)
statfs("/nfs/home/", {f_type="NFS_SUPER_MAGIC", f_bsize=16384, f_blocks=3731367, f_bfree=454412, f_bavail=155902, f_files=7725054, f_ffree=7434759, f_fsid={0, 0}, f_namelen=255, f_frsize=16384}) = 0
fstat64(1, {st_mode=S_IFCHR|0620, st_rdev=makedev(136, 1), ...}) = 0
mmap2(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0xb7f33000
write(1, "255\n") = 4
255

In my example the FreeBSD Server is not returning max filesize..

Gruss
Bernd

Re: Solved - How to change the FSINFO for nfsd?

[P.V.Anthony]

Quoting "J. Bruce Fields":
> I think you'reprobably barking up the wrong tree, but the max file size
> as returned by fsinfo is set in fs/nfsd/nfs3proc.c:nfsd3_proc_fsinfo().

Just in case someone needs to know this information. Please note that  
I am not a
programmer. I am just hacking away.

Most of the credit for the hack goes to the person in the forum  
mentioned below
for finding the Mac NFS behavior.
http://forums.creativecow.net/thread/8/975362

The changes done seem to work well.

I am dissapointed that Apple made changes to NFS to a point that we  
have to use an Apple NFS. I thought that NFS is open. Anyway with  
these changes, a Linux server can be used.

Anyway this problem only happens from with in the video editing application,
Final Cut Pro, an Apple product. I hope someday a good video editing  
application
comes to Linux. I am willing to pay for Linux video editing  
application. It must
be good that is.

Anyway here is the patch done.

------------------- start ------------------------------

pv at videoserver2 ~ $ diff -ruN
/usr/src/linux-2.6.22-gentoo-r8/fs/nfsd/nfs3proc.c.org
/usr/src/linux-2.6.22-gentoo-r8/fs/nfsd/nfs3proc.c
--- /usr/src/linux-2.6.22-gentoo-r8/fs/nfsd/nfs3proc.c.org 2008-05-09
13:07:19.000000000 +0800
+++ /usr/src/linux-2.6.22-gentoo-r8/fs/nfsd/nfs3proc.c 2008-05-13
23:54:56.000000000 +0800
@@ -555,7 +555,7 @@
resp->f_wtpref = max_blocksize;
resp->f_wtmult = PAGE_SIZE;
resp->f_dtpref = PAGE_SIZE;
- resp->f_maxfilesize = ~(u32) 0;
+ resp->f_maxfilesize = 0xffffffffffffffff;
resp->f_properties = NFS3_FSF_DEFAULT;

nfserr = fh_verify(rqstp, &argp->fh, 0, MAY_NOP);
@@ -570,7 +570,7 @@
if (sb->s_magic == 0x4d44 /* MSDOS_SUPER_MAGIC */) {
resp->f_properties = NFS3_FSF_BILLYBOY;
}
- resp->f_maxfilesize = sb->s_maxbytes;
+ resp->f_maxfilesize = 0xffffffffffffffff;
}

fh_put(&argp->fh);

---------------- end -------------------------------------------------

P.V.Anthony

Re: Solved - How to change the FSINFO for nfsd?

[Jan Engelhardt]

On Saturday 2008-05-17 19:57, P.V.Anthony wrote:
>
> pv at videoserver2 ~ $ diff -ruN
> /usr/src/linux-2.6.22-gentoo-r8/fs/nfsd/nfs3proc.c.org
> /usr/src/linux-2.6.22-gentoo-r8/fs/nfsd/nfs3proc.c
> --- /usr/src/linux-2.6.22-gentoo-r8/fs/nfsd/nfs3proc.c.org 2008-05-09
> 13:07:19.000000000 +0800
> +++ /usr/src/linux-2.6.22-gentoo-r8/fs/nfsd/nfs3proc.c 2008-05-13
> 23:54:56.000000000 +0800
> @@ -555,7 +555,7 @@
> resp->f_wtpref = max_blocksize;
> resp->f_wtmult = PAGE_SIZE;
> resp->f_dtpref = PAGE_SIZE;
> - resp->f_maxfilesize = ~(u32) 0;
> + resp->f_maxfilesize = 0xffffffffffffffff;

Hm, you could have just changed u32 to u64 :-)

> resp->f_properties = NFS3_FSF_DEFAULT;
>
> nfserr = fh_verify(rqstp, &argp->fh, 0, MAY_NOP);
> @@ -570,7 +570,7 @@
> if (sb->s_magic == 0x4d44 /* MSDOS_SUPER_MAGIC */) {
> resp->f_properties = NFS3_FSF_BILLYBOY;
> }
> - resp->f_maxfilesize = sb->s_maxbytes;
> + resp->f_maxfilesize = 0xffffffffffffffff;
> }
>

Re: Solved - How to change the FSINFO for nfsd?

[Trond Myklebust]

On Sat, 2008-05-17 at 20:25 +0200, Jan Engelhardt wrote:
> On Saturday 2008-05-17 19:57, P.V.Anthony wrote:
> > resp->f_properties = NFS3_FSF_DEFAULT;
> >
> > nfserr = fh_verify(rqstp, &argp->fh, 0, MAY_NOP);
> > @@ -570,7 +570,7 @@
> > if (sb->s_magic == 0x4d44 /* MSDOS_SUPER_MAGIC */) {
> > resp->f_properties = NFS3_FSF_BILLYBOY;
> > }
> > - resp->f_maxfilesize = sb->s_maxbytes;
> > + resp->f_maxfilesize = 0xffffffffffffffff;

   This is plain wrong! Pretending that your filesystem supports 64-bit
files won't magically make it so.

Trond

Re: Solved - How to change the FSINFO for nfsd?

[J. Bruce Fields]

On Sat, May 17, 2008 at 02:36:42PM -0400, Trond Myklebust wrote:
> On Sat, 2008-05-17 at 20:25 +0200, Jan Engelhardt wrote:
> > On Saturday 2008-05-17 19:57, P.V.Anthony wrote:
> > > resp->f_properties = NFS3_FSF_DEFAULT;
> > >
> > > nfserr = fh_verify(rqstp, &argp->fh, 0, MAY_NOP);
> > > @@ -570,7 +570,7 @@
> > > if (sb->s_magic == 0x4d44 /* MSDOS_SUPER_MAGIC */) {
> > > resp->f_properties = NFS3_FSF_BILLYBOY;
> > > }
> > > - resp->f_maxfilesize = sb->s_maxbytes;
> > > + resp->f_maxfilesize = 0xffffffffffffffff;
> 
>    This is plain wrong! Pretending that your filesystem supports 64-bit
> files won't magically make it so.

Yes.  Could you tell us what filesystem you're using?

It's a bit bizarre that this would fix anything; I'd think the more
likely result would be errors on write and/or some kind of file
corruption.

--b.

Re: Solved - How to change the FSINFO for nfsd?

[P.V.Anthony]

J. Bruce Fields wrote:
> On Sat, May 17, 2008 at 02:36:42PM -0400, Trond Myklebust wrote:
>> On Sat, 2008-05-17 at 20:25 +0200, Jan Engelhardt wrote:
>>> On Saturday 2008-05-17 19:57, P.V.Anthony wrote:
>>>> resp->f_properties = NFS3_FSF_DEFAULT;
>>>>
>>>> nfserr = fh_verify(rqstp, &argp->fh, 0, MAY_NOP);
>>>> @@ -570,7 +570,7 @@
>>>> if (sb->s_magic == 0x4d44 /* MSDOS_SUPER_MAGIC */) {
>>>> resp->f_properties = NFS3_FSF_BILLYBOY;
>>>> }
>>>> - resp->f_maxfilesize = sb->s_maxbytes;
>>>> + resp->f_maxfilesize = 0xffffffffffffffff;
>>    This is plain wrong! Pretending that your filesystem supports 64-bit
>> files won't magically make it so.
> 
> Yes.  Could you tell us what filesystem you're using?
> 
> It's a bit bizarre that this would fix anything; I'd think the more
> likely result would be errors on write and/or some kind of file
> corruption.

The filesystem used is ext3. So far no file corruption. The os is 64bit
gentoo.

Will be trying xfs without the patch and will report back. I think XFS
is 64bit and will see if Mac OS X with Final Cut Pro works with it.

Please not that the problem only happens when using the video editing
application Final Cut Pro. Without the patch, when capturing video to
the mounted NFS, the captured video is broken into 2Gb size files. This
is even when the server and client is set to version 3.

The old version of FCP does not have this behaviour.

I am guessing that Apple is some way forcing people to buy their Xserve
servers. To use their servers for NFS.

If a file is dragged on to the NFS mount then it will work fine. It only
does not work through the FCP application. Like when capturing or
exporting to a NFS mount.

P.V.Anthony

Re: Solved - How to change the FSINFO for nfsd?

[P.V.Anthony]

Using xfs filesystem on a gentoo 64bit linux with 2.6.22 kernel,
when mounting the FSINFO return got was 7f ff ff ff ff ff ff ff.

Assuming that xfs is 64bit,
the FSINFO returned is still not ff ff ff ff ff ff ff ff.

Used wireshark to see the FSINFO return.

P.V.Anthony

Re: Solved - How to change the FSINFO for nfsd?

[Trond Myklebust]

On Tue, 2008-05-20 at 16:15 +0800, P.V.Anthony wrote:
> Using xfs filesystem on a gentoo 64bit linux with 2.6.22 kernel,
> when mounting the FSINFO return got was 7f ff ff ff ff ff ff ff.
> 
> Assuming that xfs is 64bit,
> the FSINFO returned is still not ff ff ff ff ff ff ff ff.
> 
> Used wireshark to see the FSINFO return.
> 
> P.V.Anthony

XFS is 64-bit, but file sizes are of type off_t or loff_t, and are
therefore _signed_.
0x7fffffffffffffff is therefore indeed the maximum allowed file size for
a 64-bit filesystem.

Trond

How to change the FSINFO for nfsd?

[P.V.Anthony]

Hi,

Currently a gentoo linux server is used as a nfs server for some Mac OS
X clients. These Mac OS X clients are doing video editing.

Currently the video are captured directly to the NFS mounts.

Here is the problem. The video files are broken into 2GB files. This
because the video editing software(Final Cut Pro) breaks the files into
2GB sizes. It does not seem that the Mac OS X is not doing the breaking
up of files. It is the video editing software.

Some commercial company found an interesting behaviour. All the Mac OS X
NFS server FSINFO response in which the max file size is set to 'FFFF
FFFF FFFF FFFF' (all '1's).

The link about the above information can be found at this link below.
Read towards the bottom of the link.
http://forums.creativecow.net/thread/8/975362

I really would like to know where to change in the NFS source code so
that the FSINFO response from the server in which the max file size is
set to 'FFFF FFFF FFFF FFFF' (all '1's).

P.V.Anthony

Re: How to change the FSINFO for nfsd?

[Jeff Moyer]

"P.V.Anthony" <pvantony@singnet.com.sg> writes:

> Hi,
>
> Currently a gentoo linux server is used as a nfs server for some Mac OS
> X clients. These Mac OS X clients are doing video editing.
>
> Currently the video are captured directly to the NFS mounts.
>
> Here is the problem. The video files are broken into 2GB files. This
> because the video editing software(Final Cut Pro) breaks the files into
> 2GB sizes. It does not seem that the Mac OS X is not doing the breaking
> up of files. It is the video editing software.
>
> Some commercial company found an interesting behaviour. All the Mac OS X
> NFS server FSINFO response in which the max file size is set to 'FFFF
> FFFF FFFF FFFF' (all '1's).
>
> The link about the above information can be found at this link below.
> Read towards the bottom of the link.
> http://forums.creativecow.net/thread/8/975362
>
> I really would like to know where to change in the NFS source code so
> that the FSINFO response from the server in which the max file size is
> set to 'FFFF FFFF FFFF FFFF' (all '1's).

You're really stabbing in the dark, here.  I suggest you email the
software manufacturer of Final Cut Pro (Apple?) with a support request.

Thanks,

Jeff

Re: How to change the FSINFO for nfsd?

[P.V.Anthony]

Jeff Moyer wrote:

> You're really stabbing in the dark, here.  I suggest you email the
> software manufacturer of Final Cut Pro (Apple?) with a support request.

You are right but Apple is not going to help because I am using Linux.
They would ask me to buy a Apple Xserve.

I know it will work because the commercial company made the changes and
it works. I also understand that they will not share the code change
because it is a commercial thing.

Here is their email. At least they are willing to share this.
-------------- start ----------------------

Yes we arrive to an interesting conclusion. Over NFS the Max OSX10.5
client uses the NFS FSINFO request to discover NFS server capabilities
such as the max file size. Specifically Mac NFS servers return a value
of all '1's for the max file size field in the FSINFO response. This
value is not realistic but tell the Mac client that this is a Mac NFS
server that can handle files larger then 2GB.

We are a storage vendor so we changed the code for our NFS server to
return the same FSINFO information and that resolved the file chopping
problem. It also fixed another bug where copies over the finder of files
over 4GB failed. Now they don't.

-------------- end   ----------------------

If possible, please share how to change the code to make the FSINFO
present all '1's for the max file size field.

P.V.Anthony

Re: How to change the FSINFO for nfsd?

[Ian Kent]

On Thu, 2008-05-08 at 08:49 +0800, P.V.Anthony wrote:
> Jeff Moyer wrote:
> 
> > You're really stabbing in the dark, here.  I suggest you email the
> > software manufacturer of Final Cut Pro (Apple?) with a support request.
> 
> You are right but Apple is not going to help because I am using Linux.
> They would ask me to buy a Apple Xserve.
> 
> I know it will work because the commercial company made the changes and
> it works. I also understand that they will not share the code change
> because it is a commercial thing.
> 
> Here is their email. At least they are willing to share this.
> -------------- start ----------------------
> 
> Yes we arrive to an interesting conclusion. Over NFS the Max OSX10.5
> client uses the NFS FSINFO request to discover NFS server capabilities
> such as the max file size. Specifically Mac NFS servers return a value
> of all '1's for the max file size field in the FSINFO response. This
> value is not realistic but tell the Mac client that this is a Mac NFS
> server that can handle files larger then 2GB.
> 
> We are a storage vendor so we changed the code for our NFS server to
> return the same FSINFO information and that resolved the file chopping
> problem. It also fixed another bug where copies over the finder of files
> over 4GB failed. Now they don't.
> 
> -------------- end   ----------------------
> 
> If possible, please share how to change the code to make the FSINFO
> present all '1's for the max file size field.

This isn't the place to ask how to make changes to the kernel NFS
client. You should probably ask this on the linux-nfs list.

Ian

Re: How to change the FSINFO for nfsd?

[P.V.Anthony]

Ian Kent wrote:

> This isn't the place to ask how to make changes to the kernel NFS
> client. You should probably ask this on the linux-nfs list.

Thank you for the information. I will try there.

P.V.Anthony