stale NFS file handle problem on XFS inode64

stale NFS file handle problem on XFS inode64

[hank peng]

Hi:
I have a 4TB XFS filesystem mounted with "inode64" option, and I
exported it using NFS v3, on NFS client, I entered a subdirectory and
got "stale NFS file handle" error.
ls -il showed that its inode number exceed 32 bit limit.
NFS server: kernel version is 2.6.35.6
NFS client: kernel version is 2.6.18

Any ideas on how to deal with this problem?

-- 
The simplest is not all best but the best is surely the simplest!

_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs

Re: stale NFS file handle problem on XFS inode64

[Emmanuel Florac]

Le Mon, 21 Nov 2011 21:00:57 +0800
hank peng <pengxihan@gmail.com> écrivait:

> NFS server: kernel version is 2.6.35.6
> NFS client: kernel version is 2.6.18
> 
> Any ideas on how to deal with this problem?

Are the client and the server running a 64 bits kernel?

-- 
------------------------------------------------------------------------
Emmanuel Florac     |   Direction technique
                    |   Intellique
                    |	<eflorac@intellique.com>
                    |   +33 1 78 94 84 02
------------------------------------------------------------------------

_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs

Re: stale NFS file handle problem on XFS inode64

[hank peng]

no, client and server are both 32 bits kernel.

2011/11/21 Emmanuel Florac <eflorac@intellique.com>:
> Le Mon, 21 Nov 2011 21:00:57 +0800
> hank peng <pengxihan@gmail.com> écrivait:
>
>> NFS server: kernel version is 2.6.35.6
>> NFS client: kernel version is 2.6.18
>>
>> Any ideas on how to deal with this problem?
>
> Are the client and the server running a 64 bits kernel?
>
> --
> ------------------------------------------------------------------------
> Emmanuel Florac     |   Direction technique
>                    |   Intellique
>                    |   <eflorac@intellique.com>
>                    |   +33 1 78 94 84 02
> ------------------------------------------------------------------------
>



-- 
The simplest is not all best but the best is surely the simplest!

_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs

Re: stale NFS file handle problem on XFS inode64

[Emmanuel Florac]

Le Mon, 21 Nov 2011 23:02:32 +0800
hank peng <pengxihan@gmail.com> écrivait:

> no, client and server are both 32 bits kernel.
> 

I'm afraid the 32 bits nfs client can't manage 64 bits inodes at all.
Maybe with a much more recent kernel, something like 2.6.29 or better? 

See for instance
http://oss.sgi.com/archives/xfs/2009-11/msg00174.html

Your best choice is to revert to not using the "inode64" option.
Unfortunately the easiest way is to backup and restore the whole
filesystem.

Alternatively, switch to samba...

-- 
------------------------------------------------------------------------
Emmanuel Florac     |   Direction technique
                    |   Intellique
                    |	<eflorac@intellique.com>
                    |   +33 1 78 94 84 02
------------------------------------------------------------------------

_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs

Re: stale NFS file handle problem on XFS inode64

[hank peng]

2011/11/21 Emmanuel Florac <eflorac@intellique.com>:
> Le Mon, 21 Nov 2011 23:02:32 +0800
> hank peng <pengxihan@gmail.com> écrivait:
>
>> no, client and server are both 32 bits kernel.
>>
>
> I'm afraid the 32 bits nfs client can't manage 64 bits inodes at all.
> Maybe with a much more recent kernel, something like 2.6.29 or better?
>

I noticed that there is a kernel parameter :
nfs.enable_ino64=
			[NFS] enable 64-bit inode numbers.
			If zero, the NFS client will fake up a 32-bit inode
			number for the readdir() and stat() syscalls instead
			of returning the full 64-bit number.
			The default is to return 64-bit inode numbers.

can I set it to zero, so NFS client can get converted 32bit inode ?

> See for instance
> http://oss.sgi.com/archives/xfs/2009-11/msg00174.html
>
> Your best choice is to revert to not using the "inode64" option.
> Unfortunately the easiest way is to backup and restore the whole
> filesystem.
>
> Alternatively, switch to samba...
>
> --
> ------------------------------------------------------------------------
> Emmanuel Florac     |   Direction technique
>                    |   Intellique
>                    |   <eflorac@intellique.com>
>                    |   +33 1 78 94 84 02
> ------------------------------------------------------------------------
>



-- 
The simplest is not all best but the best is surely the simplest!

_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs

Re: stale NFS file handle problem on XFS inode64

[Emmanuel Florac]

Le Mon, 21 Nov 2011 23:28:23 +0800
hank peng <pengxihan@gmail.com> écrivait:

> I noticed that there is a kernel parameter :
> nfs.enable_ino64=
> 			[NFS] enable 64-bit inode numbers.
> 			If zero, the NFS client will fake up a 32-bit
> inode number for the readdir() and stat() syscalls instead
> 			of returning the full 64-bit number.
> 			The default is to return 64-bit inode numbers.
> 
> can I set it to zero, so NFS client can get converted 32bit inode ?

Well you must try it and hope for the best!

-- 
------------------------------------------------------------------------
Emmanuel Florac     |   Direction technique
                    |   Intellique
                    |	<eflorac@intellique.com>
                    |   +33 1 78 94 84 02
------------------------------------------------------------------------

_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs

Re: stale NFS file handle problem on XFS inode64

[Christoph Hellwig]

On Mon, Nov 21, 2011 at 09:00:57PM +0800, hank peng wrote:
> Hi:
> I have a 4TB XFS filesystem mounted with "inode64" option, and I
> exported it using NFS v3, on NFS client, I entered a subdirectory and
> got "stale NFS file handle" error.
> ls -il showed that its inode number exceed 32 bit limit.
> NFS server: kernel version is 2.6.35.6
> NFS client: kernel version is 2.6.18

Do you export the root directory of the XFS filesystem, or a
subdirectory in it?  If it's the former it should work in theory,
althrough I'm not sure how well 64-bit inode numbers work with a client
that old, if it's the latter it needs a few tweaks to work, see

	http://xfs.org/index.php/XFS_FAQ#Q:_Why_doesn.27t_NFS-exporting_subdirectories_of_inode64-mounted_filesystem_work.3F

for details.

_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs

Re: stale NFS file handle problem on XFS inode64

[hank peng]

2011/11/22 Christoph Hellwig <hch@infradead.org>:
> On Mon, Nov 21, 2011 at 09:00:57PM +0800, hank peng wrote:
>> Hi:
>> I have a 4TB XFS filesystem mounted with "inode64" option, and I
>> exported it using NFS v3, on NFS client, I entered a subdirectory and
>> got "stale NFS file handle" error.
>> ls -il showed that its inode number exceed 32 bit limit.
>> NFS server: kernel version is 2.6.35.6
>> NFS client: kernel version is 2.6.18
>
> Do you export the root directory of the XFS filesystem, or a
> subdirectory in it?  If it's the former it should work in theory,
> althrough I'm not sure how well 64-bit inode numbers work with a client
> that old, if it's the latter it needs a few tweaks to work, see
>
The following is what I did:
1. mkfs.xfs /dev/sdb   (whole disk)
2. create a mount point in my root filesystem:
mkdir /mnt/mymount
3. monut xfs with inode64
mount /dev/sdb /mnt/mymount -o inode64
4. export it in NFS /etc/exports
/mnt/mymount   *(rw, no_root,squash, sync)

so, is there any problem here?



>        http://xfs.org/index.php/XFS_FAQ#Q:_Why_doesn.27t_NFS-exporting_subdirectories_of_inode64-mounted_filesystem_work.3F
>
> for details.
>



-- 
The simplest is not all best but the best is surely the simplest!

_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs

Re: stale NFS file handle problem on XFS inode64

[Dave Chinner]

On Tue, Nov 22, 2011 at 08:19:15AM +0800, hank peng wrote:
> 2011/11/22 Christoph Hellwig <hch@infradead.org>:
> > On Mon, Nov 21, 2011 at 09:00:57PM +0800, hank peng wrote:
> >> Hi:
> >> I have a 4TB XFS filesystem mounted with "inode64" option, and I
> >> exported it using NFS v3, on NFS client, I entered a subdirectory and
> >> got "stale NFS file handle" error.
> >> ls -il showed that its inode number exceed 32 bit limit.
> >> NFS server: kernel version is 2.6.35.6
> >> NFS client: kernel version is 2.6.18
> >
> > Do you export the root directory of the XFS filesystem, or a
> > subdirectory in it?  If it's the former it should work in theory,
> > althrough I'm not sure how well 64-bit inode numbers work with a client
> > that old, if it's the latter it needs a few tweaks to work, see
> >
> The following is what I did:
> 1. mkfs.xfs /dev/sdb   (whole disk)
> 2. create a mount point in my root filesystem:
> mkdir /mnt/mymount
> 3. monut xfs with inode64
> mount /dev/sdb /mnt/mymount -o inode64
> 4. export it in NFS /etc/exports
> /mnt/mymount   *(rw, no_root,squash, sync)
                       ^^^^^^^^^^^^^^ no_root_squash
> 
> so, is there any problem here?
> 
> >        http://xfs.org/index.php/XFS_FAQ#Q:_Why_doesn.27t_NFS-exporting_subdirectories_of_inode64-mounted_filesystem_work.3F

It's in the FAQ entry christoph pointed you to:

"However, exporting the root of the filesystem works"

Cheers,

Dave.
-- 
Dave Chinner
david@fromorbit.com

_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs

Re: stale NFS file handle problem on XFS inode64

[hank peng]

2011/11/22 Dave Chinner <david@fromorbit.com>:
> On Tue, Nov 22, 2011 at 08:19:15AM +0800, hank peng wrote:
>> 2011/11/22 Christoph Hellwig <hch@infradead.org>:
>> > On Mon, Nov 21, 2011 at 09:00:57PM +0800, hank peng wrote:
>> >> Hi:
>> >> I have a 4TB XFS filesystem mounted with "inode64" option, and I
>> >> exported it using NFS v3, on NFS client, I entered a subdirectory and
>> >> got "stale NFS file handle" error.
>> >> ls -il showed that its inode number exceed 32 bit limit.
>> >> NFS server: kernel version is 2.6.35.6
>> >> NFS client: kernel version is 2.6.18
>> >
>> > Do you export the root directory of the XFS filesystem, or a
>> > subdirectory in it?  If it's the former it should work in theory,
>> > althrough I'm not sure how well 64-bit inode numbers work with a client
>> > that old, if it's the latter it needs a few tweaks to work, see
>> >
>> The following is what I did:
>> 1. mkfs.xfs /dev/sdb   (whole disk)
>> 2. create a mount point in my root filesystem:
>> mkdir /mnt/mymount
>> 3. monut xfs with inode64
>> mount /dev/sdb /mnt/mymount -o inode64
>> 4. export it in NFS /etc/exports
>> /mnt/mymount   *(rw, no_root,squash, sync)
>                       ^^^^^^^^^^^^^^ no_root_squash
>>
sorry, it is my type mistake, it is no_root_squash.

I think I have already exported the root of my XFS filesystem, am I right?


>> so, is there any problem here?
>>
>> >        http://xfs.org/index.php/XFS_FAQ#Q:_Why_doesn.27t_NFS-exporting_subdirectories_of_inode64-mounted_filesystem_work.3F
>
> It's in the FAQ entry christoph pointed you to:
>
> "However, exporting the root of the filesystem works"
>
> Cheers,
>
> Dave.
> --
> Dave Chinner
> david@fromorbit.com
>



-- 
The simplest is not all best but the best is surely the simplest!

_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs

Re: stale NFS file handle problem on XFS inode64

[hank peng]

unfortunately, seting nfs.enable_ino64 didn't work although it gave me
32 bit inode number indeed, I think there must be something else
wrong.


2011/11/21 Emmanuel Florac <eflorac@intellique.com>:
> Le Mon, 21 Nov 2011 23:28:23 +0800
> hank peng <pengxihan@gmail.com> écrivait:
>
>> I noticed that there is a kernel parameter :
>> nfs.enable_ino64=
>>                       [NFS] enable 64-bit inode numbers.
>>                       If zero, the NFS client will fake up a 32-bit
>> inode number for the readdir() and stat() syscalls instead
>>                       of returning the full 64-bit number.
>>                       The default is to return 64-bit inode numbers.
>>
>> can I set it to zero, so NFS client can get converted 32bit inode ?
>
> Well you must try it and hope for the best!
>
> --
> ------------------------------------------------------------------------
> Emmanuel Florac     |   Direction technique
>                    |   Intellique
>                    |   <eflorac@intellique.com>
>                    |   +33 1 78 94 84 02
> ------------------------------------------------------------------------
>



-- 
The simplest is not all best but the best is surely the simplest!

_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs

Re: stale NFS file handle problem on XFS inode64

[Emmanuel Florac]

Le Tue, 22 Nov 2011 11:22:15 +0800 vous écriviez:

> I think I have already exported the root of my XFS filesystem, am I
> right?

Yes, and the fsid stuff only relates to NFS v4, and as you aren't
providing any fsid, you must be running NFS 3. Now maybe you could use
NFS 4 instead of NFS 3?

-- 
------------------------------------------------------------------------
Emmanuel Florac     |   Direction technique
                    |   Intellique
                    |	<eflorac@intellique.com>
                    |   +33 1 78 94 84 02
------------------------------------------------------------------------

_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs

Re: stale NFS file handle problem on XFS inode64

[Christoph Hellwig]

On Tue, Nov 22, 2011 at 08:19:15AM +0800, hank peng wrote:
> > Do you export the root directory of the XFS filesystem, or a
> > subdirectory in it? ??If it's the former it should work in theory,
> > althrough I'm not sure how well 64-bit inode numbers work with a client
> > that old, if it's the latter it needs a few tweaks to work, see
> >
> The following is what I did:
> 1. mkfs.xfs /dev/sdb   (whole disk)
> 2. create a mount point in my root filesystem:
> mkdir /mnt/mymount
> 3. monut xfs with inode64
> mount /dev/sdb /mnt/mymount -o inode64
> 4. export it in NFS /etc/exports
> /mnt/mymount   *(rw, no_root,squash, sync)
> 
> so, is there any problem here?

That should work fine, and I've tried it a lot of times.  Do you see
the same issue when mounting the fs on the server (not recommended for
production use, just for testing!) and accessing it from the 2.6.35
kernel nfs client?

_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs

Re: stale NFS file handle problem on XFS inode64

[Christoph Hellwig]

On Tue, Nov 22, 2011 at 09:35:48PM +0100, Emmanuel Florac wrote:
> Le Tue, 22 Nov 2011 11:22:15 +0800 vous ??criviez:
> 
> > I think I have already exported the root of my XFS filesystem, am I
> > right?
> 
> Yes, and the fsid stuff only relates to NFS v4, and as you aren't
> providing any fsid, you must be running NFS 3. Now maybe you could use
> NFS 4 instead of NFS 3?

The fsid option is in no way related to NFSv4.  It just overrides the
default fsid taken from the st_dev field of the mounted filesystem.

_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs

Re: stale NFS file handle problem on XFS inode64

[hank peng]

2011/11/23 Christoph Hellwig <hch@infradead.org>:
> On Tue, Nov 22, 2011 at 08:19:15AM +0800, hank peng wrote:
>> > Do you export the root directory of the XFS filesystem, or a
>> > subdirectory in it? ??If it's the former it should work in theory,
>> > althrough I'm not sure how well 64-bit inode numbers work with a client
>> > that old, if it's the latter it needs a few tweaks to work, see
>> >
>> The following is what I did:
>> 1. mkfs.xfs /dev/sdb   (whole disk)
>> 2. create a mount point in my root filesystem:
>> mkdir /mnt/mymount
>> 3. monut xfs with inode64
>> mount /dev/sdb /mnt/mymount -o inode64
>> 4. export it in NFS /etc/exports
>> /mnt/mymount   *(rw, no_root,squash, sync)
>>
>> so, is there any problem here?
>
> That should work fine, and I've tried it a lot of times.  Do you see
> the same issue when mounting the fs on the server (not recommended for
> production use, just for testing!) and accessing it from the 2.6.35
> kernel nfs client?
>
yes, I have already done that. I mount the expoted NFS directory in
same machine(kernel version is 2.6.35.6), but it still didn't work.

Then, I used fsid=$(my exported filesystem ID) option in /etc/exports
and tried again, it still gave me back "stale NFS file handle" when
entering a specific subdirectory.
I noticed that from NFS client(in same machine with NFS server) side,
the inode number of this subdirectory was changed to be within 32
limit, but  still "stale NFS file handle" came back.

In local filesytem, I entered the mounted point and subdirectory, all is OK.

>



-- 
The simplest is not all best but the best is surely the simplest!

_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs

Re: stale NFS file handle problem on XFS inode64

[Christoph Hellwig]

On Wed, Nov 23, 2011 at 09:19:26AM +0800, hank peng wrote:
> yes, I have already done that. I mount the expoted NFS directory in
> same machine(kernel version is 2.6.35.6), but it still didn't work.

Ok, sounds like this is a server side issue.

> 
> Then, I used fsid=$(my exported filesystem ID) option in /etc/exports
> and tried again, it still gave me back "stale NFS file handle" when
> entering a specific subdirectory.
> I noticed that from NFS client(in same machine with NFS server) side,
> the inode number of this subdirectory was changed to be within 32
> limit, but  still "stale NFS file handle" came back.

Do you mean the NFS mount displayed a 32-bit inode number, but it was still
64-bit on the XFS filesystem directly?  Can you post the inode numbers
(e.g. from ls -i) for both sides?

_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs