[Cluster-devel] NFS v2 not compatible with GFS2

[Cluster-devel] NFS v2 not compatible with GFS2

[Craig Johnston]

We have found that NFS version 2 does not seem to be compatible with
an NFS exported GFS2 file system.  The NFS client can mount the file
system, but any file I/O results in Stale File Handle and I/O errors.

It looks very much like the problem described here -
https://bugzilla.redhat.com/show_bug.cgi?id=229345

Has there been any progress on resolving this issue, or are we out of
luck.  We have a legacy device that will only use NFS v2, and we would
very much like to use GFS2 as we do for the rest of our system.

To replicate the problem for a GFS2 exported file system (say
/data/mygfs2_fs on host foo ), mount it on another host as
"% mount -o nfsvers=2 foo:/data/mygfs2_fs /mnt/tmp"

You will find that you can list the directory entries, but any attempt
to read or write to files will fail.

Note, we find that the GFS file system does not seem to have problems
with NFS v2 clients, only GFS2.

Any help would be appreciated.
Craig

[Cluster-devel] NFS v2 not compatible with GFS2

[Steven Whitehouse]

Hi,

On Tue, 2009-04-07 at 13:04 -0700, Craig Johnston wrote:
> We have found that NFS version 2 does not seem to be compatible with
> an NFS exported GFS2 file system.  The NFS client can mount the file
> system, but any file I/O results in Stale File Handle and I/O errors.
> 
> It looks very much like the problem described here -
> https://bugzilla.redhat.com/show_bug.cgi?id=229345
> 
> Has there been any progress on resolving this issue, or are we out of
> luck.  We have a legacy device that will only use NFS v2, and we would
> very much like to use GFS2 as we do for the rest of our system.
> 
> To replicate the problem for a GFS2 exported file system (say
> /data/mygfs2_fs on host foo ), mount it on another host as
> "% mount -o nfsvers=2 foo:/data/mygfs2_fs /mnt/tmp"
> 
> You will find that you can list the directory entries, but any attempt
> to read or write to files will fail.
> 
> Note, we find that the GFS file system does not seem to have problems
> with NFS v2 clients, only GFS2.
> 
> Any help would be appreciated.
> Craig
> 
Yes, it is a bug inherited from GFS. GFS2 uses the same filehandles
(i.e. too large for NFSv2) so that for the time being, only NFSv3+ will
work with GFS2. I don't know why GFS appears to work and GFS2 doesn't -
they should be identical in that respect,

Steve.

[Cluster-devel] NFS v2 not compatible with GFS2

[Craig Johnston]

On Wed, Apr 8, 2009 at 1:54 AM, Steven Whitehouse <swhiteho@redhat.com> wrote:
>
> Hi,
>
> On Tue, 2009-04-07 at 13:04 -0700, Craig Johnston wrote:
> > We have found that NFS version 2 does not seem to be compatible with
> > an NFS exported GFS2 file system.  The NFS client can mount the file
> > system, but any file I/O results in Stale File Handle and I/O errors.
> >
> > It looks very much like the problem described here -
> > https://bugzilla.redhat.com/show_bug.cgi?id=229345
> >
> > Has there been any progress on resolving this issue, or are we out of
> > luck.  We have a legacy device that will only use NFS v2, and we would
> > very much like to use GFS2 as we do for the rest of our system.
> >
> > To replicate the problem for a GFS2 exported file system (say
> > /data/mygfs2_fs on host foo ), mount it on another host as
> > "% mount -o nfsvers=2 foo:/data/mygfs2_fs /mnt/tmp"
> >
> > You will find that you can list the directory entries, but any attempt
> > to read or write to files will fail.
> >
> > Note, we find that the GFS file system does not seem to have problems
> > with NFS v2 clients, only GFS2.
> >
> > Any help would be appreciated.
> > Craig
> >
> Yes, it is a bug inherited from GFS. GFS2 uses the same filehandles
> (i.e. too large for NFSv2) so that for the time being, only NFSv3+ will
> work with GFS2. I don't know why GFS appears to work and GFS2 doesn't -
> they should be identical in that respect,
>
> Steve.
>
>
Bummer.  This is a capability we we really need, as we have to
interface with some older systems running VxWorks that only supports
NFS v2.  We have verified that GFS supports NFS v2.  Does this mean
there is hope that GFS2 could also support NFS v2 since they use the
same filehandles?  Where should I start poking around if I wanted to
delve into the source?

Thanks,
Craig

[Cluster-devel] NFS v2 not compatible with GFS2

[Steven Whitehouse]

Hi,

On Wed, 2009-04-08 at 08:16 -0700, Craig Johnston wrote:
> On Wed, Apr 8, 2009 at 1:54 AM, Steven Whitehouse <swhiteho@redhat.com> wrote:
> >
> > Hi,
> >
> > On Tue, 2009-04-07 at 13:04 -0700, Craig Johnston wrote:
> > > We have found that NFS version 2 does not seem to be compatible with
> > > an NFS exported GFS2 file system.  The NFS client can mount the file
> > > system, but any file I/O results in Stale File Handle and I/O errors.
> > >
> > > It looks very much like the problem described here -
> > > https://bugzilla.redhat.com/show_bug.cgi?id=229345
> > >
> > > Has there been any progress on resolving this issue, or are we out of
> > > luck.  We have a legacy device that will only use NFS v2, and we would
> > > very much like to use GFS2 as we do for the rest of our system.
> > >
> > > To replicate the problem for a GFS2 exported file system (say
> > > /data/mygfs2_fs on host foo ), mount it on another host as
> > > "% mount -o nfsvers=2 foo:/data/mygfs2_fs /mnt/tmp"
> > >
> > > You will find that you can list the directory entries, but any attempt
> > > to read or write to files will fail.
> > >
> > > Note, we find that the GFS file system does not seem to have problems
> > > with NFS v2 clients, only GFS2.
> > >
> > > Any help would be appreciated.
> > > Craig
> > >
> > Yes, it is a bug inherited from GFS. GFS2 uses the same filehandles
> > (i.e. too large for NFSv2) so that for the time being, only NFSv3+ will
> > work with GFS2. I don't know why GFS appears to work and GFS2 doesn't -
> > they should be identical in that respect,
> >
> > Steve.
> >
> >
> Bummer.  This is a capability we we really need, as we have to
> interface with some older systems running VxWorks that only supports
> NFS v2.  We have verified that GFS supports NFS v2.  Does this mean
> there is hope that GFS2 could also support NFS v2 since they use the
> same filehandles?  Where should I start poking around if I wanted to
> delve into the source?
> 
> Thanks,
> Craig

By all means have a look in the source. I suspect that there would be
issues if you ever had a filesystem larger than 2^32 blocks with GFS
using the NFSv2 handles. It might just be silently ignoring that issue,
so I'd be careful.

I would like to see NFSv2 working with GFS2, so I'd be happy to accept
patches if you can work out how to stuff all the required information
into such a small file handle. Also it would need to leave the existing
file handles for NFSv3 alone as we must not change that format in an
incompatible way.

All the code is easy to find in ops_export.c and its all self-contained
in that file,

Steve.

[Cluster-devel] NFS v2 not compatible with GFS2

[agspoon]

On Apr 8, 2009 8:23am, Steven Whitehouse <swhiteho@redhat.com> wrote:
> Hi,



> On Wed, 2009-04-08 at 08:16 -0700, Craig Johnston wrote:

> > On Wed, Apr 8, 2009 at 1:54 AM, Steven Whitehouse swhiteho at redhat.com>  
> wrote:

> > >

> > > Hi,

> > >

> > > On Tue, 2009-04-07 at 13:04 -0700, Craig Johnston wrote:

> > > > We have found that NFS version 2 does not seem to be compatible with

> > > > an NFS exported GFS2 file system. The NFS client can mount the file

> > > > system, but any file I/O results in Stale File Handle and I/O  
> errors.

> > > >

> > > > It looks very much like the problem described here -

> > > > https://bugzilla.redhat.com/show_bug.cgi?id=229345

> > > >

> > > > Has there been any progress on resolving this issue, or are we out  
> of

> > > > luck. We have a legacy device that will only use NFS v2, and we  
> would

> > > > very much like to use GFS2 as we do for the rest of our system.

> > > >

> > > > To replicate the problem for a GFS2 exported file system (say

> > > > /data/mygfs2_fs on host foo ), mount it on another host as

> > > > "% mount -o nfsvers=2 foo:/data/mygfs2_fs /mnt/tmp"

> > > >

> > > > You will find that you can list the directory entries, but any  
> attempt

> > > > to read or write to files will fail.

> > > >

> > > > Note, we find that the GFS file system does not seem to have  
> problems

> > > > with NFS v2 clients, only GFS2.

> > > >

> > > > Any help would be appreciated.

> > > > Craig

> > > >

> > > Yes, it is a bug inherited from GFS. GFS2 uses the same filehandles

> > > (ie too large for NFSv2) so that for the time being, only NFSv3+ will

> > > work with GFS2. I don't know why GFS appears to work and GFS2 doesn't  
> -

> > > they should be identical in that respect,

> > >

> > > Steve.

> > >

> > >

> > Bummer. This is a capability we we really need, as we have to

> > interface with some older systems running VxWorks that only supports

> > NFS v2. We have verified that GFS supports NFS v2. Does this mean

> > there is hope that GFS2 could also support NFS v2 since they use the

> > same filehandles? Where should I start poking around if I wanted to

> > delve into the source?

> >

> > Thanks,

> > Craig



> By all means have a look in the source. I suspect that there would be

> issues if you ever had a filesystem larger than 2^32 blocks with GFS

> using the NFSv2 handles. It might just be silently ignoring that issue,

> so I'd be careful.



> I would like to see NFSv2 working with GFS2, so I'd be happy to accept

> patches if you can work out how to stuff all the required information

> into such a small file handle. Also it would need to leave the existing

> file handles for NFSv3 alone as we must not change that format in an

> incompatible way.



> All the code is easy to find in ops_export.c and its all self-contained

> in that file,



> Steve.






Well, we found a fix that works for us. To be specific, we are working with  
GFS2 in RHEL5.3.

Inspecting ops_export.c for GFS2, and comparing that to the implementation  
in GFS, we discovered that gfs2_decode_fh() is using fh_len rather than  
fh_type. Per the published API, fh_len does not necessarily tell the whole  
story, as it is the maximum size not necessarily the actual size of the  
file handle. For NFSv2 the encoded file handle is of type  
GFS2_SMALL_FH_SIZE, but this was being ignored in the decode method. The  
following patch has been tested extensively in our mixed NFSv3 & NFSv2  
environment without any trouble.

----------------------------------------------------------------------------------------------------------------------
diff -up linux-2.6.18.i686/fs/gfs2/ops_export.c.gfs2-nfs2  
linux-2.6.18.i686/fs/gfs2/ops_export.c
--- linux-2.6.18.i686/fs/gfs2/ops_export.c.gfs2-nfs2 2009-04-16  
09:57:19.000000000 -0700
+++ linux-2.6.18.i686/fs/gfs2/ops_export.c 2009-04-16 10:02:02.000000000  
-0700
@@ -42,7 +42,7 @@ static struct dentry *gfs2_decode_fh(str

memset(&parent, 0, sizeof(struct gfs2_inum));

- switch (fh_len) {
+ switch (fh_type) {
case GFS2_LARGE_FH_SIZE:
parent.no_formal_ino = ((u64)be32_to_cpu(fh[4])) << 32;
parent.no_formal_ino |= be32_to_cpu(fh[5]);
----------------------------------------------------------------------------------------------------------------------

Craig
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://listman.redhat.com/archives/cluster-devel/attachments/20090417/c9a537bd/attachment.htm>

[Cluster-devel] NFS v2 not compatible with GFS2

[Steven Whitehouse]

Hi,

On Fri, 2009-04-17 at 22:07 +0000, agspoon at gmail.com wrote:
> On Apr 8, 2009 8:23am, Steven Whitehouse <swhiteho@redhat.com> wrote:
> > Hi,
> > 
> > 
> > 
> > On Wed, 2009-04-08 at 08:16 -0700, Craig Johnston wrote:
> > 
> > > On Wed, Apr 8, 2009 at 1:54 AM, Steven Whitehouse
> swhiteho at redhat.com> wrote:
> > 
> > > >
> > 
> > > > Hi,
> > 
> > > >
> > 
> > > > On Tue, 2009-04-07 at 13:04 -0700, Craig Johnston wrote:
> > 
> > > > > We have found that NFS version 2 does not seem to be
> compatible with
> > 
> > > > > an NFS exported GFS2 file system. The NFS client can mount the
> file
> > 
> > > > > system, but any file I/O results in Stale File Handle and I/O
> errors.
> > 
> > > > >
> > 
> > > > > It looks very much like the problem described here -
> > 
> > > > > https://bugzilla.redhat.com/show_bug.cgi?id=229345
> > 
> > > > >
> > 
> > > > > Has there been any progress on resolving this issue, or are we
> out of
> > 
> > > > > luck. We have a legacy device that will only use NFS v2, and
> we would
> > 
> > > > > very much like to use GFS2 as we do for the rest of our
> system.
> > 
> > > > >
> > 
> > > > > To replicate the problem for a GFS2 exported file system (say
> > 
> > > > > /data/mygfs2_fs on host foo ), mount it on another host as
> > 
> > > > > "% mount -o nfsvers=2 foo:/data/mygfs2_fs /mnt/tmp"
> > 
> > > > >
> > 
> > > > > You will find that you can list the directory entries, but any
> attempt
> > 
> > > > > to read or write to files will fail.
> > 
> > > > >
> > 
> > > > > Note, we find that the GFS file system does not seem to have
> problems
> > 
> > > > > with NFS v2 clients, only GFS2.
> > 
> > > > >
> > 
> > > > > Any help would be appreciated.
> > 
> > > > > Craig
> > 
> > > > >
> > 
> > > > Yes, it is a bug inherited from GFS. GFS2 uses the same
> filehandles
> > 
> > > > (i.e. too large for NFSv2) so that for the time being, only
> NFSv3+ will
> > 
> > > > work with GFS2. I don't know why GFS appears to work and GFS2
> doesn't -
> > 
> > > > they should be identical in that respect,
> > 
> > > >
> > 
> > > > Steve.
> > 
> > > >
> > 
> > > >
> > 
> > > Bummer. This is a capability we we really need, as we have to
> > 
> > > interface with some older systems running VxWorks that only
> supports
> > 
> > > NFS v2. We have verified that GFS supports NFS v2. Does this mean
> > 
> > > there is hope that GFS2 could also support NFS v2 since they use
> the
> > 
> > > same filehandles? Where should I start poking around if I wanted
> to
> > 
> > > delve into the source?
> > 
> > >
> > 
> > > Thanks,
> > 
> > > Craig
> > 
> > 
> > 
> > By all means have a look in the source. I suspect that there would
> be
> > 
> > issues if you ever had a filesystem larger than 2^32 blocks with GFS
> > 
> > using the NFSv2 handles. It might just be silently ignoring that
> issue,
> > 
> > so I'd be careful.
> > 
> > 
> > 
> > I would like to see NFSv2 working with GFS2, so I'd be happy to
> accept
> > 
> > patches if you can work out how to stuff all the required
> information
> > 
> > into such a small file handle. Also it would need to leave the
> existing
> > 
> > file handles for NFSv3 alone as we must not change that format in an
> > 
> > incompatible way.
> > 
> > 
> > 
> > All the code is easy to find in ops_export.c and its all
> self-contained
> > 
> > in that file,
> > 
> > 
> > 
> > Steve.
> > 
> > 
> > 
> > 
> > 
> 
> Well, we found a fix that works for us. To be specific, we are working
> with GFS2 in RHEL5.3.
> 
> Inspecting ops_export.c for GFS2, and comparing that to the
> implementation in GFS, we discovered that gfs2_decode_fh() is using
> fh_len rather than fh_type. Per the published API, fh_len does not
> necessarily tell the whole story, as it is the maximum size not
> necessarily the actual size of the file handle. For NFSv2 the encoded
> file handle is of type GFS2_SMALL_FH_SIZE, but this was being ignored
> in the decode method. The following patch has been tested extensively
> in our mixed NFSv3 & NFSv2 environment without any trouble.
> 
Ok. Can you file a bugzilla with that patch in it, and I'll get it on
the list,

Steve.

> ----------------------------------------------------------------------------------------------------------------------
> diff -up linux-2.6.18.i686/fs/gfs2/ops_export.c.gfs2-nfs2
> linux-2.6.18.i686/fs/gfs2/ops_export.c
> --- linux-2.6.18.i686/fs/gfs2/ops_export.c.gfs2-nfs2 2009-04-16
> 09:57:19.000000000 -0700
> +++ linux-2.6.18.i686/fs/gfs2/ops_export.c 2009-04-16
> 10:02:02.000000000 -0700
> @@ -42,7 +42,7 @@ static struct dentry *gfs2_decode_fh(str
> 
> memset(&parent, 0, sizeof(struct gfs2_inum));
> 
> - switch (fh_len) {
> + switch (fh_type) {
> case GFS2_LARGE_FH_SIZE:
> parent.no_formal_ino = ((u64)be32_to_cpu(fh[4])) << 32;
> parent.no_formal_ino |= be32_to_cpu(fh[5]);
> ----------------------------------------------------------------------------------------------------------------------
> 
> Craig