nfs is stupid ("getfh failed")

nfs is stupid ("getfh failed")

[Michael Rothwell]

Two systems that worked fine for weeks, both running 2.4.[7,8] kernels. The
server is running 2.4.8 and exporting a reiserfs filesystem via nfs. Or it
was, anyway. The server was shut down and brought back up (power failure).
The client was then
rebooted.

server# cat /etc/exports
/export 192.168.1.*(rw,no_root_squash)
/export/home 192.168.1.*(rw,no_root_squash)

client# mount /export
mount: 192.168.1.1:/export failed, reason given by server: Permission denied

server# tail /var/log/messages
Sep  6 09:37:43 gateway rpc.mountd: authenticated mount request from
192.168.1.133:933 for /export (/export)
Sep  6 09:37:43 gateway rpc.mountd: getfh failed: Operation not permitted

... so,  rebooting two working systems seems to kill NFS. Any ideas why?

On a related topic, will Linux ever have a better file-service protocol?

Re: nfs is stupid ("getfh failed")

[Jamie Lokier]

Michael Rothwell wrote:
> server# tail /var/log/messages
> Sep  6 09:37:43 gateway rpc.mountd: authenticated mount request from
> 192.168.1.133:933 for /export (/export)
> Sep  6 09:37:43 gateway rpc.mountd: getfh failed: Operation not permitted

I'm seeing this message quite often with one Linux 2.4.7 system
automounting another.  As long as A has B's filesystem mounted, all is
ok.  Then A times out, unmounts, and later wants to remount B's
filesystem.  Then, sometimes, I see a message much like yours.

It doesn't seem to need a reboot to cause this problem, and the fix I
have found is to kill and restart the NFS server: /etc/init.d/nfs
restart.

I have no idea why it happens, or why restarting nfsd or mountd fixes it.

-- Jamie

Re: nfs is stupid ("getfh failed")

[Neil Brown]

On Friday September 7, lk@tantalophile.demon.co.uk wrote:
> Michael Rothwell wrote:
> > server# tail /var/log/messages
> > Sep  6 09:37:43 gateway rpc.mountd: authenticated mount request from
> > 192.168.1.133:933 for /export (/export)
> > Sep  6 09:37:43 gateway rpc.mountd: getfh failed: Operation not permitted
> 
> I'm seeing this message quite often with one Linux 2.4.7 system
> automounting another.  As long as A has B's filesystem mounted, all is
> ok.  Then A times out, unmounts, and later wants to remount B's
> filesystem.  Then, sometimes, I see a message much like yours.
> 
> It doesn't seem to need a reboot to cause this problem, and the fix I
> have found is to kill and restart the NFS server: /etc/init.d/nfs
> restart.
> 
> I have no idea why it happens, or why restarting nfsd or mountd fixes it.

Show me your /etc/exports....

If you export a directory and a subdirectory of that directory - both
on the same filesysem, you can get this.
If you export a directory to both an IP address (or subnet) and a
hostname (or wildcard or netgroup) this can also happen.

NeilBrown

Re: nfs is stupid ("getfh failed")

[Jamie Lokier]

Neil Brown wrote:
> > I'm seeing this message quite often with one Linux 2.4.7 system
> > automounting another.  As long as A has B's filesystem mounted, all is
> > ok.  Then A times out, unmounts, and later wants to remount B's
> > filesystem.  Then, sometimes, I see a message much like yours.
> > 
> > It doesn't seem to need a reboot to cause this problem, and the fix I
> > have found is to kill and restart the NFS server: /etc/init.d/nfs
> > restart.
> > 
> > I have no idea why it happens, or why restarting nfsd or mountd fixes it.
> 
> Show me your /etc/exports....
> 
> If you export a directory and a subdirectory of that directory - both
> on the same filesysem, you can get this.
> If you export a directory to both an IP address (or subnet) and a
> hostname (or wildcard or netgroup) this can also happen.

Yes, I'm doing the second of those.  No alternative -- I need write
access from some hosts, and read access to all the rest (who are
dynamically allocated) on the subnet.

It's clearly a bug in the NFS server then.

-- Jamie

/etc/exports:

# I want ro granted to all hosts in the 172.30.* subnet, but rw
# granted to the machines in @aquarius_outside which is within that
# subnet.  Doesn't work, whichever order I write the lines.
# (The machines in @aquarius_hosts always get read only access).
#
# Solution, though I don't like it: write the host names explicitly.
#
# The 192.168.64.192/19 subnetwork is for temporarily assigned IPs on
# the Aquarius test network.  /kickstart is used for kickstart
# installations on machines plugged in temporarily.

/home		172.30.0.0/16(ro,root_squash) \
		galatea.degree2.com(rw,root_squash) \
		ariel.degree2.com(rw,root_squash) \
		@aquarius_hosts(rw,root_squash) \
		@aquarius_outside(rw,root_squash)

/kickstart	172.30.0.0/16(ro,root_squash) \
		@aquarius_hosts(ro,root_squash) \
		@aquarius_outside(ro,root_squash) \
		192.168.64.192/255.255.255.224(ro,root_squash)

Re: nfs is stupid ("getfh failed")

[Neil Brown]

On Tuesday September 11, lk@tantalophile.demon.co.uk wrote:
> 
> Yes, I'm doing the second of those.  No alternative -- I need write
> access from some hosts, and read access to all the rest (who are
> dynamically allocated) on the subnet.
> 
> It's clearly a bug in the NFS server then.

Actually, it's a bug in nfs-utils, that I have a patch for, but
haven't got arround to commiting it to the CVS on sourceforge yet.

NeilBrown


Index: utils/mountd/auth.c
===================================================================
RCS file: /cvsroot/nfs/nfs-utils/utils/mountd/auth.c,v
retrieving revision 1.6
diff -u -r1.6 auth.c
--- utils/mountd/auth.c	2000/11/27 23:46:35	1.6
+++ utils/mountd/auth.c	2001/07/24 10:38:37
@@ -78,20 +78,10 @@
 	}
 	auth_fixpath(path);
 
-	/* First try it w/o doing a hostname lookup... */
-	*hpp = get_hostent((const char *)&addr, sizeof(addr), AF_INET);
-	exp = export_find(*hpp, path);
-
-	if (!exp) {
-	    /* Ok, that didn't fly.  Try it with a reverse lookup. */
-	    free (*hpp);
-	    *hpp = gethostbyaddr((const char *)&addr, sizeof(addr),
-				 AF_INET);
-	    if (!(*hpp)) {
-		*error = no_entry;
-		*hpp = get_hostent((const char *)&addr, sizeof(addr), AF_INET);
-		return NULL;
-	    } else {
+	if (!(*hpp = gethostbyaddr((const char *)&addr, sizeof(addr), AF_INET)))
+		*hpp = get_hostent((const char *)&addr, sizeof(addr),
+				   AF_INET);
+	else {
 		/* must make sure the hostent is authorative. */
 		char **sp;
 		struct hostent *forward = NULL;
@@ -123,14 +113,12 @@
 			*error = no_forward_dns;
 			return NULL;
 		}
-	    }
+	}
 
-	    if (!(exp = export_find(*hpp, path))) {
+	if (!(exp = export_find(*hpp, path))) {
 		*error = no_entry;
 		return NULL;
-	    }
 	}
-
 	if (!exp->m_mayexport) {
 		*error = not_exported;
 		return NULL;

Re: nfs is stupid ("getfh failed")

[Neil Brown]

On Thursday September 6, rothwell@holly-springs.nc.us wrote:
> Two systems that worked fine for weeks, both running 2.4.[7,8] kernels. The
> server is running 2.4.8 and exporting a reiserfs filesystem via nfs. Or it
> was, anyway. The server was shut down and brought back up (power failure).
> The client was then
> rebooted.
> 
> server# cat /etc/exports
> /export 192.168.1.*(rw,no_root_squash)
> /export/home 192.168.1.*(rw,no_root_squash)

This is not allowed, and makes no sense.
exporting "/export" exports everything below /export that is in the
same filesystem, so exporting "/export/home" as well servers no
purpose and is not allowed (unless /export/home is a separate
filesystem).

Simply remove the second line and your problems should go away.


> client# mount /export
> mount: 192.168.1.1:/export failed, reason given by server: Permission denied
> 
> server# tail /var/log/messages
> Sep  6 09:37:43 gateway rpc.mountd: authenticated mount request from
> 192.168.1.133:933 for /export (/export)
> Sep  6 09:37:43 gateway rpc.mountd: getfh failed: Operation not permitted
> 
> ... so,  rebooting two working systems seems to kill NFS. Any ideas why?
> 
> On a related topic, will Linux ever have a better file-service protocol?

This is an unanswerable question.  The future of Linux is completely
undefined until it happens.

What exactly do you mean by "better" anyway?

NeilBrown
 

> 
> 
> 
> 
> -
> To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> Please read the FAQ at  http://www.tux.org/lkml/

Re: nfs is stupid ("getfh failed")

[Michael Rothwell]

From: "Neil Brown" <neilb@cse.unsw.edu.au>

> This is not allowed, and makes no sense.

It apparently is (or was, anyway) allowed, because it worked until the
server was rebooted. Fluke?
Are you saying that, if I export "/export", I can mount "/export/home" from
a client machine? That's nice.

> Simply remove the second line and your problems should go away.

Thanks. I actually switched to two explicit exports; /export/files and
/export/home, which works.


> > On a related topic, will Linux ever have a better file-service protocol?
>
> This is an unanswerable question.  The future of Linux is completely
> undefined until it happens.

Just wondering if there's been any talk, plans, etc. of an alternative for
NFS.

> What exactly do you mean by "better" anyway?

Better security, better performance.

Thanks,

-M

Re: nfs is stupid ("getfh failed")

[Tim Walberg]

[-- Attachment #1: Type: text/plain, Size: 1230 bytes --]

On 09/07/2001 08:58 -0400, Michael Rothwell wrote:
>>	From: "Neil Brown" <neilb@cse.unsw.edu.au>
>>	
>>	> This is not allowed, and makes no sense.
>>	
>>	It apparently is (or was, anyway) allowed, because it worked until the
>>	server was rebooted. Fluke?
>>	Are you saying that, if I export "/export", I can mount "/export/home" from
>>	a client machine? That's nice.
>>	

Yes. At least that's how Solaris, Irix, HP/UX, and several others do it. The
general rule is something along the lines of "if /a is exported, anything that
is a subdirectory of /a is accessible via NFS, so long as it is on the same
device (partition - actually major/minor pair)". So, if /export/home in your
case lives in the same file system as /export, it is accessible via NFS
as well. Now if /export/home is it's own mount point (i.e. a separate file
system), then it needs to be exported explicitly.

>>	> Simply remove the second line and your problems should go away.
>>	
>>	Thanks. I actually switched to two explicit exports; /export/files and
>>	/export/home, which works.
>>	

This works as well, with the caveat that you cannot then mount /export itself
on client machines.

				tw

-- 
twalberg@mindspring.com

[-- Attachment #2: Type: application/pgp-signature, Size: 175 bytes --]

Re: nfs is stupid ("getfh failed")

[Johan Kullstam]

"Michael Rothwell" <rothwell@holly-springs.nc.us> writes:

> Just wondering if there's been any talk, plans, etc. of an alternative for
> NFS.

there's coda.

-- 
J o h a n  K u l l s t a m
[kullstam@ne.mediaone.net]
Don't Fear the Penguin!

Re: nfs is stupid ("getfh failed")

[Alan Cox]

> "Michael Rothwell" <rothwell@holly-springs.nc.us> writes:
> 
> > Just wondering if there's been any talk, plans, etc. of an alternative for
> > NFS.
> 
> there's coda.

You probably want inter-mezzo rather than coda for that kind of file
service. Coda is a research project, inter-mezzo is kind of "distilled coda
and afs", making it a real fs.  http://www.inter-mezzo.org

Re: nfs is stupid ("getfh failed")

[Henning P. Schmiedehausen]

Alan Cox <alan@lxorguk.ukuu.org.uk> writes:

>> "Michael Rothwell" <rothwell@holly-springs.nc.us> writes:
>> 
>> > Just wondering if there's been any talk, plans, etc. of an alternative for
>> > NFS.
>> 
>> there's coda.

>You probably want inter-mezzo rather than coda for that kind of file
>service. Coda is a research project, inter-mezzo is kind of "distilled coda
>and afs", making it a real fs.  http://www.inter-mezzo.org

And it currently looks "Linux only", which kind of defeats IMHO the
purpose.

In a heterogenous environment there is NFS (and SMB aka CIFS aka <name
of the day>, of course. The horrors! The horrors! :-) ) and nothing
much else.

	Regards
		Henning

-- 
Dipl.-Inf. (Univ.) Henning P. Schmiedehausen       -- Geschaeftsfuehrer
INTERMETA - Gesellschaft fuer Mehrwertdienste mbH     hps@intermeta.de

Am Schwabachgrund 22  Fon.: 09131 / 50654-0   info@intermeta.de
D-91054 Buckenhof     Fax.: 09131 / 50654-20   

Re: nfs is stupid ("getfh failed")

[Michael Rothwell]

On 09 Sep 2001 10:05:00 +0000, Henning P. Schmiedehausen wrote:

> And it currently looks "Linux only", which kind of defeats IMHO the
> purpose.

Considering I use only Linux at home, that's fine with me. :) For
Windows interaction, I use samba. I have not even booted my Sparc into
Solaris in ... oh, about a year.

Is intermezzo usable? Reliable? Any horror stories? Success stories? 

-M

Re: nfs is stupid ("getfh failed")

[Neil Brown]

On Friday September 7, rothwell@holly-springs.nc.us wrote:
> 
> Just wondering if there's been any talk, plans, etc. of an alternative for
> NFS.
> 
> > What exactly do you mean by "better" anyway?
> 
> Better security, better performance.
> 
> Thanks,
> 
> -M

NFSv2 and 3 do allow better security, but it isn't often implemented.
I am working on putting some infrastructure in place so that
crypto-authentication can be added to nfsd in a nice modular way.
Ofcourse the client will need to speak the same authentication
protocol too.

Then there is NFSv4 which might improve performance in some
circumstances, though it could do more....

SUNs "cachefs" concept can be used to improve read performance by
caching a lot more of server-data on the client.  That could be
implemented for Linux, but I don't know of anyone with serious plans.

NeilBrown

Re: nfs is stupid ("getfh failed")

[Marcus Sundberg]

neilb@cse.unsw.edu.au (Neil Brown) writes:

> SUNs "cachefs" concept can be used to improve read performance by
> caching a lot more of server-data on the client.  That could be
> implemented for Linux, but I don't know of anyone with serious plans.

cachefs sucks. It doesn't seem to cache stat(2) information.
Doing ls -F in a ~100-entries directory takes several seconds over
a link with 50ms round-trip time.

//Marcus
-- 
---------------------------------+---------------------------------
         Marcus Sundberg         |      Phone: +46 707 452062
   Embedded Systems Consultant   |     Email: marcus@cendio.se
        Cendio Systems AB        |      http://www.cendio.com

Re: nfs is stupid ("getfh failed")

[Neil Brown]

On  September 10, marcus@cendio.se wrote:
> neilb@cse.unsw.edu.au (Neil Brown) writes:
> 
> > SUNs "cachefs" concept can be used to improve read performance by
> > caching a lot more of server-data on the client.  That could be
> > implemented for Linux, but I don't know of anyone with serious plans.
> 
> cachefs sucks. It doesn't seem to cache stat(2) information.
> Doing ls -F in a ~100-entries directory takes several seconds over
> a link with 50ms round-trip time.

Well, I said "concept" not "implementation", but I suspect that
Solaris cachefs does cache stat information.  Maybe you just need to
increase the timeouts for the attribute cache.

NeilBrown

Re: nfs is stupid ("getfh failed")

[Marcus Sundberg]

neilb@cse.unsw.edu.au (Neil Brown) writes:

> On  September 10, marcus@cendio.se wrote:
> > cachefs sucks. It doesn't seem to cache stat(2) information.
> > Doing ls -F in a ~100-entries directory takes several seconds over
> > a link with 50ms round-trip time.
> 
> Well, I said "concept" not "implementation", but I suspect that
> Solaris cachefs does cache stat information.  Maybe you just need to
> increase the timeouts for the attribute cache.

Considering that I did several ls'es on the order of milliseconds
apart I doubt that would help...

//Marcus
-- 
---------------------------------+---------------------------------
         Marcus Sundberg         |      Phone: +46 707 452062
   Embedded Systems Consultant   |     Email: marcus@cendio.se
        Cendio Systems AB        |      http://www.cendio.com

Re: nfs is stupid ("getfh failed")

[Trond Myklebust]

>>>>> " " == Marcus Sundberg <marcus@cendio.se> writes:

     > neilb@cse.unsw.edu.au (Neil Brown) writes:
    >> On September 10, marcus@cendio.se wrote:
    >> > cachefs sucks. It doesn't seem to cache stat(2) information.
    >> > Doing ls -F in a ~100-entries directory takes several seconds
    >> > over a link with 50ms round-trip time.
    >>
    >> Well, I said "concept" not "implementation", but I suspect that
    >> Solaris cachefs does cache stat information.  Maybe you just
    >> need to increase the timeouts for the attribute cache.

     > Considering that I did several ls'es on the order of
     > milliseconds apart I doubt that would help...

The NFS close-to-open cache consistency requirement forces them to
compare the attribute cache to the server every time someone does a
call to open(). This is true whether or not one uses cachefs.

After the file has actually been opened, you can call fstat() as many
times as you like. The cached attributes to be used, and they will
then be checked on the server only after the cache times out.

Cheers,
   Trond

Re: nfs is stupid ("getfh failed")

[Neil Brown]

On  September 12, marcus@cendio.se wrote:
> neilb@cse.unsw.edu.au (Neil Brown) writes:
> 
> > On  September 10, marcus@cendio.se wrote:
> > > cachefs sucks. It doesn't seem to cache stat(2) information.
> > > Doing ls -F in a ~100-entries directory takes several seconds over
> > > a link with 50ms round-trip time.
> > 
> > Well, I said "concept" not "implementation", but I suspect that
> > Solaris cachefs does cache stat information.  Maybe you just need to
> > increase the timeouts for the attribute cache.
> 
> Considering that I did several ls'es on the order of milliseconds
> apart I doubt that would help...

Odd..
I just tried out cachefs (for the first time) on Solaris2.6.
I mounted my home directory (which has 125 entries) and did 
 ls -F
while watching network traffic.

Except of the first time, and after every 30 seconds (the default
attribute cache timeout) there was only 1 RPC request for each
  ls -F
and that was to check the modify time on the directory.
But then that is exactly the same traffic that I see when I do
  ls -F
in my home directory over normal NFS (v2).

I could do 100 "ls -F" runns in about 4 seconds.
This is on regular 100Mbit ethernet.

NeilBrown