NFSv4 Kernel-server requires 2x manual restart after reboot? Maybe a timing problem?

NFSv4 Kernel-server requires 2x manual restart after reboot? Maybe a timing problem?

[Ben DJ]

I've installed NFSv4 as nfs-kernel-server.  I'm slowly figuring out
how to get it to configured and working.

After a system reboot, if I try to mount an export,

	mount -t nfs4 192.168.1.2:/test /mnt/test

it just "sits there", never mounting.

So I 1st check what's running,

	ps ax | egrep -i "idmapd|mountd|statd|nfsd|sm-notify"
	 3731 ?        Ss     0:00 /usr/sbin/rpc.idmapd
	 3732 ?        Ss     0:00 /usr/sbin/rpc.idmapd
	 3753 ?        Ss     0:00 /usr/sbin/rpc.statd --no-notify
	 3758 ?        S<     0:00 [nfsd4]
	 3759 ?        S<     0:00 [nfsd]
	 3760 ?        S<     0:00 [nfsd]
	 3761 ?        S<     0:00 [nfsd]
	 3762 ?        S<     0:00 [nfsd]
	 3772 ?        Ss     0:00 /usr/sbin/rpc.mountd

and that looks OK.

After some experiments, I've discovered if I CTRL-C the mount attempt
and restart the server TWICE,

	service nfsserver restart
		Shutting down kernel based NFS server: nfsd statd mountd
                                                done
		Starting kernel based NFS server: idmapdexportfs: Warning: /nfs/test
does not support NFS export.
		 mountd statd nfsd sm-notify
                                                done
	
	service nfsserver restart
		Shutting down kernel based NFS server: nfsd statd mountd
                                                done
		Starting kernel based NFS server: idmapd mountd statd nfsd sm-notify
                                                done

Notice that the 2nd time the "idmapd" appears to be re-started.

then repeat the mount attempt,

	mount -t nfs4 192.168.1.2:/test /mnt/test

it now works.

Checking what's running again,

	ps ax | egrep -i "idmapd|mountd|statd|nfsd|sm-notify"
	
	 3731 ?        Ss     0:00 /usr/sbin/rpc.idmapd
	 3732 ?        Ss     0:00 /usr/sbin/rpc.idmapd
	 5213 ?        S<s    0:00 /usr/sbin/rpc.mountd
	 5216 ?        S<s    0:00 /usr/sbin/rpc.statd --no-notify
	 5221 ?        S<     0:00 [nfsd4]
	 5222 ?        S<     0:00 [nfsd]
	 5223 ?        S<     0:00 [nfsd]
	 5224 ?        S<     0:00 [nfsd]
	 5225 ?        S<     0:00 [nfsd]

It all looks the same as before.  Notice that the rpc.idmapd PIDs are
the same, even though the "service nfsserver restart" 2nd time around
looked like it restarted it.

I don't really understand what the 2X nfsserver restart does.  There
clearly IS an effect. And it looks like it has to do with idmapd.

I can say that

	service nfsserver stop
	service nfs stop
	service quotad stop
	service rpcbind stop
	umount -a
	mount  -a
	service rpcbind start
	service quotad start
	service nfs start
	service nfsserver start

often gives me similar errors,

	Warning: /nfs/test does not support NFS export

that go away if I make sure there's a delay after starting nfs,

	service nfs start && sleep 5
	service nfsserver start

Maybe there's some sort of timing problem on startup?

Any clues?

BenDJ

Re: NFSv4 Kernel-server requires 2x manual restart after reboot? Maybe a timing problem?

[J. Bruce Fields]

On Fri, Oct 23, 2009 at 10:07:23AM -0700, Ben DJ wrote:
> I've installed NFSv4 as nfs-kernel-server.

What distro and versions are you using?

> I'm slowly figuring out
> how to get it to configured and working.
> 
> After a system reboot, if I try to mount an export,
> 
> 	mount -t nfs4 192.168.1.2:/test /mnt/test
> 
> it just "sits there", never mounting.
> 
> So I 1st check what's running,
> 
> 	ps ax | egrep -i "idmapd|mountd|statd|nfsd|sm-notify"
> 	 3731 ?        Ss     0:00 /usr/sbin/rpc.idmapd
> 	 3732 ?        Ss     0:00 /usr/sbin/rpc.idmapd
> 	 3753 ?        Ss     0:00 /usr/sbin/rpc.statd --no-notify
> 	 3758 ?        S<     0:00 [nfsd4]
> 	 3759 ?        S<     0:00 [nfsd]
> 	 3760 ?        S<     0:00 [nfsd]
> 	 3761 ?        S<     0:00 [nfsd]
> 	 3762 ?        S<     0:00 [nfsd]
> 	 3772 ?        Ss     0:00 /usr/sbin/rpc.mountd
> 
> and that looks OK.
> 
> After some experiments, I've discovered if I CTRL-C the mount attempt
> and restart the server TWICE,
> 
> 	service nfsserver restart
> 		Shutting down kernel based NFS server: nfsd statd mountd
>                                                 done
> 		Starting kernel based NFS server: idmapdexportfs: Warning: /nfs/test
> does not support NFS export.

Huh.  What filesystem is at /nfs/test?

--b.

> 		 mountd statd nfsd sm-notify
>                                                 done
> 	
> 	service nfsserver restart
> 		Shutting down kernel based NFS server: nfsd statd mountd
>                                                 done
> 		Starting kernel based NFS server: idmapd mountd statd nfsd sm-notify
>                                                 done
> 
> Notice that the 2nd time the "idmapd" appears to be re-started.
> 
> then repeat the mount attempt,
> 
> 	mount -t nfs4 192.168.1.2:/test /mnt/test
> 
> it now works.
> 
> Checking what's running again,
> 
> 	ps ax | egrep -i "idmapd|mountd|statd|nfsd|sm-notify"
> 	
> 	 3731 ?        Ss     0:00 /usr/sbin/rpc.idmapd
> 	 3732 ?        Ss     0:00 /usr/sbin/rpc.idmapd
> 	 5213 ?        S<s    0:00 /usr/sbin/rpc.mountd
> 	 5216 ?        S<s    0:00 /usr/sbin/rpc.statd --no-notify
> 	 5221 ?        S<     0:00 [nfsd4]
> 	 5222 ?        S<     0:00 [nfsd]
> 	 5223 ?        S<     0:00 [nfsd]
> 	 5224 ?        S<     0:00 [nfsd]
> 	 5225 ?        S<     0:00 [nfsd]
> 
> It all looks the same as before.  Notice that the rpc.idmapd PIDs are
> the same, even though the "service nfsserver restart" 2nd time around
> looked like it restarted it.
> 
> I don't really understand what the 2X nfsserver restart does.  There
> clearly IS an effect. And it looks like it has to do with idmapd.
> 
> I can say that
> 
> 	service nfsserver stop
> 	service nfs stop
> 	service quotad stop
> 	service rpcbind stop
> 	umount -a
> 	mount  -a
> 	service rpcbind start
> 	service quotad start
> 	service nfs start
> 	service nfsserver start
> 
> often gives me similar errors,
> 
> 	Warning: /nfs/test does not support NFS export
> 
> that go away if I make sure there's a delay after starting nfs,
> 
> 	service nfs start && sleep 5
> 	service nfsserver start
> 
> Maybe there's some sort of timing problem on startup?
> 
> Any clues?
> 
> BenDJ
> --
> To unsubscribe from this list: send the line "unsubscribe linux-nfs" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: NFSv4 Kernel-server requires 2x manual restart after reboot? Maybe a timing problem?

[Ben DJ]

Hi Bruce,

On Fri, Oct 23, 2009 at 10:27 AM, J. Bruce Fields <bfields@fieldses.org=
> wrote:
> On Fri, Oct 23, 2009 at 10:07:23AM -0700, Ben DJ wrote:
>> I've installed NFSv4 as nfs-kernel-server.
>
> What distro and versions are you using?

I'm currently working on Opensuse 11.2 RC1.

At the moment,

kernel-default-base-2.6.31.3-1.1.x86_64
kernel-default-2.6.31.3-1.1.x86_64
kernel-default-devel-2.6.31.3-1.1.x86_64

quota-nfs-3.17-3.1.x86_64
nfsidmap-0.21-3.14.x86_64
nfs-doc-1.1.3-20.1.x86_64
nfs-client-1.1.3-20.1.x86_64
nfs-kernel-server-1.1.3-20.1.x86_64


> Huh. =A0What filesystem is at /nfs/test?

I'm not entirely sure what you're asking for.

/nfs/test is on my "/" volume, which is ext3 (someday, ext4 -- but not =
yet).

Is that what you mean?

Also I found that switching from mounts in /etc/fstab to using autofs
gets rid of the mount-after-boot issues

cat /etc/auto.master
	/test /etc/auto.misc
cat /etc/auto.misc
	cd -fstype=3Diso9660,ro,nosuid,nodev :/dev/cdrom
	nfs4_test -fstype=3Dnfs4,rw 192.168.1.2:/
chkconfig autofs on
reboot

But now, immediately after reboot,

ls /test/nfs4_test/
	bin/  boot/  dev/  etc/  home/  lib/  lib64/  lost+found/  media/
mnt/  mnt.auto/  nfs/  opt/  proc/  root/  sbin/  selinux/  srv/  sys/
 tmp/  usr/  var/

which is mounted, and feels like progress.

But the mount is of 192.168.1.2's "/" root.  I expected the NFSv4
exported-root @ /nfs, instead.

Thanks for your time.

BenDJ



Thanks.

BedDJ

Re: NFSv4 Kernel-server requires 2x manual restart after reboot? Maybe a timing problem?

[J. Bruce Fields]

On Fri, Oct 23, 2009 at 10:57:57AM -0700, Ben DJ wrote:
> Hi Bruce,
>=20
> On Fri, Oct 23, 2009 at 10:27 AM, J. Bruce Fields <bfields@fieldses.o=
rg> wrote:
> > On Fri, Oct 23, 2009 at 10:07:23AM -0700, Ben DJ wrote:
> >> I've installed NFSv4 as nfs-kernel-server.
> >
> > What distro and versions are you using?
>=20
> I'm currently working on Opensuse 11.2 RC1.
>=20
> At the moment,
>=20
> kernel-default-base-2.6.31.3-1.1.x86_64
> kernel-default-2.6.31.3-1.1.x86_64
> kernel-default-devel-2.6.31.3-1.1.x86_64
>=20
> quota-nfs-3.17-3.1.x86_64
> nfsidmap-0.21-3.14.x86_64
> nfs-doc-1.1.3-20.1.x86_64
> nfs-client-1.1.3-20.1.x86_64
> nfs-kernel-server-1.1.3-20.1.x86_64
>=20
>=20
> > Huh. =C2=A0What filesystem is at /nfs/test?
>=20
> I'm not entirely sure what you're asking for.
>=20
> /nfs/test is on my "/" volume, which is ext3 (someday, ext4 -- but no=
t yet).
>=20
> Is that what you mean?

Right.  Actually I should have asked what the output of the "mount" and
"exportfs -v" commands were on the server.

--b.

> Also I found that switching from mounts in /etc/fstab to using autofs
> gets rid of the mount-after-boot issues
>=20
> cat /etc/auto.master
> 	/test /etc/auto.misc
> cat /etc/auto.misc
> 	cd -fstype=3Diso9660,ro,nosuid,nodev :/dev/cdrom
> 	nfs4_test -fstype=3Dnfs4,rw 192.168.1.2:/
> chkconfig autofs on
> reboot
>=20
> But now, immediately after reboot,
>=20
> ls /test/nfs4_test/
> 	bin/  boot/  dev/  etc/  home/  lib/  lib64/  lost+found/  media/
> mnt/  mnt.auto/  nfs/  opt/  proc/  root/  sbin/  selinux/  srv/  sys=
/
>  tmp/  usr/  var/
>=20
> which is mounted, and feels like progress.
>=20
> But the mount is of 192.168.1.2's "/" root.  I expected the NFSv4
> exported-root @ /nfs, instead.
>=20
> Thanks for your time.
>=20
> BenDJ
>=20
>=20
>=20
> Thanks.
>=20
> BedDJ

Re: NFSv4 Kernel-server requires 2x manual restart after reboot? Maybe a timing problem?

[Ben DJ]

Hi Bruce,

On Fri, Oct 23, 2009 at 11:24 AM, J. Bruce Fields <bfields@fieldses.org> wrote:

I'm getting closer to "works". At least the way I think it should :-/.

With,

cat /etc/auto.master
	/mnt.dev /etc/auto.device
	/mnt.nfs /etc/auto.nfsv4

cat /etc/auto.device
	cd          -fstype=iso9660,ro,nosuid,nodev :/dev/cdrom

cat /etc/auto.nfsv4
	test -fstype=nfs4,rw server.bendj.loc:/

After reboot,

mount
	/dev/dm-1 on / type ext3 (rw,acl,user_xattr)
	proc on /proc type proc (rw)
	sysfs on /sys type sysfs (rw)
	debugfs on /sys/kernel/debug type debugfs (rw)
	udev on /dev type tmpfs (rw)
	devpts on /dev/pts type devpts (rw,mode=0620,gid=5)
	/dev/md0 on /boot type ext3 (rw,acl,user_xattr)
	/dev/dm-0 on /home type ext3 (rw,acl,user_xattr)
	fusectl on /sys/fs/fuse/connections type fusectl (rw)
	nfsd on /proc/fs/nfsd type nfsd (rw)
	rpc_pipefs on /var/lib/nfs/rpc_pipefs type rpc_pipefs (rw)

exportfs -v
	/nfs 192.168.1.0/24(ro,wdelay,root_squash,no_subtree_check,fsid=0)

and the mounts ARE automounted,

ls /mnt.nfs/test /mnt.dev
	/mnt.dev:
		cd/

	/mnt.nfs/test:
		bin/  boot/  dev/  etc/  home/  lib/  lib64/  lost+found/  media/
mnt/  mnt.auto/  mnt.dev/  mnt.nfs/  nfs/  opt/  proc/  root/  sbin/
selinux/  srv/  sys/  tmp/  usr/  var/

Looks like autofs mounting rahter than /etc/fstab is the way-to-go.  I
don't know if this means there IS a timing problem with the /etc/fstab
way of doing things.

Also, the /mnt.nfs/test still shows the actual SERVER root, rather
than the specified NFS (ch)root.  That shouldn't be happening, right?

And just to check,

	service nfsserver restart
		Shutting down kernel based NFS server: nfsd statd mountd


                       done
		Starting kernel based NFS server: idmapd mountd statd nfsd sm-notify


                       done

works the 1st time around.

BenDJ

Re: NFSv4 Kernel-server requires 2x manual restart after reboot? Maybe a timing problem?

[Suresh Jayaraman]

On 10/23/2009 10:37 PM, Ben DJ wrote:
> I've installed NFSv4 as nfs-kernel-server.  I'm slowly figuring out
> how to get it to configured and working.
> 
> After a system reboot, if I try to mount an export,
> 
> 	mount -t nfs4 192.168.1.2:/test /mnt/test

Shouldn't it be 192.168.1.2:/ (excluding share name in case of NFSv4) or
has it changed?

Are you able to mount successfully if you just use the server name
like below:
	mount -t nfs4 192.168.1.2:/ /mnt/test (in your case)

> it just "sits there", never mounting.

I'm also running a 11.2 RC1 and tried to mount nfs4 the way you
mentioned, but I don't see the hang you're observing:

mount -t nfs4 192.168.100.254:/export  /mnt/nfs/
mount.nfs4: mounting 192.168.100.254:/export failed, reason given by
server:  No such file or directory

Can you try to run mount with '-v' and post the output?

> So I 1st check what's running,
> 
> 	ps ax | egrep -i "idmapd|mountd|statd|nfsd|sm-notify"
> 	 3731 ?        Ss     0:00 /usr/sbin/rpc.idmapd
> 	 3732 ?        Ss     0:00 /usr/sbin/rpc.idmapd

I'm not sure why you have two instances of idmapd.. Did you start idmapd
once again?

> 
> After some experiments, I've discovered if I CTRL-C the mount attempt
> and restart the server TWICE,

This is odd.

> 	service nfsserver restart
> 		Shutting down kernel based NFS server: nfsd statd mountd
>                                                 done
> 		Starting kernel based NFS server: idmapdexportfs: Warning: /nfs/test
> does not support NFS export.
> 		 mountd statd nfsd sm-notify
>                                                 done
> 	
> 	service nfsserver restart
> 		Shutting down kernel based NFS server: nfsd statd mountd
>                                                 done
> 		Starting kernel based NFS server: idmapd mountd statd nfsd sm-notify
>                                                 done
> 
> Notice that the 2nd time the "idmapd" appears to be re-started.
> 
> then repeat the mount attempt,
> 
> 	mount -t nfs4 192.168.1.2:/test /mnt/test
> 
> it now works.
> 
> 
> I don't really understand what the 2X nfsserver restart does.  There
> clearly IS an effect. And it looks like it has to do with idmapd.

Have you enabled nfsserver during startup (chkconfig nfsserver on)?

> I can say that
> 
> 	service nfsserver stop
> 	service nfs stop
> 	service quotad stop
> 	service rpcbind stop
> 	umount -a
> 	mount  -a
> 	service rpcbind start
> 	service quotad start
> 	service nfs start
> 	service nfsserver start
> 
> often gives me similar errors,
> 
> 	Warning: /nfs/test does not support NFS export
> 
> that go away if I make sure there's a delay after starting nfs,
> 
> 	service nfs start && sleep 5
> 	service nfsserver start
> 
> Maybe there's some sort of timing problem on startup?
> 
> Any clues?
> 

not much at the moment..

Thanks,


-- 
Suresh Jayaraman