Mounting NFS root FS

Mounting NFS root FS

[William Estrada]

Hi guys,

  I have been trying to make FC5's kernel do a boot with an NFS root file system.  I see
the support is in the kernel(?).  I have tried this:

> [root@Server ~]# cat /tftpboot/pxelinux.cfg/0A0101
> SERIAL 0 9600
> Say
> 
> SAY Hello
> 
> SAY Trying NFS
> SAY ramdisk_size=10000 debug ip=dhcp initrd=NFS/initrd.gz lang=us apm=power-off console=ttyS0,9600 console=tty0 quiet root=/dev/nfs nfsroot=10.1.1.12:/tftpboot/NFS/Root_FS init=/bin/bash
> 
> Default NFS/vmlinuz
> 
> append  ramdisk_size=10000 debug ip=10.1.1.50 initrd=NFS/initrd.gz lang=us apm=power-off console=ttyS0,9600 console=tty0 quiet  root=/dev/nfs nfsroot=10.1.1.12:/tftpboot/NFS/Root_FS init=/bin/bash

  I get "mount: could not find file system: '/dev/root'" and then a kernel panic.

  I am using the FC5 kernel and the FC5 initrd.gz as you can see above.  

> [root@Server ~]# ls -lrt /tftpboot/NFS/
> total 2636
> drwxr-xr-x  4 root root    4096 Nov 26 18:41 SAVE
> -rwxr-xr-x  1 root root  932077 Nov 26 18:51 initrd.gz
> drwxr-xr-x 13 root root    4096 Nov 26 19:18 Root_FS
> -rwxr-xr-x  1 root root 1732515 Nov 26 19:37 vmlinuz

  I tried to build a new kernel many times, but that process failed.

  Am I missing something?  Do I need to change linuxrc?  Does someone have a simple example
of how to do an NFS Root FS?

  Would appreciate any points.

-- 
  Thanks for your time.

William Estrada

Email      : MrUmunhum at popdial dot com
Resume     : www.Mt-Umunhum-Wireless.net/resume/william_estrada.html
HTTP       : www.Mt-Umunhum-Wireless.net

Re: Mounting NFS root FS

[Jan Engelhardt]

>
> I have been trying to make FC5's kernel do a boot with an NFS root file
> system.  I see
> the support is in the kernel(?).  I have tried this:
>
>> root=/dev/nfs nfsroot=10.1.1.12:/tftpboot/NFS/Root_FS

This feature is almost deprecated. One is supposed to use initramfs,
/sbin/ip or some DHCP client, and a mount program nowadays.


	-`J'
-- 

Re: Mounting NFS root FS

[Willy Tarreau]

On Sat, Dec 02, 2006 at 08:07:45PM +0100, Jan Engelhardt wrote:
> >
> > I have been trying to make FC5's kernel do a boot with an NFS root file
> > system.  I see
> > the support is in the kernel(?).  I have tried this:
> >
> >> root=/dev/nfs nfsroot=10.1.1.12:/tftpboot/NFS/Root_FS
> 
> This feature is almost deprecated. One is supposed to use initramfs,
> /sbin/ip or some DHCP client, and a mount program nowadays.

But I think that there are plenty of light terminals still booting like
this which will not be able to upgrade anymore then. Even right here,
my web server (parisc) boot from the network that way. At least an
initramfs would need to be able to cope with the same syntax, otherwise
we might break some systems where it's not even easy to change those
info in the boot loader.

Willy

Re: Mounting NFS root FS

[Jan Engelhardt]

>> > I have been trying to make FC5's kernel do a boot with an NFS root file
>> > system.  I see
>> > the support is in the kernel(?).  I have tried this:
>> >
>> >> root=/dev/nfs nfsroot=10.1.1.12:/tftpboot/NFS/Root_FS
>> 
>> This feature is almost deprecated. One is supposed to use initramfs,
>> /sbin/ip or some DHCP client, and a mount program nowadays.
>
>But I think that there are plenty of light terminals still booting like
>this which will not be able to upgrade anymore then. Even right here,
>my web server (parisc) boot from the network that way. At least an
>initramfs would need to be able to cope with the same syntax,

No problem:

<<</init<<<
#!/bin/bash

for o in `cat /proc/cmdline`; do
    case "$o" in
        nfsroot=*)
            arg="${o##nfsroot=}";
            ;;
    esac;
done;

### further parse $arg
>>>

simplified example of how this can be accomplished. This is why
initrds/initramfs are so much more powerful than in-kernel software.


	-`J'
-- 

Re: Mounting NFS root FS

[Willy Tarreau]

On Sat, Dec 02, 2006 at 10:56:38PM +0100, Jan Engelhardt wrote:
> 
> >> > I have been trying to make FC5's kernel do a boot with an NFS root file
> >> > system.  I see
> >> > the support is in the kernel(?).  I have tried this:
> >> >
> >> >> root=/dev/nfs nfsroot=10.1.1.12:/tftpboot/NFS/Root_FS
> >> 
> >> This feature is almost deprecated. One is supposed to use initramfs,
> >> /sbin/ip or some DHCP client, and a mount program nowadays.
> >
> >But I think that there are plenty of light terminals still booting like
> >this which will not be able to upgrade anymore then. Even right here,
> >my web server (parisc) boot from the network that way. At least an
> >initramfs would need to be able to cope with the same syntax,
> 
> No problem:
> 
> <<</init<<<
> #!/bin/bash
> 
> for o in `cat /proc/cmdline`; do
>     case "$o" in
>         nfsroot=*)
>             arg="${o##nfsroot=}";
>             ;;
>     esac;
> done;
> 
> ### further parse $arg
> >>>
> 
> simplified example of how this can be accomplished. This is why
> initrds/initramfs are so much more powerful than in-kernel software.

I'm not saying initramfs is not powerful, and indeed your example is
the common way of parsing cmdline for me too. What I'm saying is that
before nfsroot stops being supported, we'll need a working replacement
(and not "### further parse $arg"), if possible within the kernel tree
so that people who used to build kernels to boot such machines will
still be able to build kernels for them, even if this implies using
an initramfs with some tools in it.

The real danger of removing support for in-kernel features like this
is to leave people with no solution at all (because they don't know
how to proceed), and their workarounds are often worse than the
problem that we tried to fix in the first place.

Willy

Re: Mounting NFS root FS

[Trond Myklebust]

On Sat, 2006-12-02 at 23:55 +0100, Willy Tarreau wrote:
> I'm not saying initramfs is not powerful, and indeed your example is
> the common way of parsing cmdline for me too. What I'm saying is that
> before nfsroot stops being supported, we'll need a working replacement
> (and not "### further parse $arg"), if possible within the kernel tree
> so that people who used to build kernels to boot such machines will
> still be able to build kernels for them, even if this implies using
> an initramfs with some tools in it.
> 
> The real danger of removing support for in-kernel features like this
> is to leave people with no solution at all (because they don't know
> how to proceed), and their workarounds are often worse than the
> problem that we tried to fix in the first place.

It hasn't been removed yet. However most distributions choose not to
enable it because that would force them to compile the NFS client into
the main kernel instead of leaving it as a module.

As for the initramfs support, hpa has assured me that his klibc
distribution already has a full solution for NFS mounting on current
kernels.

Trond

Re: Mounting NFS root FS

[Willy Tarreau]

On Sat, Dec 02, 2006 at 09:37:18PM -0500, Trond Myklebust wrote:
> On Sat, 2006-12-02 at 23:55 +0100, Willy Tarreau wrote:
> > I'm not saying initramfs is not powerful, and indeed your example is
> > the common way of parsing cmdline for me too. What I'm saying is that
> > before nfsroot stops being supported, we'll need a working replacement
> > (and not "### further parse $arg"), if possible within the kernel tree
> > so that people who used to build kernels to boot such machines will
> > still be able to build kernels for them, even if this implies using
> > an initramfs with some tools in it.
> > 
> > The real danger of removing support for in-kernel features like this
> > is to leave people with no solution at all (because they don't know
> > how to proceed), and their workarounds are often worse than the
> > problem that we tried to fix in the first place.
> 
> It hasn't been removed yet. However most distributions choose not to
> enable it because that would force them to compile the NFS client into
> the main kernel instead of leaving it as a module.

That's a valid point, but in fact, building with NFS client and serial
port support in the kernel on some archs is as common as building with
IDE driver and VGA console in the kernel on x86. With some architectures
used in light networked workstations, it's very common to boot from the
network (sparc & parisc come to mind, sorry to those I forgot), hence
this common practise.

> As for the initramfs support, hpa has assured me that his klibc
> distribution already has a full solution for NFS mounting on current
> kernels.

That's again where we see the limits of this ever-developping 2.6.
I'm not saying that doing this from initramfs+tools is a bad solution,
since it solves lots of problems, it's just that it is *much* different
from what was previously done.

People who have installed a distro on their machines will not be
able to upgrade their kernel past a certain point by hand. Upgrading
distro packages in such environments is generally not always an
option (particularly boot packages such as boot loader and kernel),
because the boot server is not necessarily running on the same
OS/distro, and sometimes the kernel needs different build options.

Then the remaining solution to get stability and security fixes
is often to [cross-]compile a more recent kernel, and to put it
on the boot server. Fortunately Adrian maintains 2.6.16 :-/

> Trond

Regards,
Willy

Re: Mounting NFS root FS

[Trond Myklebust]

On Sun, 2006-12-03 at 07:02 +0100, Willy Tarreau wrote:

> That's a valid point, but in fact, building with NFS client and serial
> port support in the kernel on some archs is as common as building with
> IDE driver and VGA console in the kernel on x86. With some architectures
> used in light networked workstations, it's very common to boot from the
> network (sparc & parisc come to mind, sorry to those I forgot), hence
> this common practise.

I have no influence over the distributions' choice of kernel compiler
options. The fact is, though, that few of them support nfsroot out of
the box. AFAICS FC-6 is one of those that appears not to.

> > As for the initramfs support, hpa has assured me that his klibc
> > distribution already has a full solution for NFS mounting on current
> > kernels.
> 
> That's again where we see the limits of this ever-developping 2.6.
> I'm not saying that doing this from initramfs+tools is a bad solution,
> since it solves lots of problems, it's just that it is *much* different
> from what was previously done.
> 
> People who have installed a distro on their machines will not be
> able to upgrade their kernel past a certain point by hand. Upgrading
> distro packages in such environments is generally not always an
> option (particularly boot packages such as boot loader and kernel),
> because the boot server is not necessarily running on the same
> OS/distro, and sometimes the kernel needs different build options.

Most people that run nfsroot systems do so because that makes
provisioning of new machines easy: if you only have one system image,
then upgrading it is less of a challenge.

> Then the remaining solution to get stability and security fixes
> is often to [cross-]compile a more recent kernel, and to put it
> on the boot server. Fortunately Adrian maintains 2.6.16 :-/

No comment.

Trond

Re: Mounting NFS root FS

[Willy Tarreau]

On Sun, Dec 03, 2006 at 02:05:10AM -0500, Trond Myklebust wrote:
> On Sun, 2006-12-03 at 07:02 +0100, Willy Tarreau wrote:
> 
> > That's a valid point, but in fact, building with NFS client and serial
> > port support in the kernel on some archs is as common as building with
> > IDE driver and VGA console in the kernel on x86. With some architectures
> > used in light networked workstations, it's very common to boot from the
> > network (sparc & parisc come to mind, sorry to those I forgot), hence
> > this common practise.
> 
> I have no influence over the distributions' choice of kernel compiler
> options. The fact is, though, that few of them support nfsroot out of
> the box. AFAICS FC-6 is one of those that appears not to.
> 
> > > As for the initramfs support, hpa has assured me that his klibc
> > > distribution already has a full solution for NFS mounting on current
> > > kernels.
> > 
> > That's again where we see the limits of this ever-developping 2.6.
> > I'm not saying that doing this from initramfs+tools is a bad solution,
> > since it solves lots of problems, it's just that it is *much* different
> > from what was previously done.
> > 
> > People who have installed a distro on their machines will not be
> > able to upgrade their kernel past a certain point by hand. Upgrading
> > distro packages in such environments is generally not always an
> > option (particularly boot packages such as boot loader and kernel),
> > because the boot server is not necessarily running on the same
> > OS/distro, and sometimes the kernel needs different build options.
> 
> Most people that run nfsroot systems do so because that makes
> provisioning of new machines easy: if you only have one system image,
> then upgrading it is less of a challenge.

It's one use, but another one is for diskless terminals, often built
from old systems. In this case, it's to avoid the cost, noise, power
consumption and failures associated to disks. It's quite often done
one radically different archs/OS between the server and the clients,
making the upgrade more complicated.

Willy

Re: Mounting NFS root FS

[Jan Engelhardt]

>It's one use, but another one is for diskless terminals, often built
>from old systems. In this case, it's to avoid the cost, noise, power
>consumption and failures associated to disks. It's quite often done
>one radically different archs/OS between the server and the clients,
>making the upgrade more complicated.

unionfs is becoming popular, and it's one of those things that can't
do without initramfs at all, for example.

>> I have no influence over the distributions' choice of kernel compiler
>> options. The fact is, though, that few of them support nfsroot out of
>> the box. AFAICS FC-6 is one of those that appears not to.

(File a bug report, heh.)

So what, noone supports unionfs OOTB, which leaves me with what options?
Right, hacking it up myself by modifying the initramfs scripts my
distro's mkinitrd gave me.


	-`J'
-- 

Re: Mounting NFS root FS

[Trond Myklebust]

On Sun, 2006-12-03 at 09:30 +0100, Willy Tarreau wrote:
> It's one use, but another one is for diskless terminals, often built
> from old systems. In this case, it's to avoid the cost, noise, power
> consumption and failures associated to disks. It's quite often done
> one radically different archs/OS between the server and the clients,
> making the upgrade more complicated.

It is naive to believe that the only thing you need to keep up to date
is the kernel itself: if you are at all worried about security, then
upgrades are a fact of life. Depending on your choice of distribution,
then you can make that process easy or difficult.

As for the kernel, nobody has promised you that we would keep all
possible implementations of a given feature around forever. As and when
we develop better ways of implementing a set of features, we may want to
remove the old ways. You will be given advance notice in
Documentation/feature-removal-schedule.txt, and it will be your choice
whether or not you want to follow the upgrade path, or stick with your
existing setup.
What we will not do is to maintain a bunch of parallel kernel trees or
unduly bloat the kernel forever in order to support old systems: that is
the job of those distributions that promise you n years legacy support.

  Trond

Re: Mounting NFS root FS

[Janne Karhunen]

On 12/2/06, William Estrada <MrUmunhum@popdial.com> wrote:
> Hi guys,
>
>   I have been trying to make FC5's kernel do a boot
> with an NFS root file system.  I see the support is in the
> kernel(?).

Is this really properly possible (with read/write access and
locking in place)? AFAIK NFS client lock state data seems
to require persistent storage .. ?


-- 
// Janne

Re: Mounting NFS root FS

[Trond Myklebust]

On Mon, 2006-12-04 at 13:51 +0200, Janne Karhunen wrote:
> On 12/2/06, William Estrada <MrUmunhum@popdial.com> wrote:
> > Hi guys,
> >
> >   I have been trying to make FC5's kernel do a boot
> > with an NFS root file system.  I see the support is in the
> > kernel(?).
> 
> Is this really properly possible (with read/write access and
> locking in place)? AFAIK NFS client lock state data seems
> to require persistent storage .. ?

1) Yes, but not on the root partition (unless you use an initrd to start
rpc.statd before mounting the NFS partition).

2) NFS provides persistent storage.

Trond

Re: Mounting NFS root FS

[Janne Karhunen]

On Monday 04 December 2006 17:29, Trond Myklebust wrote:

> > >   I have been trying to make FC5's kernel do a boot
> > > with an NFS root file system.  I see the support is in the
> > > kernel(?).
> >
> > Is this really properly possible (with read/write access and
> > locking in place)? AFAIK NFS client lock state data seems
> > to require persistent storage .. ?
>
> 1) Yes, but not on the root partition (unless you use an initrd to start
> rpc.statd before mounting the NFS partition).

Ok. 


> 2) NFS provides persistent storage.

To me this sounds like a chicken and an egg problem. It 
both depends and provides this at the same time :/. But 
hey, if it's supposed to work then OK.

Anyhoo, I tried this at some stage and failed as random
clients seemed to occasionally get stuck in insmod¹ at
boot (infinite wait on lock that never gets released). 
At that stage guess was that server could not properly 
recognize client reboot given stale client lock data.
But if it's supposed to work I guess I have to give it 
another shot and do better analysis on it.

What about NLM/NSM protocol issues - do they properly 
deal with packet loss and clients that stay down (client 
holding a lock crashing and staying down; will the lock 
ever be released)?

¹ And why does insmod require a lock on module at load??


-- 
// Janne

Re: Mounting NFS root FS

[Trond Myklebust]

On Mon, 2006-12-04 at 19:12 +0200, Janne Karhunen wrote:
> > 2) NFS provides persistent storage.
> 
> To me this sounds like a chicken and an egg problem. It 
> both depends and provides this at the same time :/. But 
> hey, if it's supposed to work then OK.

??? Locking depends on persistent storage, but persistent storage never
depended on locking. Neither rpc.statd nor lockd, nor the nfs client
depend on locking working a priori.

> Anyhoo, I tried this at some stage and failed as random
> clients seemed to occasionally get stuck in insmod¹ at
> boot (infinite wait on lock that never gets released). 
> At that stage guess was that server could not properly 
> recognize client reboot given stale client lock data.
> But if it's supposed to work I guess I have to give it 
> another shot and do better analysis on it.
> 
> What about NLM/NSM protocol issues - do they properly 
> deal with packet loss and clients that stay down (client 
> holding a lock crashing and staying down; will the lock 
> ever be released)?

1) Packet loss is dealt with by retrying ad-infinitum.

2) No. The problem of client crashes was fixed in NFSv4 with the
addition of lease-based locks.

> ¹ And why does insmod require a lock on module at load??

Does it? I've no idea why it should need that.

Trond

Re: Mounting NFS root FS

[Janne Karhunen]

On Monday 04 December 2006 20:21, Trond Myklebust wrote:

> > > 2) NFS provides persistent storage.
> >
> > To me this sounds like a chicken and an egg problem. It
> > both depends and provides this at the same time :/. But
> > hey, if it's supposed to work then OK.
>
> ??? Locking depends on persistent storage, but persistent storage never
> depended on locking.

Except for the fact that to be able to mount anything RW you
generally _want_ to have locks. And can't have locks without 
the mount. Not that it wouldn't work, it's just that I would
not do it [for obvious reasons].


> 2) No. The problem of client crashes was fixed in NFSv4 with the
> addition of lease-based locks.

This was NFSv3 system, so that would still be an issue.


-- 
// Janne

Re: Mounting NFS root FS

[Trond Myklebust]

On Mon, 2006-12-04 at 22:05 +0200, Janne Karhunen wrote:
> On Monday 04 December 2006 20:21, Trond Myklebust wrote:
> 
> > > > 2) NFS provides persistent storage.
> > >
> > > To me this sounds like a chicken and an egg problem. It
> > > both depends and provides this at the same time :/. But
> > > hey, if it's supposed to work then OK.
> >
> > ??? Locking depends on persistent storage, but persistent storage never
> > depended on locking.
> 
> Except for the fact that to be able to mount anything RW you
> generally _want_ to have locks. And can't have locks without 
> the mount. Not that it wouldn't work, it's just that I would
> not do it [for obvious reasons].

You just need to be careful to set it up correctly in the initrd: either
make sure that you mount the root partition as 'nolock' or else make
sure that you mount /var/lib/nfs, and start rpc.statd before you start
init and any other applications that might need locking.

  Trond

Re: Mounting NFS root FS

[H. Peter Anvin]

Trond Myklebust wrote:
> On Mon, 2006-12-04 at 22:05 +0200, Janne Karhunen wrote:
>> On Monday 04 December 2006 20:21, Trond Myklebust wrote:
>>
>>>>> 2) NFS provides persistent storage.
>>>> To me this sounds like a chicken and an egg problem. It
>>>> both depends and provides this at the same time :/. But
>>>> hey, if it's supposed to work then OK.
>>> ??? Locking depends on persistent storage, but persistent storage never
>>> depended on locking.
>> Except for the fact that to be able to mount anything RW you
>> generally _want_ to have locks. And can't have locks without 
>> the mount. Not that it wouldn't work, it's just that I would
>> not do it [for obvious reasons].
> 
> You just need to be careful to set it up correctly in the initrd: either
> make sure that you mount the root partition as 'nolock' or else make
> sure that you mount /var/lib/nfs, and start rpc.statd before you start
> init and any other applications that might need locking.
> 

The nfsmount which is in the klibc distribution supports running an ad 
hoc portmap daemon, which allows locking to be done by forwarding 
information to the "real" portmap for when the real rpc.statd is run.

	-hpa

Re: Mounting NFS root FS

[Jan Engelhardt]

>> 2) NFS provides persistent storage.
>
>To me this sounds like a chicken and an egg problem. It 
>both depends and provides this at the same time :/. But 
>hey, if it's supposed to work then OK.

Way 1:

mount -nt tmpfs none /var/lib/nfs;
mount -nt nfs fserve:/tftpboot/linux /mnt;
mount -n --move /var/lib/nfs /mnt/var/lib/nfs/;
./run_init -c /mnt /sbin/init; # or similar

And you can also start locking after pivot_rooting to /mnt, that would 
not even require (/mnt)/var/lib/nfs to be a separate mount.


Ok, did I miss it all?


	-`J'
-- 

Re: Mounting NFS root FS

[Janne Karhunen]

On Monday 04 December 2006 22:03, Jan Engelhardt wrote:

> >> 2) NFS provides persistent storage.
> >
> >To me this sounds like a chicken and an egg problem. It
> >both depends and provides this at the same time :/. But
> >hey, if it's supposed to work then OK.
>
> Way 1:
>
> mount -nt tmpfs none /var/lib/nfs;
> mount -nt nfs fserve:/tftpboot/linux /mnt;
> mount -n --move /var/lib/nfs /mnt/var/lib/nfs/;
> ./run_init -c /mnt /sbin/init; # or similar

Statd should probably be started before nfs mount to get it 
right. But doesn't statd require state data ( some sort of 
generation number ) from persistent storage to work? This 
would start with a blank slate.


-- 
// Janne

Re: Mounting NFS root FS

[Trond Myklebust]

On Mon, 2006-12-04 at 21:03 +0100, Jan Engelhardt wrote:
> >> 2) NFS provides persistent storage.
> >
> >To me this sounds like a chicken and an egg problem. It 
> >both depends and provides this at the same time :/. But 
> >hey, if it's supposed to work then OK.
> 
> Way 1:
> 
> mount -nt tmpfs none /var/lib/nfs;
> mount -nt nfs fserve:/tftpboot/linux /mnt;
> mount -n --move /var/lib/nfs /mnt/var/lib/nfs/;

Nope. As Janne implied, the /var/lib/nfs partition _must_ be persistent,
since it is used to store information about the servers on which the
client holds locks and therefore needs to notify in case it reboots.
Using tmpfs defeats the whole purpose of having an rpc.statd in the
first place.

> ./run_init -c /mnt /sbin/init; # or similar
> 
> And you can also start locking after pivot_rooting to /mnt, that would 
> not even require (/mnt)/var/lib/nfs to be a separate mount.

Much better idea. You can delay starting rpc.statd until you have set up
your filesystem provided that you are not running any programs that
require NLM locking. If you do need to run such a program before you
start rpc.statd, then you will have to use the '-onolock' mount option.

Cheers,
  Trond

Re: Mounting NFS root FS

[Jan Engelhardt]

>> Way 1:
>> 
>> mount -nt tmpfs none /var/lib/nfs;
>> mount -nt nfs fserve:/tftpboot/linux /mnt;
>> mount -n --move /var/lib/nfs /mnt/var/lib/nfs/;
>
>Nope. As Janne implied, the /var/lib/nfs partition _must_ be persistent,
>since it is used to store information about the servers on which the
>client holds locks and therefore needs to notify in case it reboots.
>Using tmpfs defeats the whole purpose of having an rpc.statd in the
>first place.

Well, tmpfs is persistens as long as the machine is powered on (and
you did not manage to fubar the tmpfs etc.), no?

Or, you can also get it out of tmpfs again and restart statd once
you've pivoted.

>> ./run_init -c /mnt /sbin/init; # or similar
>> 
>> And you can also start locking after pivot_rooting to /mnt, that would 
>> not even require (/mnt)/var/lib/nfs to be a separate mount.
>
>Much better idea. You can delay starting rpc.statd until you have set up
>your filesystem provided that you are not running any programs that
>require NLM locking. If you do need to run such a program before you
>start rpc.statd, then you will have to use the '-onolock' mount option.

Since we're on the topic locking, is it because I am not running
statd on the client that my NFS client hangs during boot phase?


	-`J'
-- 

Re: Mounting NFS root FS

[Trond Myklebust]

On Tue, 2006-12-05 at 19:43 +0100, Jan Engelhardt wrote:
> >> Way 1:
> >> 
> >> mount -nt tmpfs none /var/lib/nfs;
> >> mount -nt nfs fserve:/tftpboot/linux /mnt;
> >> mount -n --move /var/lib/nfs /mnt/var/lib/nfs/;
> >
> >Nope. As Janne implied, the /var/lib/nfs partition _must_ be persistent,
> >since it is used to store information about the servers on which the
> >client holds locks and therefore needs to notify in case it reboots.
> >Using tmpfs defeats the whole purpose of having an rpc.statd in the
> >first place.
> 
> Well, tmpfs is persistens as long as the machine is powered on (and
> you did not manage to fubar the tmpfs etc.), no?

'persistent' as in 'persistent across client reboots'. See above
paragraph for what the purpose of rpc.statd is.

> Or, you can also get it out of tmpfs again and restart statd once
> you've pivoted.

Then you would lose all the locks that you set before you restarted
statd.

> >> ./run_init -c /mnt /sbin/init; # or similar
> >> 
> >> And you can also start locking after pivot_rooting to /mnt, that would 
> >> not even require (/mnt)/var/lib/nfs to be a separate mount.
> >
> >Much better idea. You can delay starting rpc.statd until you have set up
> >your filesystem provided that you are not running any programs that
> >require NLM locking. If you do need to run such a program before you
> >start rpc.statd, then you will have to use the '-onolock' mount option.
> 
> Since we're on the topic locking, is it because I am not running
> statd on the client that my NFS client hangs during boot phase?

If you have applications that try to set locks before rpc.statd is up
and running, then that would explain it.

  Trond

Re: Mounting NFS root FS

[Jan Engelhardt]

>> >> ./run_init -c /mnt /sbin/init; # or similar
>> >> 
>> >> And you can also start locking after pivot_rooting to /mnt, that would 
>> >> not even require (/mnt)/var/lib/nfs to be a separate mount.
>> >
>> >Much better idea. You can delay starting rpc.statd until you have set up
>> >your filesystem provided that you are not running any programs that
>> >require NLM locking. If you do need to run such a program before you
>> >start rpc.statd, then you will have to use the '-onolock' mount option.
>> 
>> Since we're on the topic locking, is it because I am not running
>> statd on the client that my NFS client hangs during boot phase?
>
>If you have applications that try to set locks before rpc.statd is up
>and running, then that would explain it.

Even if the nfs mount is mounted using -o ro,nolock?


	-`J'
-- 

Re: Mounting NFS root FS

[Trond Myklebust]

On Tue, 2006-12-05 at 20:59 +0100, Jan Engelhardt wrote:
> >> >> ./run_init -c /mnt /sbin/init; # or similar
> >> >> 
> >> >> And you can also start locking after pivot_rooting to /mnt, that would 
> >> >> not even require (/mnt)/var/lib/nfs to be a separate mount.
> >> >
> >> >Much better idea. You can delay starting rpc.statd until you have set up
> >> >your filesystem provided that you are not running any programs that
> >> >require NLM locking. If you do need to run such a program before you
> >> >start rpc.statd, then you will have to use the '-onolock' mount option.
> >> 
> >> Since we're on the topic locking, is it because I am not running
> >> statd on the client that my NFS client hangs during boot phase?
> >
> >If you have applications that try to set locks before rpc.statd is up
> >and running, then that would explain it.
> 
> Even if the nfs mount is mounted using -o ro,nolock?

No. The 'nolock' option means that the NFS client will use the VFS posix
locks only, which will not depend on rpc.statd.

Trond

Re: Mounting NFS root FS

[Jan Engelhardt]

On Dec 5 2006 15:12, Trond Myklebust wrote:
>On Tue, 2006-12-05 at 20:59 +0100, Jan Engelhardt wrote:
JE>>>> Since we're on the topic locking, is it because I am not running
JE>>>> statd on the client that my NFS client hangs during boot phase?
TM>>>
TM>>>If you have applications that try to set locks before rpc.statd is up
TM>>>and running, then that would explain it.
JE>> 
JE>>Even if the nfs mount is mounted using -o ro,nolock?
TM>
TM>No. The 'nolock' option means that the NFS client will use the VFS posix
TM>locks only, which will not depend on rpc.statd.
TM>

By trying a little I found out that it is unionfs which has problems 
with locking.
So on the NFS side (nfs-only, no unionfs above it) all is fine.


	-`J'
-- 

Re: Mounting NFS root FS

[Hans-Peter Jansen]

Am Montag, 4. Dezember 2006 12:51 schrieb Janne Karhunen:
> On 12/2/06, William Estrada <MrUmunhum@popdial.com> wrote:
> > Hi guys,
> >
> >   I have been trying to make FC5's kernel do a boot
> > with an NFS root file system.  I see the support is in the
> > kernel(?).
>
> Is this really properly possible (with read/write access and
> locking in place)? AFAIK NFS client lock state data seems
> to require persistent storage .. ?

Out of curiosity: what's the rational of mounting multiple diskless nodes on 
one rw nfs root filesystem [with locking in place]? 

In my experience, this results in a big mess. I depend heavily on diskless 
setups, where I spent significant time to provide sharing as much data as 
possible between the clients _without_ further interference. The best setup 
I found to get there is using unionfs (which still has issues, mostly due 
to missing/broken mmap support).

Pete