Re: Could NFS daemons be started via inetd?

Re: Could NFS daemons be started via inetd?

[Jesse Pollard]

VDA <VDA@port.imtp.ilyichevsk.odessa.ua>:
> Hi,
> 
> I am setting up NFS on my Linux box.
> When I start server daemons from init scripts or by hand,
> everything is working fine.
> 
> I tried to arrange these daemons to be run by inetd
> but after I issue an NFS mount command inetd starts spawning
> tons on rpc.mountd daemons. Log is filled with:
> rpc.mountd[179]: connect from 127.0.0.1
> rpc.mountd[180]: connect from 127.0.0.1
> ...
> and load average goes up (went up to 40)
> I repeatedly killall'ed rpc.mountd and eventually inetd
> noticed failing service and disabled it.
> 
> Does anybody tried this? If you do, I am very interested in your
> inetd.conf and/or NFS part of startup script. Mine is:
....

Simple answer - no.

The reason it can't is in two parts:

1. these daemons create their own socket rather than recieving one
   from inetd.
2. The daemons must connect to rpcbind or portmap. Usually this daemon
   is not running at the time inetd is started.

The nfs daemon actually starts multiple daemons (it forks up to n servers
for "optimum" client support). Each server daemon may be servicing a
different client (or even the same client, different request). I have
seen performance improvements (NOT on a Linux server...) of one server
for each mount on each client (up to 18-25 servers).

These daemons must run all the time or performance will be REALLY bad. Each
connection may request ~ 8K bytes of data. 

-------------------------------------------------------------------------
Jesse I Pollard, II
Email: pollard@navo.hpc.mil

Any opinions expressed are solely my own.

Re: Could NFS daemons be started via inetd?

[VDA]

Hello Jesse,

Wednesday, August 22, 2001, 4:41:00 PM, you wrote:
>> I am setting up NFS on my Linux box.
>> When I start server daemons from init scripts or by hand,
>> everything is working fine.
>> 
>> I tried to arrange these daemons to be run by inetd
>> but after I issue an NFS mount command inetd starts spawning
>> tons on rpc.mountd daemons. Log is filled with:
>> ...
>> Does anybody tried this? If you do, I am very interested in your
>> inetd.conf and/or NFS part of startup script. Mine is:

> Simple answer - no.
> The reason it can't is in two parts:
> 1. these daemons create their own socket rather than recieving one
>    from inetd.

Please enlighten me: do daemons need to be writted with some support
code specific for inetd in order to be compatible with it?

> 2. The daemons must connect to rpcbind or portmap. Usually this daemon
>    is not running at the time inetd is started.

I have rpc.portmap started just before inetd in my init script.
BTW, how portmapper

> These daemons must run all the time or performance will be REALLY bad. Each
> connection may request ~ 8K bytes of data.

For now, my machine is under very light load (around 0.0) so this is
not a problem.
But I experiment a lot with different Linux toys
(NFS,Samba,TFTP,BOOTP/DHCP,PostgreSQL,MySQL). I'd like to configure it
so that under light load daemons die out, leaving bare minimum
(ideally, inetd only)

I presume daemons spawned by inetd are supposed to be designed so they
lurk around for some time in case some more requests are coming for them
and die after a timeout? Can they be made to spawn additional daemons when
load increases? This behavior would be nice - no need to guess how
many nfsd daemons to spawn for NFS, httpd's for HTTP etc.

Want to know why I'm asking all this?
Go to http://port.imtp.ilyichevsk.odessa.ua/vda/
-- 
Best regards,
VDA                            mailto:VDA@port.imtp.ilyichevsk.odessa.ua

Re: Could NFS daemons be started via inetd?

[Lehmann]

On Sat, Aug 25, 2001 at 05:13:57PM +0300, VDA <VDA@port.imtp.ilyichevsk.odessa.ua> wrote:
> > Simple answer - no.
> > The reason it can't is in two parts:
> > 1. these daemons create their own socket rather than recieving one
> >    from inetd.

this is fine, other servers do that as well and work.

> > 2. The daemons must connect to rpcbind or portmap. Usually this daemon
> >    is not running at the time inetd is started.

this would be a bug in the initscripts, inetd is a rpc client so your
initscripts better make sure it is running (when neecsssary) before inetd.

> I have rpc.portmap started just before inetd in my init script.
> BTW, how portmapper

portmap often is too slow on startup. move it earlier in the ss sequence
or add a sleep 2 somewhere.

> > These daemons must run all the time or performance will be REALLY bad. Each
> > connection may request ~ 8K bytes of data.

just like othe rudp daemons they can linger, which works fine. also,
mountd does not need to be run all the time, it is very rarely used.

having said that, this is not kernel-related and shhould be taken
elsewhere. i managed to make (userspace) mountd run from inetd, but never
managed to get the nfsd working that way.

-- 
      -----==-                                             |
      ----==-- _                                           |
      ---==---(_)__  __ ____  __       Marc Lehmann      +--
      --==---/ / _ \/ // /\ \/ /       pcg@goof.com      |e|
      -=====/_/_//_/\_,_/ /_/\_\       XX11-RIPE         --+
    The choice of a GNU generation                       |
                                                         |

Re: Could NFS daemons be started via inetd?

[Jesse Pollard]

On Sat, 25 Aug 2001, VDA wrote:
>Hello Jesse,
>
>Wednesday, August 22, 2001, 4:41:00 PM, you wrote:
>>> I am setting up NFS on my Linux box.
>>> When I start server daemons from init scripts or by hand,
>>> everything is working fine.
>>> 
>>> I tried to arrange these daemons to be run by inetd
>>> but after I issue an NFS mount command inetd starts spawning
>>> tons on rpc.mountd daemons. Log is filled with:
>>> ...
>>> Does anybody tried this? If you do, I am very interested in your
>>> inetd.conf and/or NFS part of startup script. Mine is:
>
>> Simple answer - no.
>> The reason it can't is in two parts:
>> 1. these daemons create their own socket rather than recieving one
>>    from inetd.
>
>Please enlighten me: do daemons need to be writted with some support
>code specific for inetd in order to be compatible with it?

not really:

Daemons started by inetd inherit the socket as inetd actually holds the
open socket.

NFS daemons, on the other hand, open an available socket (some use unreserved
sockets), then connect to portmap/rpcbind to inform portmap of the mapping.

Sendmail is similar - without a special flag to say "don't initialize socket,
use inherited", it won't work either - even though both can specify port
25 for use.

>> 2. The daemons must connect to rpcbind or portmap. Usually this daemon
>>    is not running at the time inetd is started.
>
>I have rpc.portmap started just before inetd in my init script.
>BTW, how portmapper
>
>> These daemons must run all the time or performance will be REALLY bad. Each
>> connection may request ~ 8K bytes of data.
>
>For now, my machine is under very light load (around 0.0) so this is
>not a problem.
>But I experiment a lot with different Linux toys
>(NFS,Samba,TFTP,BOOTP/DHCP,PostgreSQL,MySQL). I'd like to configure it
>so that under light load daemons die out, leaving bare minimum
>(ideally, inetd only)

That is exactly what you want in a file server (even if there are a lot
of NFS daemons present. What you don't want is to have to wait .1 second
for the process to be forked/execed. This ends up happening for every block
read from disk.

>I presume daemons spawned by inetd are supposed to be designed so they
>lurk around for some time in case some more requests are coming for them
>and die after a timeout? Can they be made to spawn additional daemons when
>load increases? This behavior would be nice - no need to guess how
>many nfsd daemons to spawn for NFS, httpd's for HTTP etc.

That depends on the daemon.

Things like telnetd (ftpd, rshd, rlogind ...) don't wait around. Once they are
spawned off by inetd, inetd goes back to waiting for another connection on the
original socket. UDP daemons don't wait around either - these don't have a
permanent connection anyway and can only process one packet, then must either
exit, or provide something else to tell the remote system to use a different
port.

It is possible for daemons to do this, just not often done. Some data server
processes can do that. There just aren't any common ones (too much work in
providing user identification, authentication, and authorization - slows them
down too much).

NFS doesn't use connections established that way:

1. client contacts portmap to locate the service/port desired (mountd is
   part of this)
2. portmap returns target UDP port.
3. client uses that port to send NFS request to (handled by biod on the client).
4. all NFS daemons (depending on implmentations) will wakeup and read the
   NFS request (first one there gets it)... ((( better implementations will
	have the daemons waiting in a queue and only one will wake up at a
	time)))
5. The NFS Daemon replies to the biod on the client, which in turn passes
   the data back to the kernel and on to the user process.

This is also why a delay/timeout can occur if the server crashes. (the port
changes). The servers lockd/rpcstatd resynchronize with the remote hosts forcing
a remount of the NFS file system. This re-identifies the port nfsd is listening
on to the client. It also reestablishes any file locks (NOT guaranteed between
client systems though - it attempts to provide shared locking.. it just
doesn't quite work. Better than nothing, it does work with multiple processes
on ONE client. Each client is consistant with itself, but not with other
clients).

>Want to know why I'm asking all this?
>Go to http://port.imtp.ilyichevsk.odessa.ua/vda/
-- 
-------------------------------------------------------------------------
Jesse I Pollard, II
Email: jesse@cats-chateau.net

Any opinions expressed are solely my own.

Could NFS daemons be started via inetd?

[VDA]

Hi,

I am setting up NFS on my Linux box.
When I start server daemons from init scripts or by hand,
everything is working fine.

I tried to arrange these daemons to be run by inetd
but after I issue an NFS mount command inetd starts spawning
tons on rpc.mountd daemons. Log is filled with:
rpc.mountd[179]: connect from 127.0.0.1
rpc.mountd[180]: connect from 127.0.0.1
...
and load average goes up (went up to 40)
I repeatedly killall'ed rpc.mountd and eventually inetd
noticed failing service and disabled it.

Does anybody tried this? If you do, I am very interested in your
inetd.conf and/or NFS part of startup script. Mine is:

inetd.conf
----------
...
# NFS daemons (FIXME: not working)
rquotad/1-2     dgram   rpc/udp wait    root    /usr/sbin/tcpd  /usr/sbin/rpc.rquotad
nfs/2-3         dgram   rpc/udp wait    root    /usr/sbin/tcpd  /usr/sbin/rpc.nfsd
mountd/1-3      dgram   rpc/udp wait    root    /usr/sbin/tcpd  /usr/sbin/rpc.mountd
nlockmgr/1-4    dgram   rpc/udp wait    root    /usr/sbin/tcpd  /usr/sbin/rpc.lockd
rstatd/1-3      dgram   rpc/udp wait    root    /usr/sbin/tcpd  /usr/sbin/rpc.rstatd
...

init script
-----------
...
echo "* Starting NFS services:"
echo "  DEBUG: modprobe nfsd"
modprobe nfsd
echo "  DEBUG: exportfs -r - I will convert /etc/exports to /var/lib/nfs/* conv"
/usr/sbin/exportfs -r         # Does /etc/exports -> /var/lib/nfs/* conv
echo "  DEBUG: rpc.rquotad - needed?"
/usr/sbin/rpc.rquotad
echo "  DEBUG: rpc.nfsd 1 - 1 server started"
/usr/sbin/rpc.nfsd 1
echo "  DEBUG: rpc.mountd - listen to mount requests"
/usr/sbin/rpc.mountd
# With newer kernels, this starts by itself, but this won't hurt:
#echo "  DEBUG: rpc.lockd"
#/usr/sbin/rpc.lockd
echo "  DEBUG: rpc.statd - lock recovery after crash/reboot?"
/usr/sbin/rpc.statd
...

kernel: 2.4.5

Please CC me. I'm not on the list.