RE: multiple servers per automount

RE: multiple servers per automount

[Ogden, Aaron A.]

Ouch.  As you may know, the limit is *much* lower in linux.  Something
that I've been struggling with recently...

Under normal circumstances I would not be concerned with 'limitations'
of a few hundred active NFS mounts, but such limitations certainly limit
scalability for the extreme cases.

-----Original Message-----
From: Mike Waychison [mailto:Michael.Waychison@Sun.COM] 
Sent: Thursday, October 09, 2003 1:24 PM
To: Ian Kent
Cc: autofs@linux.kernel.org
Subject: Re: [autofs] multiple servers per automount


Ian Kent wrote:

>On a related not Mike.
>
>Is the number of NFS mounts on Solaris unlimited or is it table driven?
>I can't find a reference to it anywhere.
>
>  
>
I've asked around, and I'm told that there is no set limit (besides 
having enough memory).  I'm told that more recent Solaris releases have 
been tested up to around 260,000 concurrent NFS mounts.

Mike Waychison

_______________________________________________
autofs mailing list
autofs@linux.kernel.org
http://linux.kernel.org/mailman/listinfo/autofs

Re: multiple servers per automount

[Mike Waychison]

Ogden, Aaron A. wrote:

>Ouch.  As you may know, the limit is *much* lower in linux.  Something
>that I've been struggling with recently...
>
>Under normal circumstances I would not be concerned with 'limitations'
>of a few hundred active NFS mounts, but such limitations certainly limit
>scalability for the extreme cases.
>  
>

The maximum number of plain pseudo-block device filesystems on a given 
filesystem is limitted to 256. (This includes proc, autofs, nfs..).

This is because pseudo-block filesystems all use major 0, and each have 
a different minor (thus the 256 limit). 

There are however patches floating around (look at SuSe's kernels, I'm 
not sure about RH) that allow n majors to be used (default 5).  This 
gives you 1280 mounts, a big step up :)

Mike Waychison

Re: multiple servers per automount

[Ian Kent]

On Thu, 9 Oct 2003, Mike Waychison wrote:

> Ogden, Aaron A. wrote:
>
> >Ouch.  As you may know, the limit is *much* lower in linux.  Something
> >that I've been struggling with recently...
> >
> >Under normal circumstances I would not be concerned with 'limitations'
> >of a few hundred active NFS mounts, but such limitations certainly limit
> >scalability for the extreme cases.
> >
> >
>
> The maximum number of plain pseudo-block device filesystems on a given
> filesystem is limitted to 256. (This includes proc, autofs, nfs..).
>
> This is because pseudo-block filesystems all use major 0, and each have
> a different minor (thus the 256 limit).
>
> There are however patches floating around (look at SuSe's kernels, I'm
> not sure about RH) that allow n majors to be used (default 5).  This
> gives you 1280 mounts, a big step up :)
>

But as Aaron and I know things go pear shaped at just shy of 800 mounts
with RedHat kernels. They have the more-unnamed patch.

So this would indicate that even if there is a device system that can
increase the number of unnamed devices that subsystems like NFS cannot
handle this many mounts.

-- 

   ,-._|\    Ian Kent
  /      \   Perth, Western Australia
  *_.--._/   E-mail: raven@themaw.net
        v    Web: http://themaw.net/

RE: multiple servers per automount

[Ogden, Aaron A.]

-----Original Message-----
From: Ian Kent [mailto:raven@themaw.net] 
Sent: Thursday, October 09, 2003 8:09 PM
To: Mike Waychison
Cc: Ogden, Aaron A.; autofs mailing list; nfs@lists.sourceforge.net
Subject: Re: [autofs] multiple servers per automount

>> The maximum number of plain pseudo-block device filesystems on a
given
>> filesystem is limitted to 256. (This includes proc, autofs, nfs..).
>>
>> This is because pseudo-block filesystems all use major 0, and each
have
>> a different minor (thus the 256 limit).
>>
>> There are however patches floating around (look at SuSe's kernels,
I'm
>> not sure about RH) that allow n majors to be used (default 5).  This
>> gives you 1280 mounts, a big step up :)
>>
>
> But as Aaron and I know things go pear shaped at just shy of 800
mounts
> with RedHat kernels. They have the more-unnamed patch.
>
> So this would indicate that even if there is a device system that can
> increase the number of unnamed devices that subsystems like NFS cannot
> handle this many mounts.

Maybe.  I'm not 100% certain though.  Currently I am holding steady at
710 active mounts, I am going to write a little script to mount more in
small increments, ie. read a list of ~1000 mountpoints from /home, mount
a few of them, check the filesystems, and repeat... this way I will know
exactly where things break down.

RE: multiple servers per automount

[Ogden, Aaron A.]

-----Original Message-----
From: Mike Waychison [mailto:Michael.Waychison@Sun.COM] 
Sent: Thursday, October 09, 2003 4:37 PM
To: Ogden, Aaron A.
Cc: autofs mailing list; Ian Kent
Subject: Re: [autofs] multiple servers per automount


> I don't see how the autofs code would cause such a problem...

I don't either.  There are a lot of variables at work though...

> 2.6 mainline still has the 256 pseudo-block device limit.
Fortunately, 
> the patches used in distros should still apply with minimal fuzz.
Devfs 
> probably won't help matters here.  

That is depressing... here I was thinking that 2.6 would be some kind of
utopian wonderland free of problems.  :-(

> I think the next step in 2.6 is to change the bitmap used to grow as 
> needed..  especially now that minors are 20 bits, or 1048576 minors
:))

Yes that is what I was thinking about, unfortunately I don't know the
details, I'm working off of a hazy memory.  I thought devfs didn't use
the major/minor concept anymore?  I'm curious to see how it will work
with storage devices, esp. hot-pluggable devices like SCSI and USB
drives.  Right now their device names change depending on the order that
they are connected, it's quite confusing.  Someone at Dell wrote
software to keep the device names constant by using aliases for them.
It's a nice concept but it's just a hack, the real problem is that we
don't have /dev/c0t0d0s0 as in Solaris.

RE: multiple servers per automount

[Ogden, Aaron A.]

Yes, we are using the RH kernels as a base, the limit is supposed to be
1200+ but my experiments have shown it to be just shy of 800.  Or maybe
that results is due to a bug in the autofs code.  I think the real
solution to this problem is Richard Gooch's devfs and kernel 2.6.  It's
anyone's guess when there will be a distribution using both of those but
I think Fedora might be the first.

-----Original Message-----
From: Mike Waychison [mailto:Michael.Waychison@Sun.COM] 
Sent: Thursday, October 09, 2003 2:00 PM
To: Ogden, Aaron A.
Cc: autofs mailing list; Ian Kent
Subject: Re: [autofs] multiple servers per automount


Ogden, Aaron A. wrote:

>Ouch.  As you may know, the limit is *much* lower in linux.  Something
>that I've been struggling with recently...
>
>Under normal circumstances I would not be concerned with 'limitations'
>of a few hundred active NFS mounts, but such limitations certainly
limit
>scalability for the extreme cases.
>  
>

The maximum number of plain pseudo-block device filesystems on a given 
filesystem is limitted to 256. (This includes proc, autofs, nfs..).

This is because pseudo-block filesystems all use major 0, and each have 
a different minor (thus the 256 limit). 

There are however patches floating around (look at SuSe's kernels, I'm 
not sure about RH) that allow n majors to be used (default 5).  This 
gives you 1280 mounts, a big step up :)

Mike Waychison

Re: multiple servers per automount

[Mike Waychison]

Ogden, Aaron A. wrote:

>Yes, we are using the RH kernels as a base, the limit is supposed to be
>1200+ but my experiments have shown it to be just shy of 800.  Or maybe
>that results is due to a bug in the autofs code.  
>

I don't see how the autofs code would cause such a problem...

>I think the real
>solution to this problem is Richard Gooch's devfs and kernel 2.6.  It's
>anyone's guess when there will be a distribution using both of those but
>I think Fedora might be the first.
>

2.6 mainline still has the 256 pseudo-block device limit.  Fortunately, 
the patches used in distros should still apply with minimal fuzz.  Devfs 
probably won't help matters here.  

I think the next step in 2.6 is to change the bitmap used to grow as 
needed..  especially now that minors are 20 bits, or 1048576 minors  :))

Mike Waychison

Re: multiple servers per automount

[Ian Kent]

On Thu, 9 Oct 2003, Mike Waychison wrote:

>
> 2.6 mainline still has the 256 pseudo-block device limit.  Fortunately,
> the patches used in distros should still apply with minimal fuzz.  Devfs
> probably won't help matters here.
>
> I think the next step in 2.6 is to change the bitmap used to grow as
> needed..  especially now that minors are 20 bits, or 1048576 minors  :))

Missed that.

I'll have to look more closely.


-- 

   ,-._|\    Ian Kent
  /      \   Perth, Western Australia
  *_.--._/   E-mail: raven@themaw.net
        v    Web: http://themaw.net/

RE: multiple servers per automount

[Ian Kent]

On Thu, 9 Oct 2003, Ogden, Aaron A. wrote:

>
> Yes, we are using the RH kernels as a base, the limit is supposed to be
> 1200+ but my experiments have shown it to be just shy of 800.  Or maybe
> that results is due to a bug in the autofs code.  I think the real
> solution to this problem is Richard Gooch's devfs and kernel 2.6.  It's
> anyone's guess when there will be a distribution using both of those but
> I think Fedora might be the first.
>

Looks like the limit is here to stay as 2.6 looks similar to 2.4 in that
area and udev is destined to be used as a replacement for devfs (which
never caught on). udev does not address this issue either.

Any know if/how this issue will be addressed?

-- 

   ,-._|\    Ian Kent
  /      \   Perth, Western Australia
  *_.--._/   E-mail: raven@themaw.net
        v    Web: http://themaw.net/

multiple servers per automount

[Rich Sahlender]

This is probably a faq but I havn't seen it asked in recent months...

I have a situation where two network appliance filers with identical
file systems are used to serve read only mounts for CAD software tools.
Both Solaris and HP-UX support an automount like:

cadtool	-ro	tools1,tools2:/vol/vol1/tools/&

and will automatically switch to the other server if the initial mount
becomes unavailable.

Last time I was able to check Linux autofs did not support the failover
and complained about the syntax. Has this changed or are there plans to
support the syntax above with failover for read only mounts?

Thanks,
Rich Sahlender

Re: multiple servers per automount

[H. Peter Anvin]

Rich Sahlender wrote:
> This is probably a faq but I havn't seen it asked in recent months...
> 
> I have a situation where two network appliance filers with identical
> file systems are used to serve read only mounts for CAD software tools.
> Both Solaris and HP-UX support an automount like:
> 
> cadtool	-ro	tools1,tools2:/vol/vol1/tools/&
> 
> and will automatically switch to the other server if the initial mount
> becomes unavailable.
> 
> Last time I was able to check Linux autofs did not support the failover
> and complained about the syntax. Has this changed or are there plans to
> support the syntax above with failover for read only mounts?
> 

This should not be an autofs feature but an NFS feature, for obvious
reasons: it's equally applicable for static mounts.

Furhtermore, failover HAS to be done at the NFS layer.

The fact that Slowlaris does it in autofs is not a good motivation --
it's idiotic design.

	-hpa

Re: multiple servers per automount

[Rich Sahlender]

On Wed, Oct 08, 2003 at 11:57:16AM -0700, H. Peter Anvin wrote:
> Rich Sahlender wrote:
> > This is probably a faq but I havn't seen it asked in recent months...
> > 
> > I have a situation where two network appliance filers with identical
> > file systems are used to serve read only mounts for CAD software tools.
> > Both Solaris and HP-UX support an automount like:
> > 
> > cadtool	-ro	tools1,tools2:/vol/vol1/tools/&
> > 
> > and will automatically switch to the other server if the initial mount
> > becomes unavailable.
> > 
> > Last time I was able to check Linux autofs did not support the failover
> > and complained about the syntax. Has this changed or are there plans to
> > support the syntax above with failover for read only mounts?
> > 
> 
> This should not be an autofs feature but an NFS feature, for obvious
> reasons: it's equally applicable for static mounts.
> 
> Furhtermore, failover HAS to be done at the NFS layer.
> 
> The fact that Slowlaris does it in autofs is not a good motivation --
> it's idiotic design.
> 
> 	-hpa

I disagree. These are read only mounts. How is it an idiotic design to
provide a little extra availability by switching to another server in the
list if the mounted server becomes unavailable?

Furthermore, the fact that it is equally applicable for static mounts
does not make it an obvious NFS issue. I submit it's both an NFS and an
autofs issue...

Re: multiple servers per automount

[Mike Waychison]

H. Peter Anvin wrote:

>This should not be an autofs feature but an NFS feature, for obvious
>reasons: it's equally applicable for static mounts.
>
>Furhtermore, failover HAS to be done at the NFS layer.
>
>The fact that Slowlaris does it in autofs is not a good motivation --
>it's idiotic design.
>  
>

*Ahem*.  Solaris has had NFS client failover since 2.6.   Linux is still 
playing catch up in this arena.
Failover is not handled in the automount layer, but rather passed on to 
nfs layer itself.  man mount_nfs.

Mike Waychison

Re: multiple servers per automount

[Ian Kent]

On Wed, 8 Oct 2003, Mike Waychison wrote:

> H. Peter Anvin wrote:
>
> >This should not be an autofs feature but an NFS feature, for obvious
> >reasons: it's equally applicable for static mounts.
> >
> >Furhtermore, failover HAS to be done at the NFS layer.
> >
> >The fact that Slowlaris does it in autofs is not a good motivation --
> >it's idiotic design.

Ohh .. Nasty .... mmmmaaahhh.

> >
> >
>
> *Ahem*.  Solaris has had NFS client failover since 2.6.   Linux is still
> playing catch up in this arena.
> Failover is not handled in the automount layer, but rather passed on to
> nfs layer itself.  man mount_nfs.
>

On a related not Mike.

Is the number of NFS mounts on Solaris unlimited or is it table driven?
I can't find a reference to it anywhere.

-- 

   ,-._|\    Ian Kent
  /      \   Perth, Western Australia
  *_.--._/   E-mail: raven@themaw.net
        v    Web: http://themaw.net/

Re: multiple servers per automount

[Mike Waychison]

Ian Kent wrote:

>On a related not Mike.
>
>Is the number of NFS mounts on Solaris unlimited or is it table driven?
>I can't find a reference to it anywhere.
>
>  
>
I've asked around, and I'm told that there is no set limit (besides 
having enough memory).  I'm told that more recent Solaris releases have 
been tested up to around 260,000 concurrent NFS mounts.

Mike Waychison