* I have some patches for the autofs-4.x daemon - what to do with them? @ 2010-11-30 22:07 Bill Ryder 2010-12-03 4:27 ` Ian Kent 0 siblings, 1 reply; 4+ messages in thread From: Bill Ryder @ 2010-11-30 22:07 UTC (permalink / raw) To: autofs [-- Attachment #1.1: Type: text/plain, Size: 2050 bytes --] Where should I submit those? Or should I just put it on github or something for anyone who wants them since the autofs-4.x daemon series has been deprecated in favour of the autofs-5.x series. We have about 2 or 3,000 clients where I work and autofs is crucial to our operation. We use ldap and use a simple automount structure. We have about 450 mountpoints and 19 maps. I don't like autofs-5 because I cant' fix automounts individually. It's very convenient to be able to restart individual mounts. Also when a daemon core dumps they don't take out every single map. Also I prefer automount to run the mount command itself - rather than building it into the daemon. The main work I've done is to build in retrying mounts depending on the errors returned by mount (with our client count we overload our fileservers often which causes retryable errors, and sometimes a client will try to mount 100 or so mount points in a very short amount of time which causes some tcp client port exhaustion - which is also retryable). I've fixed a nasty bug that will crash a daemon if an error occurs while mounting, and another request comes in for a nonexistent mount point. I have a flag which helps our samba servers - particular when accessed by macs. There's an option that tells the daemon to return 'not found' for any mount which starts with a . or contains a '*' without looking up ldap. This makes it very fast to handle macs looking to .DS_store and many other dot files/directories when someone browses the top of a map with ghosting on. The '*' was quite amusing (for me anyway) because every so often you'd see a '*' directory in an automount directory. Turns out someone had escaped the '*' and ldap returns the first match. And I reinstated the ldap-cleanup patch which is a much cleaner way to handle the multiple schemas - particularly important because we recently moved to rfc2307bis because I was sick of people being able to mount /vol/AmounTPoINT and /vol/amountpoint and so on. Anyway. What do do with this stuff? Bill Ryder [-- Attachment #1.2: Type: text/html, Size: 2479 bytes --] [-- Attachment #2: Type: text/plain, Size: 140 bytes --] _______________________________________________ autofs mailing list autofs@linux.kernel.org http://linux.kernel.org/mailman/listinfo/autofs ^ permalink raw reply [flat|nested] 4+ messages in thread
* Re: I have some patches for the autofs-4.x daemon - what to do with them? 2010-11-30 22:07 I have some patches for the autofs-4.x daemon - what to do with them? Bill Ryder @ 2010-12-03 4:27 ` Ian Kent 2010-12-06 0:34 ` Bill Ryder 0 siblings, 1 reply; 4+ messages in thread From: Ian Kent @ 2010-12-03 4:27 UTC (permalink / raw) To: Bill Ryder; +Cc: autofs On Wed, 2010-12-01 at 11:07 +1300, Bill Ryder wrote: > Where should I submit those? > > > Or should I just put it on github or something for anyone who wants > them since the autofs-4.x daemon series has been deprecated in favour > of the autofs-5.x series. Probably, but it might be good for me to create a directory on kernel.org and save them their, but a readme with applicable version and usage information would probably make sense. > > > We have about 2 or 3,000 clients where I work and autofs is crucial to > our operation. > > > We use ldap and use a simple automount structure. We have about 450 > mountpoints and 19 maps. > > > I don't like autofs-5 because I cant' fix automounts individually. > It's very convenient to be able to restart individual mounts. Also > when a daemon core dumps they don't take out every single map. Also I > prefer automount to run the mount command itself - rather than > building it into the daemon. Right, but v5 still uses mount(8) as v4 did. I understand your need for segregation but it would be good to get v5 to a state where this isn't a problem for you any more. Tell me more about the "take out every single map" problem. With current v5 you should be able to just start it up again and your mounts should continue to function. The problem of course come when you have a non-responsive server, then you end up re-connecting to non-responsive mounts. Of course, there is still a time where new mounts cannot be made until the startup. > > > The main work I've done is to build in retrying mounts depending on > the errors returned by mount (with our client count we overload our > fileservers often which causes retryable errors, and sometimes a > client will try to mount 100 or so mount points in a very short amount > of time which causes some tcp client port exhaustion - which is also > retryable). So you can't export to allow clients to use insecure ports? > > > I've fixed a nasty bug that will crash a daemon if an error occurs > while mounting, and another request comes in for a nonexistent mount > point. > > > I have a flag which helps our samba servers - particular when > accessed by macs. There's an option that tells the daemon to return > 'not found' for any mount which starts with a . or contains a '*' > without looking up ldap. This makes it very fast to handle macs > looking to .DS_store and many other dot files/directories when someone > browses the top of a map with ghosting on. The '*' was quite amusing > (for me anyway) because every so often you'd see a '*' directory in an > automount directory. Turns out someone had escaped the '*' and ldap > returns the first match. > > > And I reinstated the ldap-cleanup patch which is a much cleaner way to > handle the multiple schemas - particularly important because we > recently moved to rfc2307bis because I was sick of people being able > to mount > > > /vol/AmounTPoINT and /vol/amountpoint and so on. Sounds useful, we should talk more about these things. > > > Anyway. What do do with this stuff? > > > Bill Ryder > > > > > _______________________________________________ > autofs mailing list > autofs@linux.kernel.org > http://linux.kernel.org/mailman/listinfo/autofs ^ permalink raw reply [flat|nested] 4+ messages in thread
* Re: I have some patches for the autofs-4.x daemon - what to do with them? 2010-12-03 4:27 ` Ian Kent @ 2010-12-06 0:34 ` Bill Ryder 2010-12-06 1:11 ` Ian Kent 0 siblings, 1 reply; 4+ messages in thread From: Bill Ryder @ 2010-12-06 0:34 UTC (permalink / raw) To: Ian Kent; +Cc: autofs [-- Attachment #1.1: Type: text/plain, Size: 4366 bytes --] On Fri, Dec 3, 2010 at 5:27 PM, Ian Kent <raven@themaw.net> wrote: > On Wed, 2010-12-01 at 11:07 +1300, Bill Ryder wrote: > > Where should I submit those? > > > > > > > Probably, but it might be good for me to create a directory on > kernel.org and save them their, but a readme with applicable version and > usage information would probably make sense. > OK. How about if I break out the patches against 4.1.4 against ftp://ftp.kernel.org/pub/linux/daemons/autofs/v4/autofs-4.1.4 I'll try to make the changelog detailed enough in each patch so it's clear what it does. > > > I don't like autofs-5 because I cant' fix automounts individually. > > It's very convenient to be able to restart individual mounts. Also > > when a daemon core dumps they don't take out every single map. Also I > > prefer automount to run the mount command itself - rather than > > building it into the daemon. > > Right, but v5 still uses mount(8) as v4 did. > Oh. I thought I saw some mount code in it when I was taking a cursory look. (looking over the code now I suspect I saw the rpc_get_exports code and didn't look properly). That's good. I thought it was a bit crazy at the time but didn't look properly. Sorry about that. (I thought that time was trying to be saved by not running mount). > I understand your need for segregation but it would be good to get v5 to > a state where this isn't a problem for you any more. > Yeah. When I last looked (which was admittedly last year now) I was looking for a way to restart the thread responsible for a particular map if it had crashed or just needed a poke or if I wanted to change the ldap query it was using. At that time I couldn't even find a way to easily see which thread owned which map. Let alone start up one map with a hand rolled ldap string. > Tell me more about the "take out every single map" problem. With current > v5 you should be able to just start it up again and your mounts should > continue to function. The problem of course come when you have a > non-responsive server, then you end up re-connecting to non-responsive > mounts. Of course, there is still a time where new mounts cannot be made > until the startup. > As I understand it a core dump will take the autofs-5 daemon and all of it's threads down. So existing mounts will continue to function but any request requiring a new mount will not. With a process per map you only lose one map - not all of them - if it should crash. We run a lot of checkers that will bring up individual broken maps. This used to be a constant thing with 4.1.3 but 4.1.4 is much more stable. Also sometimes when I'm testing things I'll want to start up a map with special options (ldap filter usually) but leave the other maps alone. As I see it the problem with the multithreaded automounter is all the functionality you get for free from linux (starting individual processes for each map, looking at which process owns which mount points by looking at ps, and so on) you need to code for explicitly in autofs 5. > > > > > > > The main work I've done is to build in retrying mounts depending on > > the errors returned by mount (with our client count we overload our > > fileservers often which causes retryable errors, and sometimes a > > client will try to mount 100 or so mount points in a very short amount > > of time which causes some tcp client port exhaustion - which is also > > retryable). > > So you can't export to allow clients to use insecure ports? > I could do this but we have around 30 'normal' fileservers (netapps) and a lot (> 100) virtual NFS servers on 4 clusters which are more painful to administer. And of course the way to specify secure vs insecure is different. And since I already have retry code in place to handle overloaded fileservers this was an easy - if ugly - workaround (which hopefully we won't need anymore because we've changed some things which created these little storms). It was rare but it did happen. For our work it's better if a mount hangs for a while rather than failing and returning a path not found to the application. The big fault with my retry code is I look at a set of compiled in strings of errors that are retryable. Very painful when you just want to add one more error message. Ideally I'd read a file with the list of retryable errors in it when automounter starts up. Bill [-- Attachment #1.2: Type: text/html, Size: 6085 bytes --] [-- Attachment #2: Type: text/plain, Size: 140 bytes --] _______________________________________________ autofs mailing list autofs@linux.kernel.org http://linux.kernel.org/mailman/listinfo/autofs ^ permalink raw reply [flat|nested] 4+ messages in thread
* Re: I have some patches for the autofs-4.x daemon - what to do with them? 2010-12-06 0:34 ` Bill Ryder @ 2010-12-06 1:11 ` Ian Kent 0 siblings, 0 replies; 4+ messages in thread From: Ian Kent @ 2010-12-06 1:11 UTC (permalink / raw) To: Bill Ryder; +Cc: autofs On Mon, 2010-12-06 at 13:34 +1300, Bill Ryder wrote: > > > On Fri, Dec 3, 2010 at 5:27 PM, Ian Kent <raven@themaw.net> wrote: > On Wed, 2010-12-01 at 11:07 +1300, Bill Ryder wrote: > > Where should I submit those? > > > > > > > > Probably, but it might be good for me to create a directory on > kernel.org and save them their, but a readme with applicable > version and > usage information would probably make sense. > > > OK. How about if I break out the patches against 4.1.4 > against ftp://ftp.kernel.org/pub/linux/daemons/autofs/v4/autofs-4.1.4 > > > I'll try to make the changelog detailed enough in each patch so it's > clear what it does. OK, so lets work through that. snip ... > > I understand your need for segregation but it would be good to > get v5 to > a state where this isn't a problem for you any more. > > > Yeah. > > > When I last looked (which was admittedly last year now) I was looking > for a way to restart the thread responsible for a particular map if it > had crashed or just needed a poke or if I wanted to change the ldap > query it was using. At that time I couldn't even find a way to easily > see which thread owned which map. Let alone start up one map with a > hand rolled ldap string. > > > > Tell me more about the "take out every single map" problem. > With current > v5 you should be able to just start it up again and your > mounts should > continue to function. The problem of course come when you have > a > non-responsive server, then you end up re-connecting to > non-responsive > mounts. Of course, there is still a time where new mounts > cannot be made > until the startup. > > > > > As I understand it a core dump will take the autofs-5 daemon and all > of it's threads down. Yep. > > > So existing mounts will continue to function but any request requiring > a new mount will not. And, yep. > > > With a process per map you only lose one map - not all of them - if > it should crash. Now that I think about it we had a request similar to this a while ago. I'm not sure about it now, I'll need to look through the history, but you may be able to start multiple autofs instances, each using their own master map. That's essentially what you want I think. There were a few problem at the time, which were fixed I think, and there may be new problems with it now. But it should be able to be done. > > > We run a lot of checkers that will bring up individual broken maps. > This used to be a constant thing with 4.1.3 but 4.1.4 is much more > stable. > > > Also sometimes when I'm testing things I'll want to start up a map > with special options (ldap filter usually) but leave the other maps > alone. > > > As I see it the problem with the multithreaded automounter is all the > functionality you get for free from linux (starting individual > processes for each map, looking at which process owns which mount > points by looking at ps, and so on) you need to code for explicitly in > autofs 5. > > > > > > > > > > > > The main work I've done is to build in retrying mounts > depending on > > the errors returned by mount (with our client count we > overload our > > fileservers often which causes retryable errors, and > sometimes a > > client will try to mount 100 or so mount points in a very > short amount > > of time which causes some tcp client port exhaustion - which > is also > > retryable). > > > So you can't export to allow clients to use insecure ports? > > > I could do this but we have around 30 'normal' fileservers (netapps) > and a lot (> 100) virtual NFS servers on 4 clusters which are more > painful to administer. And of course the way to specify secure vs > insecure is different. > > > And since I already have retry code in place to handle overloaded > fileservers this was an easy - if ugly - workaround (which hopefully > we won't need anymore because we've changed some things which created > these little storms). It was rare but it did happen. > > > For our work it's better if a mount hangs for a while rather than > failing and returning a path not found to the application. > > > The big fault with my retry code is I look at a set of compiled in > strings of errors that are retryable. Very painful when you just want > to add one more error message. Ideally I'd read a file with the list > of retryable errors in it when automounter starts up. > > > Bill > > > > ^ permalink raw reply [flat|nested] 4+ messages in thread
end of thread, other threads:[~2010-12-06 1:11 UTC | newest] Thread overview: 4+ messages (download: mbox.gz follow: Atom feed -- links below jump to the message on this page -- 2010-11-30 22:07 I have some patches for the autofs-4.x daemon - what to do with them? Bill Ryder 2010-12-03 4:27 ` Ian Kent 2010-12-06 0:34 ` Bill Ryder 2010-12-06 1:11 ` Ian Kent
This is an external index of several public inboxes, see mirroring instructions on how to clone and mirror all data and code used by this external index.