* Re: making the mountpoints visible all the time (fwd)
@ 2003-11-19 8:13 Ian Kent
2003-11-19 16:45 ` H. Peter Anvin
0 siblings, 1 reply; 16+ messages in thread
From: Ian Kent @ 2003-11-19 8:13 UTC (permalink / raw)
To: autofs mailing list
On Sun, 16 Nov 2003, H. Peter Anvin wrote:
> Ian Kent wrote:
> >
> > But how do you know what the keys are?
> >
> > In order to make mount points visible you need to know what they are. The
> > definition of a program map is that it produces map entries as output
> > given an input key.
> >
>
> You need a new operation on a map -- enumerate. Most, but not all map
> types support it.
>
> In the case of a program map I would suggest implementing enumeration by
> passing no argument and expecting the enumerated list back.
>
That sounds good a good idea for inclusion in 4.1.1 (I have quite a list
already).
Move the decision on what the map contains to the program map implementer.
That would be fairly straight forward to implement given that I have a
placeholder function in each map module.
Ian
^ permalink raw reply [flat|nested] 16+ messages in thread
* Re: making the mountpoints visible all the time (fwd)
2003-11-19 8:13 making the mountpoints visible all the time (fwd) Ian Kent
@ 2003-11-19 16:45 ` H. Peter Anvin
2003-11-21 0:51 ` Ian Kent
0 siblings, 1 reply; 16+ messages in thread
From: H. Peter Anvin @ 2003-11-19 16:45 UTC (permalink / raw)
To: Ian Kent; +Cc: autofs mailing list
Ian Kent wrote:
>
> That sounds good a good idea for inclusion in 4.1.1 (I have quite a list
> already).
>
> Move the decision on what the map contains to the program map implementer.
> That would be fairly straight forward to implement given that I have a
> placeholder function in each map module.
>
Well, it's pretty much the only thing that makes sense, since a program
map can inherently do "anything".
Pretty much ghosting implies that there are now two operations required
of a map: key->data lookup (old), and enumerate all keys (new).
And of course autofs needs to deal with the consequences of the map
changing underneath it.
-hpa
^ permalink raw reply [flat|nested] 16+ messages in thread
* Re: making the mountpoints visible all the time (fwd)
2003-11-19 16:45 ` H. Peter Anvin
@ 2003-11-21 0:51 ` Ian Kent
2003-11-21 1:43 ` H. Peter Anvin
0 siblings, 1 reply; 16+ messages in thread
From: Ian Kent @ 2003-11-21 0:51 UTC (permalink / raw)
To: H. Peter Anvin; +Cc: autofs mailing list
On Wed, 19 Nov 2003, H. Peter Anvin wrote:
> Ian Kent wrote:
> >
> > That sounds good a good idea for inclusion in 4.1.1 (I have quite a list
> > already).
> >
> > Move the decision on what the map contains to the program map implementer.
> > That would be fairly straight forward to implement given that I have a
> > placeholder function in each map module.
> >
>
> Well, it's pretty much the only thing that makes sense, since a program
> map can inherently do "anything".
>
> Pretty much ghosting implies that there are now two operations required
> of a map: key->data lookup (old), and enumerate all keys (new).
To fit in with the existing framework I would need to require the entire
map returned for a NULL key.
>
> And of course autofs needs to deal with the consequences of the map
> changing underneath it.
I normally reread the map and try again on lookup failure. There is the
HUP signal to do a manual map reread.
--
,-._|\ Ian Kent
/ \ Perth, Western Australia
*_.--._/ E-mail: raven@themaw.net
v Web: http://themaw.net/
^ permalink raw reply [flat|nested] 16+ messages in thread* Re: making the mountpoints visible all the time (fwd)
2003-11-21 0:51 ` Ian Kent
@ 2003-11-21 1:43 ` H. Peter Anvin
2003-11-21 6:59 ` Ian Kent
0 siblings, 1 reply; 16+ messages in thread
From: H. Peter Anvin @ 2003-11-21 1:43 UTC (permalink / raw)
To: Ian Kent; +Cc: autofs mailing list
Ian Kent wrote:
>
> To fit in with the existing framework I would need to require the entire
> map returned for a NULL key.
>
That's correct.
>
>>And of course autofs needs to deal with the consequences of the map
>>changing underneath it.
>
> I normally reread the map and try again on lookup failure. There is the
> HUP signal to do a manual map reread.
>
A better option is probably to update periodically. You still need to
be able to handle query of a mount point that you don't know about
(might have appeared anew) or a mount point you thought exist not being
found.
-hpa
^ permalink raw reply [flat|nested] 16+ messages in thread
* Re: making the mountpoints visible all the time (fwd)
2003-11-21 1:43 ` H. Peter Anvin
@ 2003-11-21 6:59 ` Ian Kent
2003-11-21 8:49 ` Ruslan U. Zakirov
` (2 more replies)
0 siblings, 3 replies; 16+ messages in thread
From: Ian Kent @ 2003-11-21 6:59 UTC (permalink / raw)
To: H. Peter Anvin; +Cc: autofs mailing list
On Thu, 20 Nov 2003, H. Peter Anvin wrote:
> Ian Kent wrote:
> >
> > To fit in with the existing framework I would need to require the entire
> > map returned for a NULL key.
> >
>
> That's correct.
>
> >
> >>And of course autofs needs to deal with the consequences of the map
> >>changing underneath it.
> >
> > I normally reread the map and try again on lookup failure. There is the
> > HUP signal to do a manual map reread.
> >
>
> A better option is probably to update periodically. You still need to
> be able to handle query of a mount point that you don't know about
> (might have appeared anew) or a mount point you thought exist not being
> found.
The above works that way without the update.
I'm not sure that a periodic update really gets us much. Since in most
sites maps change infrequently.
On the other hand at work we run a cron every night on all our machines to
trigger an update of the maps.
So I'm on the fence here?
--
,-._|\ Ian Kent
/ \ Perth, Western Australia
*_.--._/ E-mail: raven@themaw.net
v Web: http://themaw.net/
^ permalink raw reply [flat|nested] 16+ messages in thread* Re: making the mountpoints visible all the time (fwd)
2003-11-21 6:59 ` Ian Kent
@ 2003-11-21 8:49 ` Ruslan U. Zakirov
2003-11-22 7:14 ` Ian Kent
2003-11-21 19:52 ` H. Peter Anvin
2003-11-21 19:56 ` H. Peter Anvin
2 siblings, 1 reply; 16+ messages in thread
From: Ruslan U. Zakirov @ 2003-11-21 8:49 UTC (permalink / raw)
To: Ian Kent; +Cc: autofs mailing list, H. Peter Anvin
Ian Kent wrote:
>On Thu, 20 Nov 2003, H. Peter Anvin wrote:
>
>
>
>>Ian Kent wrote:
>>
>>
>>>To fit in with the existing framework I would need to require the entire
>>>map returned for a NULL key.
>>>
>>>
>>>
>>That's correct.
>>
>>
>>
>>>>And of course autofs needs to deal with the consequences of the map
>>>>changing underneath it.
>>>>
>>>>
>>>I normally reread the map and try again on lookup failure. There is the
>>>HUP signal to do a manual map reread.
>>>
>>>
>>>
>>A better option is probably to update periodically. You still need to
>>be able to handle query of a mount point that you don't know about
>>(might have appeared anew) or a mount point you thought exist not being
>>found.
>>
>>
>
>The above works that way without the update.
>
>I'm not sure that a periodic update really gets us much. Since in most
>sites maps change infrequently.
>
Runtime configureable? I think somebody can be interested in
'timeinterval updates',
some in 'just before request' or 'never'(could be done in map itself).
>
>On the other hand at work we run a cron every night on all our machines to
>trigger an update of the maps.
>
If I return to my task then I have two solutions:
1) User space daemon that cache all needed info about workgroup. So it
wouldn't be
much overhead to touch programm map on every request.
2) Programm map do all things only when it's called without arguments.
Sometimes
network can take much time to finish and I have to use periodicaly
update calls(~30min).
So I vote for runtime configuration of interval:
value > 0 - time period in seconds(msec,min or other)
value = 0 - every time
value < 0 - never
I think it's normal to configure it due userspace daemon startup.
>
>So I'm on the fence here?
>
>
>
Best regards. Ruslan.
^ permalink raw reply [flat|nested] 16+ messages in thread* Re: making the mountpoints visible all the time (fwd)
2003-11-21 8:49 ` Ruslan U. Zakirov
@ 2003-11-22 7:14 ` Ian Kent
0 siblings, 0 replies; 16+ messages in thread
From: Ian Kent @ 2003-11-22 7:14 UTC (permalink / raw)
To: Ruslan U. Zakirov; +Cc: autofs mailing list, H. Peter Anvin
On Fri, 21 Nov 2003, Ruslan U. Zakirov wrote:
> >
> >On the other hand at work we run a cron every night on all our machines to
> >trigger an update of the maps.
> >
> If I return to my task then I have two solutions:
> 1) User space daemon that cache all needed info about workgroup. So it
> wouldn't be
> much overhead to touch programm map on every request.
Not sure I understand this.
For maps that are ghosted the map is cached and lookups are done within
the cache. On a failure the external map is consulted and the cache
updated with any new map info.
That's the current implementation. So I would want fit in with this for
program maps as well.
> 2) Programm map do all things only when it's called without arguments.
> Sometimes
> network can take much time to finish and I have to use periodicaly
> update calls(~30min).
Sorry. The point here is not clear to me either.
My understanding of the current request (without timed updates) is that
1) Program map is called without arguments at startup to retrieve entire
map.
2) A lookup miss causes reread of entire map (program map is called
without arguments), to update cached map.
So the problem that needs to be solved is the map inconsistency when it
changes between an entire read and a cache miss.
>
> So I vote for runtime configuration of interval:
> value > 0 - time period in seconds(msec,min or other)
> value = 0 - every time
> value < 0 - never
Any periodic update would have to be a multiple of the timer signal. This
means that you would not get the exact update period. Do you think this
would be a problem?
--
,-._|\ Ian Kent
/ \ Perth, Western Australia
*_.--._/ E-mail: raven@themaw.net
v Web: http://themaw.net/
^ permalink raw reply [flat|nested] 16+ messages in thread
* Re: making the mountpoints visible all the time (fwd)
2003-11-21 6:59 ` Ian Kent
2003-11-21 8:49 ` Ruslan U. Zakirov
@ 2003-11-21 19:52 ` H. Peter Anvin
2003-11-21 19:56 ` H. Peter Anvin
2 siblings, 0 replies; 16+ messages in thread
From: H. Peter Anvin @ 2003-11-21 19:52 UTC (permalink / raw)
To: Ian Kent; +Cc: autofs mailing list
Ian Kent wrote:
>
> The above works that way without the update.
>
> I'm not sure that a periodic update really gets us much. Since in most
> sites maps change infrequently.
>
> On the other hand at work we run a cron every night on all our machines to
> trigger an update of the maps.
>
> So I'm on the fence here?
>
At some sites the maps change rather frequently. I'd let it be a
adjustable option, however, I think few sites would want to have more
than 24 hour delay, ever; most sites wouldn't want more than a few
hours, and some sites wouldn't want more than 5 minutes.
-hpa
^ permalink raw reply [flat|nested] 16+ messages in thread
* Re: making the mountpoints visible all the time (fwd)
2003-11-21 6:59 ` Ian Kent
2003-11-21 8:49 ` Ruslan U. Zakirov
2003-11-21 19:52 ` H. Peter Anvin
@ 2003-11-21 19:56 ` H. Peter Anvin
2003-11-22 7:19 ` Ian Kent
2 siblings, 1 reply; 16+ messages in thread
From: H. Peter Anvin @ 2003-11-21 19:56 UTC (permalink / raw)
To: Ian Kent; +Cc: autofs mailing list
Ian Kent wrote:
>
> The above works that way without the update.
>
> I'm not sure that a periodic update really gets us much. Since in most
> sites maps change infrequently.
>
> On the other hand at work we run a cron every night on all our machines to
> trigger an update of the maps.
>
> So I'm on the fence here?
>
Oh yes, a lot of map types support querying "has a change occurred"
(usually in the form of a generation number, or in the case of a file
map, the mtime.) This absolutely should be taken advantage of.
(For a program map, this complicates the protocol somewhat. I don't
know necessarily of the best way to deal with that. It needs some
thinking :)
-hpa
^ permalink raw reply [flat|nested] 16+ messages in thread
* Re: making the mountpoints visible all the time (fwd)
2003-11-21 19:56 ` H. Peter Anvin
@ 2003-11-22 7:19 ` Ian Kent
0 siblings, 0 replies; 16+ messages in thread
From: Ian Kent @ 2003-11-22 7:19 UTC (permalink / raw)
To: H. Peter Anvin; +Cc: autofs mailing list
On Fri, 21 Nov 2003, H. Peter Anvin wrote:
> Ian Kent wrote:
> >
> > The above works that way without the update.
> >
> > I'm not sure that a periodic update really gets us much. Since in most
> > sites maps change infrequently.
> >
> > On the other hand at work we run a cron every night on all our machines to
> > trigger an update of the maps.
> >
> > So I'm on the fence here?
> >
>
> Oh yes, a lot of map types support querying "has a change occurred"
> (usually in the form of a generation number, or in the case of a file
> map, the mtime.) This absolutely should be taken advantage of.
Like the netid (or whatever it's called in NIS).
Not sure about LDAP.
I seem to remember some timestamps live in each entry. I'll need to check.
>
> (For a program map, this complicates the protocol somewhat. I don't
> know necessarily of the best way to deal with that. It needs some
> thinking :)
A hard one I think.
I read the entire map to often as it is so I need to come up with
something.
This is shaping up to be quite an interesting proprosal.
--
,-._|\ Ian Kent
/ \ Perth, Western Australia
*_.--._/ E-mail: raven@themaw.net
v Web: http://themaw.net/
^ permalink raw reply [flat|nested] 16+ messages in thread
* RE: making the mountpoints visible all the time (fwd)
@ 2003-11-17 5:05 Chris Croswhite
2003-11-17 5:32 ` Ian Kent
2003-12-02 13:49 ` Ian Kent
0 siblings, 2 replies; 16+ messages in thread
From: Chris Croswhite @ 2003-11-17 5:05 UTC (permalink / raw)
To: Ian Kent, Alexander Macdonald; +Cc: autofs
I can offer up about 75 Linux boxes as testing. They are used pretty heavily mounting about 300-400 differnet mount points during the day from slowlaris, HPsUx, Linux and NetApp file systems. Just let me know.
TIA,
Chris
-----Original Message-----
From: Ian Kent [mailto:raven@themaw.net]
Sent: Sun 16-Nov-03 18:53
To: Alexander Macdonald
Cc: autofs@linux.kernel.org
Subject: Re: [autofs] making the mountpoints visible all the time (fwd)
On Sun, 16 Nov 2003, Alexander Macdonald wrote:
> I'm happy to try out any patches, although I'm only using autofs for
> about 20 or so samba and nfs shares on a few pcs so it wont exactly be
> thorough enterprise level testing :)
>
> >In fact, I don't have a 2.6 environment other than what I use to test at
> >home so volenteers are needed for testing. We can't expect the changes
> >to be accepted without some testing other than my simple setup.
> >
> >
Any help is appreciated.
I'll let you know when I've got something to test.
Thanks.
--
,-._|\ Ian Kent
/ \ Perth, Western Australia
*_.--._/ E-mail: raven@themaw.net
v Web: http://themaw.net/
_______________________________________________
autofs mailing list
autofs@linux.kernel.org
http://linux.kernel.org/mailman/listinfo/autofs
^ permalink raw reply [flat|nested] 16+ messages in thread* RE: making the mountpoints visible all the time (fwd)
2003-11-17 5:05 Chris Croswhite
@ 2003-11-17 5:32 ` Ian Kent
2003-12-02 13:49 ` Ian Kent
1 sibling, 0 replies; 16+ messages in thread
From: Ian Kent @ 2003-11-17 5:32 UTC (permalink / raw)
To: Chris Croswhite; +Cc: autofs
On Sun, 16 Nov 2003, Chris Croswhite wrote:
> I can offer up about 75 Linux boxes as testing. They are used pretty heavily mounting about 300-400 differnet mount points during the day from slowlaris, HPsUx, Linux and NetApp file systems. Just let me know.
>
Will do.
This is for testing my 2.6 patch when it's done. Yes?
--
,-._|\ Ian Kent
/ \ Perth, Western Australia
*_.--._/ E-mail: raven@themaw.net
v Web: http://themaw.net/
^ permalink raw reply [flat|nested] 16+ messages in thread* RE: making the mountpoints visible all the time (fwd)
2003-11-17 5:05 Chris Croswhite
2003-11-17 5:32 ` Ian Kent
@ 2003-12-02 13:49 ` Ian Kent
1 sibling, 0 replies; 16+ messages in thread
From: Ian Kent @ 2003-12-02 13:49 UTC (permalink / raw)
To: Chris Croswhite; +Cc: autofs
Hi guys,
I have some 2.6 patches.
They can be found at
http://www.kernel.org/pub/linux/kernel/people/raven/autofs4-2.6
Apply them in number order please.
Please leave out the max_anon_2.patch initially. We need to establish what
the status of the kernel module is before trying this patch. It was
written by Mike Waychison to increase the number of anonymous devices.
Once the status patches is established it would be good to how the
max_anon patch goes with over 255 mounts and less than about 790 (rpc
doesn't allow more than that at the moment).
Try as many or as few things as your setup allows and please try and give
some coverage of what was used and what went wrong. We need to try this on
SMP setups if at all possible.
I know I don't need to say this to ypu guys but I will anyway. Please make
sure that you do staged testing. What I'm saying is steady as it goes.
First a single machine then a couple etc., you know the routine. While the
general approach of the patches is the same as what I have used for ages
in 2.4 the locking and synchronisation is different and may not be
correct (read I think it's not quite right but need evidence to confirm
it).
Looking forward to your feedback.
On Sun, 16 Nov 2003, Chris Croswhite wrote:
> I can offer up about 75 Linux boxes as testing. They are used pretty heavily mounting about 300-400 differnet mount points during the day from slowlaris, HPsUx, Linux and NetApp file systems. Just let me know.
>
> TIA,
> Chris
>
>
> -----Original Message-----
> From: Ian Kent [mailto:raven@themaw.net]
> Sent: Sun 16-Nov-03 18:53
> To: Alexander Macdonald
> Cc: autofs@linux.kernel.org
> Subject: Re: [autofs] making the mountpoints visible all the time (fwd)
> On Sun, 16 Nov 2003, Alexander Macdonald wrote:
>
> > I'm happy to try out any patches, although I'm only using autofs for
> > about 20 or so samba and nfs shares on a few pcs so it wont exactly be
> > thorough enterprise level testing :)
> >
> > >In fact, I don't have a 2.6 environment other than what I use to test at
> > >home so volenteers are needed for testing. We can't expect the changes
> > >to be accepted without some testing other than my simple setup.
> > >
> > >
>
> Any help is appreciated.
>
> I'll let you know when I've got something to test.
>
> Thanks.
>
>
--
,-._|\ Ian Kent
/ \ Perth, Western Australia
*_.--._/ E-mail: raven@themaw.net
v Web: http://themaw.net/
^ permalink raw reply [flat|nested] 16+ messages in thread
* RE: making the mountpoints visible all the time (fwd)
@ 2003-11-16 16:05 Ian Kent
2003-11-16 19:58 ` Alexander Macdonald
0 siblings, 1 reply; 16+ messages in thread
From: Ian Kent @ 2003-11-16 16:05 UTC (permalink / raw)
To: autofs mailing list
---------- Forwarded message ----------
Date: 16 Nov 2003 23:56:22 +0800
From: Ian Kent <raven@themaw.net>
To: "Rigler, Steve" <SRigler@MarathonOil.com>
Subject: RE: [autofs] making the mountpoints visible all the time
On Sat, 2003-11-15 at 05:34, Rigler, Steve wrote:
> > -----Original Message-----
> > From: autofs-bounces@linux.kernel.org
> > [mailto:autofs-bounces@linux.kernel.org]On Behalf Of
> > Alexander Macdonald
> > Sent: Friday, November 14, 2003 2:20 PM
> > To: autofs@linux.kernel.org
> > Subject: Re: [autofs] making the mountpoints visible all the time
> >
> >
> > I'm using kernel 2.6.0-test9, i could only see patches for 2.4 so i
> > guess this has the latest autofs in it allready right? I've also
> > upgraded my daemon to autofs-4.1.0-beta3. I then tried running this:-
> >
> > automount -g /mnt/network/samba/computerA file /etc/autofs/samba
> >
> > in the file /etc/autofs/samba I have this:-
> >
> > download -fstype=smbfs,username=guest,password=none
> > ://computera/Download
> > upload -fstype=smbfs,username=guest,rw,password=none
> > ://computera/Upload
> >
> > it works fine but it still doesnt do the "ghosting"
> >
> >
There is no port of these changes for 2.6 yet.
I'm working on it and hope to submit my changes within a week. Hopefully
I'll get something into the kernel.
In fact, I don't have a 2.6 environment other than what I use to test at
home so volenteers are needed for testing. We can't expect the changes
to be accepted without some testing other than my simple setup.
>
> I haven't tried the 2.6 kernel yet, so anything I suggest would be
> based on experience with the 2.4 kernel.
>
> One thing is to make sure you have an entry in /etc/modules.conf
> to force loading of the autofs4 module.
>
> It should look like:
> alias autofs autofs4
Yes.
The 4.1.0 betas contain an init script which checks and loads the
autofs4 kernel module first (ie. before autofs) so this shouldn't be
needed however it can't hurt.
--
,-._|\ Ian Kent
/ \ Perth, Western Australia
*_.--._/ E-mail: raven@themaw.net
v Web: http://themaw.net/
^ permalink raw reply [flat|nested] 16+ messages in thread* Re: making the mountpoints visible all the time (fwd)
2003-11-16 16:05 Ian Kent
@ 2003-11-16 19:58 ` Alexander Macdonald
2003-11-17 2:53 ` Ian Kent
0 siblings, 1 reply; 16+ messages in thread
From: Alexander Macdonald @ 2003-11-16 19:58 UTC (permalink / raw)
To: autofs
I'm happy to try out any patches, although I'm only using autofs for
about 20 or so samba and nfs shares on a few pcs so it wont exactly be
thorough enterprise level testing :)
>In fact, I don't have a 2.6 environment other than what I use to test at
>home so volenteers are needed for testing. We can't expect the changes
>to be accepted without some testing other than my simple setup.
>
>
^ permalink raw reply [flat|nested] 16+ messages in thread
* Re: making the mountpoints visible all the time (fwd)
2003-11-16 19:58 ` Alexander Macdonald
@ 2003-11-17 2:53 ` Ian Kent
0 siblings, 0 replies; 16+ messages in thread
From: Ian Kent @ 2003-11-17 2:53 UTC (permalink / raw)
To: Alexander Macdonald; +Cc: autofs
On Sun, 16 Nov 2003, Alexander Macdonald wrote:
> I'm happy to try out any patches, although I'm only using autofs for
> about 20 or so samba and nfs shares on a few pcs so it wont exactly be
> thorough enterprise level testing :)
>
> >In fact, I don't have a 2.6 environment other than what I use to test at
> >home so volenteers are needed for testing. We can't expect the changes
> >to be accepted without some testing other than my simple setup.
> >
> >
Any help is appreciated.
I'll let you know when I've got something to test.
Thanks.
--
,-._|\ Ian Kent
/ \ Perth, Western Australia
*_.--._/ E-mail: raven@themaw.net
v Web: http://themaw.net/
^ permalink raw reply [flat|nested] 16+ messages in thread
end of thread, other threads:[~2003-12-02 13:49 UTC | newest]
Thread overview: 16+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2003-11-19 8:13 making the mountpoints visible all the time (fwd) Ian Kent
2003-11-19 16:45 ` H. Peter Anvin
2003-11-21 0:51 ` Ian Kent
2003-11-21 1:43 ` H. Peter Anvin
2003-11-21 6:59 ` Ian Kent
2003-11-21 8:49 ` Ruslan U. Zakirov
2003-11-22 7:14 ` Ian Kent
2003-11-21 19:52 ` H. Peter Anvin
2003-11-21 19:56 ` H. Peter Anvin
2003-11-22 7:19 ` Ian Kent
-- strict thread matches above, loose matches on Subject: below --
2003-11-17 5:05 Chris Croswhite
2003-11-17 5:32 ` Ian Kent
2003-12-02 13:49 ` Ian Kent
2003-11-16 16:05 Ian Kent
2003-11-16 19:58 ` Alexander Macdonald
2003-11-17 2:53 ` Ian Kent
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.