hotplugd - event serializer and multiplexer

hotplugd - event serializer and multiplexer

[Kay Sievers]

Hi,
I've took our sweet udevd and made a generic hotplug event serializer.
It can replace the current /sbin/hotplug multiplexer bash script.

hotplugsend is invoked by the kernel and sends the data the kernel
provides over a domain socket to the hotplugd daemon. The daemon will
be started if not already running.

The hotplugd daemon reorders the events it receives, handles timeouts if
sequences are missing and executes all programs in /etc/hotplug.d/<subsys>/
thereafter in /etc/hotplug.d/default/ in each case in lexical order.

The programs are executed in the background, but only _one_ program runs
at any time for a single DEVPATH. No longer need for the various 'sleep 2'
all over the scripts. It's possible now to order the programs, so that
udev has already created all nodes, before the scripts are called.
udevd is no longer neccessary, we can call udev directly.

The maximum count of forked processes may be limited and is currently set
to 300.

It doesn't depend on any shell and therefore should be very fast for
a lot of devices connected at once. Hmm, I have only 4 :)

I know that most of us don't like a daemon for general hotplug, but I
wanted to try it. I expect more and more problems with the
syncronization of the hotplug scripts, so this is a possible solution.
No later event will ever beat the earlier for one device.

I've put this together today in 5 hours, so expect a very experimental
piece of code :)

Anybody interested, may look at it here:

  http://vrfy.org/projects/hotplugd/

thanks,
Kay


-------------------------------------------------------
This SF.Net email is sponsored by: IBM Linux Tutorials
Free Linux tutorial presented by Daniel Robbins, President and CEO of
GenToo technologies. Learn everything from fundamentals to system
administration.http://ads.osdn.com/?ad_id\x1470&alloc_id638&op=click
_______________________________________________
Linux-hotplug-devel mailing list  http://linux-hotplug.sourceforge.net
Linux-hotplug-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/linux-hotplug-devel

Re: hotplugd - event serializer and multiplexer

[Patrick Mansfield]

On Sun, Mar 07, 2004 at 10:37:57PM +0100, Kay Sievers wrote:
> Hi,
> I've took our sweet udevd and made a generic hotplug event serializer.
> It can replace the current /sbin/hotplug multiplexer bash script.
> 
> hotplugsend is invoked by the kernel and sends the data the kernel
> provides over a domain socket to the hotplugd daemon. The daemon will
> be started if not already running.
> 
> The hotplugd daemon reorders the events it receives, handles timeouts if
> sequences are missing and executes all programs in /etc/hotplug.d/<subsys>/
> thereafter in /etc/hotplug.d/default/ in each case in lexical order.

Sounds interesting. I was thinking of trying out the C version of the
/sbin/hotplug.

I was trying some tests with many devices (about 100 to 1000) using
scsi_debug, it was fairly slow, but I have not compared with/without
hotplug. I was going to check on the speeds with/without many udev.rules.
(I'm running on an older box, NUMAQ with 8 500 Mhz procesoor with 8gb of
memory).

I could see many scsi.agent processes running, and scsi.agent is not
optimal - all it does is try to load sd_mod 1000 times, but sd is built
into my kernel; and on remove, we invoke scsi.agent do set up some minor
things, and then do nothing.

> The programs are executed in the background, but only _one_ program runs
> at any time for a single DEVPATH. No longer need for the various 'sleep 2'
> all over the scripts. It's possible now to order the programs, so that
> udev has already created all nodes, before the scripts are called.
> udevd is no longer neccessary, we can call udev directly.

How does serializing these avoid kernel races in creating attributes after
the hotplug has been sent?

> The maximum count of forked processes may be limited and is currently set
> to 300.

That sounds high ;-)

> It doesn't depend on any shell and therefore should be very fast for
> a lot of devices connected at once. Hmm, I have only 4 :)

Use scsi_debug. Build it as a module and load it via:

/sbin/modprobe scsi_debug num_tgts=X max_luns=Y delay=T

Where X * Y is the number of disks that will get created. The delay T
defaults to 1 milliseconds, changing this to 0 does not help as much as
you would think (probably because we have to wake up ksoftirqd to handle
the phony IO completions, rather than run them off of a timeout).

The sysfs entries can use up quite a bit of memory.

If you want to go above 256, you need a sd many patch, and a glibc with
major/minor fixes (I have not figured out how to update my debian glibc
for the new major/minor usage), but if you build udev with klibc it will
get the newer makedev code).

i.e. my glibc gives this:

elm3b79:~# mknod /tmp/sdcon-2 b 8 768
elm3b79:~# ls -l /tmp/sdcon-2
brw-r--r--    1 root     root      11,   0 Mar  5 14:26 /tmp/sdcon-2

I used Kurt's sd many patch (I don't know what the latest status is on
this patch going into the kernel, see Kurt's last post on the subject, it
might be in mm):

http://marc.theaimsgroup.com/?l=linux-scsi&m\x107663093228380&w=2

> I know that most of us don't like a daemon for general hotplug, but I
> wanted to try it. I expect more and more problems with the
> syncronization of the hotplug scripts, so this is a possible solution.
> No later event will ever beat the earlier for one device.
> 
> I've put this together today in 5 hours, so expect a very experimental
> piece of code :)
> 
> Anybody interested, may look at it here:
> 
>   http://vrfy.org/projects/hotplugd/


-------------------------------------------------------
This SF.Net email is sponsored by: IBM Linux Tutorials
Free Linux tutorial presented by Daniel Robbins, President and CEO of
GenToo technologies. Learn everything from fundamentals to system
administration.http://ads.osdn.com/?ad_id\x1470&alloc_id638&op=click
_______________________________________________
Linux-hotplug-devel mailing list  http://linux-hotplug.sourceforge.net
Linux-hotplug-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/linux-hotplug-devel

Re: hotplugd - event serializer and multiplexer

[Kay Sievers]

On Mon, 2004-03-08 at 18:06, Patrick Mansfield wrote:
> On Sun, Mar 07, 2004 at 10:37:57PM +0100, Kay Sievers wrote:

> > The programs are executed in the background, but only _one_ program runs
> > at any time for a single DEVPATH. No longer need for the various 'sleep 2'
> > all over the scripts. It's possible now to order the programs, so that
> > udev has already created all nodes, before the scripts are called.
> > udevd is no longer neccessary, we can call udev directly.
> 
> How does serializing these avoid kernel races in creating attributes after
> the hotplug has been sent?

Basically it's just udevd moved to /sbin/hotplug with the ability to run
more programs than the single udev. If udev is the first program which
is executed with a hotplug event, all other programs wait until udev is
ready waiting for the attibutes and creating the nodes. So any program
that run _after_ udev can be sure that everything is in place.

Maybe we need some more folders in /etc/hotplug.d/. One with programs
executed anytime before the /etc/hotplug.d/<subsys>/ scripts and 
default/ only if no <subsys>/ is found?

> > It doesn't depend on any shell and therefore should be very fast for
> > a lot of devices connected at once. Hmm, I have only 4 :)
> 
> Use scsi_debug. Build it as a module and load it via:
> 
> /sbin/modprobe scsi_debug num_tgts=X max_luns=Y delay=T
> 
> Where X * Y is the number of disks that will get created. The delay T
> defaults to 1 milliseconds, changing this to 0 does not help as much as
> you would think (probably because we have to wake up ksoftirqd to handle
> the phony IO completions, rather than run them off of a timeout).
> 
> The sysfs entries can use up quite a bit of memory.

Nice, I will give it a try later tonight.

thanks,
Kay



-------------------------------------------------------
This SF.Net email is sponsored by: IBM Linux Tutorials
Free Linux tutorial presented by Daniel Robbins, President and CEO of
GenToo technologies. Learn everything from fundamentals to system
administration.http://ads.osdn.com/?ad_id\x1470&alloc_id638&op=click
_______________________________________________
Linux-hotplug-devel mailing list  http://linux-hotplug.sourceforge.net
Linux-hotplug-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/linux-hotplug-devel

Re: hotplugd - event serializer and multiplexer

[Olaf Hering]

 On Mon, Mar 08, Patrick Mansfield wrote:

> How does serializing these avoid kernel races in creating attributes after
> the hotplug has been sent?

Why do you think that fixing kernel bugs in userspace is a desired
thing? The kernel must no call the usermode helper until all attributes
are available, this includes the busid in /sys/bus/<type>/devices/<id>
Maybe I miss something, but why would anyone want to be notified that
early?

-- 
USB is for mice, FireWire is for men!

sUse lINUX ag, n√úRNBERG


-------------------------------------------------------
This SF.Net email is sponsored by: IBM Linux Tutorials
Free Linux tutorial presented by Daniel Robbins, President and CEO of
GenToo technologies. Learn everything from fundamentals to system
administration.http://ads.osdn.com/?ad_id\x1470&alloc_id638&opÃk
_______________________________________________
Linux-hotplug-devel mailing list  http://linux-hotplug.sourceforge.net
Linux-hotplug-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/linux-hotplug-devel