waiting for an unknown set of udev /dev entries to complete

waiting for an unknown set of udev /dev entries to complete

[Patrick Mansfield]

Any ideas on the best way to wait for udev to finish handling a set of
hotplug events?

That is, if I load a module or otherwise generate an unknown number of
hotplug events for a single action or event - like a modprobe of a scsi
HBA or even modprobe sd_mod - how can I tell when all hotplug events have
been handled and /dev entries created?

I'm not asking how to wait for a specific /dev entry or a single hotplug
event for a particular device.

I was thinking of:

1) (I'm looking for scsi block devices) Waiting for an equal number of
/dev/sd* nodes to show up as is found under /sys/block/sd* and
/sys/block/sd*/sd*. Is specific, and failure modes are kind of ugly (a
timeout could be used, but if hotplug is really slow and is making
progress you don't want to stop waiting).

2) Waiting for all hotplug processes to finish.  This is more general
purpose (not scsi or block specific, but doesn't handle children of
kernel.hotplug that are not named the same (at least my script below has
this problem).

I can't think of a way to handle this in a udev rule, since those are too
general (i.e. they don't know what the last event should be).

For 1), a shell script like this works:

-------------------------------------------
#! /bin/bash

# load modules, iscsi login, etc. here (does NOT have to be run in the
# background)
#
# I am using:
# ~patman/iscsi/open-iscsi/usr/iscsiadm  -m node --record 22698c --login

count=$(ls -d /sys/block/sd* /sys/block/sd*/sd* | wc -l)
devsd=$(ls /dev/sd* | wc -l)
while [[ ${devsd} -lt ${count} ]]
do
	echo $(date) waiting count ${count} devsd ${devsd}
	sleep 1
	devsd=$(ls /dev/sd* | wc -l)
done

-------------------------------------------

And for 2): Again, could fail if the hotplug handler forks and returns,
leaving its child running. Shell script like:

-------------------------------------------

#! /bin/bash

# load modules, iscsi login etc here
# ~patman/iscsi/open-iscsi/usr/iscsiadm  -m node --record 22698c --login

hotplug=$(sysctl kernel.hotplug)
while /sbin/pidof -s ${hotplug} > /dev/null 2>&1
do
	echo waiting $(date) ...
	sleep 1
done

-------------------------------------------

More background:

Module loads should check this during boot and install, but I haven't
found such checks (so far, I see some stuff in suse network code but that
is for networks). Most cases will just work for a small number of disks
and a sufficient interval between module load and mounting (for boot) or
use (for an install) of the intended device.

With current FC rawhide/devel, I am working on iscsi install, and had udev
"error" logging enabled on a medium speed dual processor system. So the
scripts are very slow, and the partition setup code is being run right
after iscsi scan (really discovery/login and then scan) and it does not
see all the devices (and I probably have some bugs still). 

It takes over a second per scsi sd, and I have about 60 of them.

I actually have to write this in python ... maybe an
os.system("everything") call.

-- Patrick Mansfield


-------------------------------------------------------
This SF.Net email is sponsored by the JBoss Inc.  Get Certified Today
Register for a JBoss Training Course.  Free Certification Exam
for All Training Attendees Through End of 2005. For more info visit:
http://ads.osdn.com/?ad_idv28&alloc_id\x16845&op=click
_______________________________________________
Linux-hotplug-devel mailing list  http://linux-hotplug.sourceforge.net
Linux-hotplug-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/linux-hotplug-devel

Re: waiting for an unknown set of udev /dev entries to complete

[Kay Sievers]

On Fri, Nov 18, 2005 at 02:30:45PM -0800, Patrick Mansfield wrote:
> Any ideas on the best way to wait for udev to finish handling a set of
> hotplug events?

I have changes in my udev tree, to export the udevd event queue. It
will probably be in 076 - if it works as expected. For every queued
or running event, a symlink in /dev/.udev/queue/ is created and removed
after the udev event process has exited. For all failing udev events,
the queue files are moved to /dev/.udev/failed.

We want this for coldplug after bootup, to try to trigger failed events
again, if requirements are fulfilled only at a later stage of the boot
process.

  kay@pim:~> tree /dev/.udev
  /dev/.udev
  |-- db
  |   |-- block@sda
  |   |-- block@sda@sda1
  ...
  |-- failed
  |   `-- class@net@sit0 -> /sys/class/net/sit0
  `-- queue
      |-- block@sda -> /sys/block/sda
      |-- class@input@input0 -> /sys/class/input/input0
  ...

> That is, if I load a module or otherwise generate an unknown number of
> hotplug events for a single action or event - like a modprobe of a scsi
> HBA or even modprobe sd_mod - how can I tell when all hotplug events have
> been handled and /dev entries created?

You should be able to look for /dev/.udev/queue/*block*, and see if
there are events still running.

> I'm not asking how to wait for a specific /dev entry or a single hotplug
> event for a particular device.

Sure.

> I was thinking of:
> 
> 1) (I'm looking for scsi block devices) Waiting for an equal number of
> /dev/sd* nodes to show up as is found under /sys/block/sd* and
> /sys/block/sd*/sd*. Is specific, and failure modes are kind of ugly (a
> timeout could be used, but if hotplug is really slow and is making
> progress you don't want to stop waiting).
> 
> 2) Waiting for all hotplug processes to finish.  This is more general
> purpose (not scsi or block specific, but doesn't handle children of
> kernel.hotplug that are not named the same (at least my script below has
> this problem).

You may not detach from the event process for short living programs,
that will leave the event in the queue. If you need to do really complicated
things, you may need to synchronize the events yourself at a higher level.

> I can't think of a way to handle this in a udev rule, since those are too
> general (i.e. they don't know what the last event should be).

Yes, there is no such logic.

> -------------------------------------------
> #! /bin/bash
> 
> # load modules, iscsi login, etc. here (does NOT have to be run in the
> # background)
> #
> # I am using:
> # ~patman/iscsi/open-iscsi/usr/iscsiadm  -m node --record 22698c --login
> 
> count=$(ls -d /sys/block/sd* /sys/block/sd*/sd* | wc -l)
> devsd=$(ls /dev/sd* | wc -l)
> while [[ ${devsd} -lt ${count} ]]
> do
> 	echo $(date) waiting count ${count} devsd ${devsd}
> 	sleep 1
> 	devsd=$(ls /dev/sd* | wc -l)
> done
> 
> -------------------------------------------

Should work fine with the exported event queue.

> And for 2): Again, could fail if the hotplug handler forks and returns,
> leaving its child running. Shell script like:
> 
> -------------------------------------------
> 
> #! /bin/bash
> 
> # load modules, iscsi login etc here
> # ~patman/iscsi/open-iscsi/usr/iscsiadm  -m node --record 22698c --login
> 
> hotplug=$(sysctl kernel.hotplug)
> while /sbin/pidof -s ${hotplug} > /dev/null 2>&1
> do
> 	echo waiting $(date) ...
> 	sleep 1
> done
> 
> -------------------------------------------

There is no "kernel.hotplug" that runs anything. Usually it's "udevsend",
which just passes the event to the udev daemon. On SUSE it's disabled
completely for a long time.
The udev daemon listens to netlink and /sbin/hotplug does almost nothing
on all recent systems. It is only used for the "input" system which lacked
driver core support until 2.6.15.

> More background:
> 
> Module loads should check this during boot and install, but I haven't
> found such checks (so far, I see some stuff in suse network code but that
> is for networks). Most cases will just work for a small number of disks
> and a sufficient interval between module load and mounting (for boot) or
> use (for an install) of the intended device.

What do you mean with: "Module loads should check this" in this case?

> With current FC rawhide/devel, I am working on iscsi install, and had udev
> "error" logging enabled on a medium speed dual processor system. So the
> scripts are very slow, and the partition setup code is being run right
> after iscsi scan (really discovery/login and then scan) and it does not
> see all the devices (and I probably have some bugs still). 

Hmm, care to explain that better, I don't really understand...

> It takes over a second per scsi sd, and I have about 60 of them.

But these are not serialized, are they?

Best,
Kay


-------------------------------------------------------
This SF.Net email is sponsored by the JBoss Inc.  Get Certified Today
Register for a JBoss Training Course.  Free Certification Exam
for All Training Attendees Through End of 2005. For more info visit:
http://ads.osdn.com/?ad_idv28&alloc_id\x16845&op=click
_______________________________________________
Linux-hotplug-devel mailing list  http://linux-hotplug.sourceforge.net
Linux-hotplug-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/linux-hotplug-devel

Re: waiting for an unknown set of udev /dev entries to complete

[Patrick Mansfield]

Hi Kay ...

On Sun, Nov 20, 2005 at 07:03:57AM +0100, Kay Sievers wrote:
> On Fri, Nov 18, 2005 at 02:30:45PM -0800, Patrick Mansfield wrote:

> > That is, if I load a module or otherwise generate an unknown number of
> > hotplug events for a single action or event - like a modprobe of a scsi
> > HBA or even modprobe sd_mod - how can I tell when all hotplug events have
> > been handled and /dev entries created?
> 
> You should be able to look for /dev/.udev/queue/*block*, and see if
> there are events still running.

OK, (I think) I need this for current fc devel, udev 075, so I'll use the
method to wait for all /dev/sd's to show up.

If udev logging is off, I might not even hit this problem. For slow
devices (perhaps iSCSI for a congested network and a slow responding
target/disk), it could still be a problem.

> > 1) (I'm looking for scsi block devices) Waiting for an equal number of
> > /dev/sd* nodes to show up as is found under /sys/block/sd* and
> > /sys/block/sd*/sd*. Is specific, and failure modes are kind of ugly (a
> > timeout could be used, but if hotplug is really slow and is making
> > progress you don't want to stop waiting).
> > 
> > 2) Waiting for all hotplug processes to finish.  This is more general
> > purpose (not scsi or block specific, but doesn't handle children of
> > kernel.hotplug that are not named the same (at least my script below has
> > this problem).
> 
> You may not detach from the event process for short living programs,
> that will leave the event in the queue. If you need to do really complicated
> things, you may need to synchronize the events yourself at a higher level.

You mean you can't wait for a particular process to finish? 

Otherwise, I don't understand how the "event queue" relates to the
problem. I see udev has a list of msg's. It forks udev_event_process, it
will exec multiple run programs. When udev_event_process, reap_sigchilds()
calls udev_done() and it removes the msg.

> There is no "kernel.hotplug" that runs anything. Usually it's "udevsend",
> which just passes the event to the udev daemon. On SUSE it's disabled
> completely for a long time.

OK, I should have remembered that (even though I'm not running SUSE) ...

> The udev daemon listens to netlink and /sbin/hotplug does almost nothing
> on all recent systems. It is only used for the "input" system which lacked
> driver core support until 2.6.15.

> > Module loads should check this during boot and install, but I haven't
> > found such checks (so far, I see some stuff in suse network code but that
> > is for networks). Most cases will just work for a small number of disks
> > and a sufficient interval between module load and mounting (for boot) or
> > use (for an install) of the intended device.
> 
> What do you mean with: "Module loads should check this" in this case?

After you load a module, like a SCSI HBA driver or even sd_mod, you should
really wait for the devices to show up before you try and use them.  For
initrd, this means you should wait for the root /dev to show up (if it has
a /sys/block/XX entry); for installs, this means wait for all /dev entries
to show up.

People have asked about this for general hotplug (plugging in a device,
loading a module, then immediately trying to access the /dev/XXX node),
but I haven't seen reports of problem for this on normal boot (initrd root
mounting) or for installs, maybe iSCSI makes hitting this more likely.

> > With current FC rawhide/devel, I am working on iscsi install, and had udev
> > "error" logging enabled on a medium speed dual processor system. So the
> > scripts are very slow, and the partition setup code is being run right
> > after iscsi scan (really discovery/login and then scan) and it does not
> > see all the devices (and I probably have some bugs still). 
> 
> Hmm, care to explain that better, I don't really understand...

Just that udev /dev creation is slow with udev_log="err", so after a
login to an iSCSI target completes (iSCSI tools issue a scan of the target
via user space), the /dev entries are not available for about 60 seconds.

So if immediately after the iSCSI login, if we look for or try to use /dev
block install targets, we can have problems.

> > It takes over a second per scsi sd, and I have about 60 of them.
> 
> But these are not serialized, are they?

Right. I can see multiple processes running. I don't know why they are
_so_ slow. With udev_log=info, they run fairly fast (i.e. I don't think
there is a problem), but with udev_log=err, it's really slow, so I guess
all that the logging is the bottleneck (maybe some fsync in syslog?).

How can I change the udev_log level without rebooting?

-- Patrick Mansfield


-------------------------------------------------------
This SF.Net email is sponsored by the JBoss Inc.  Get Certified Today
Register for a JBoss Training Course.  Free Certification Exam
for All Training Attendees Through End of 2005. For more info visit:
http://ads.osdn.com/?ad_idv28&alloc_id\x16845&op=click
_______________________________________________
Linux-hotplug-devel mailing list  http://linux-hotplug.sourceforge.net
Linux-hotplug-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/linux-hotplug-devel

Re: waiting for an unknown set of udev /dev entries to complete

[Kay Sievers]

On Sun, Nov 20, 2005 at 09:25:29AM -0800, Patrick Mansfield wrote:
> On Sun, Nov 20, 2005 at 07:03:57AM +0100, Kay Sievers wrote:
> > On Fri, Nov 18, 2005 at 02:30:45PM -0800, Patrick Mansfield wrote:
> 
> > > That is, if I load a module or otherwise generate an unknown number of
> > > hotplug events for a single action or event - like a modprobe of a scsi
> > > HBA or even modprobe sd_mod - how can I tell when all hotplug events have
> > > been handled and /dev entries created?
> > 
> > You should be able to look for /dev/.udev/queue/*block*, and see if
> > there are events still running.
> 
> OK, (I think) I need this for current fc devel, udev 075, so I'll use the
> method to wait for all /dev/sd's to show up.

It will hit Fedora devel pretty soon after the release, I expect.

> If udev logging is off, I might not even hit this problem. For slow
> devices (perhaps iSCSI for a congested network and a slow responding
> target/disk), it could still be a problem.

Hmm, we should better handle this.

> > > 1) (I'm looking for scsi block devices) Waiting for an equal number of
> > > /dev/sd* nodes to show up as is found under /sys/block/sd* and
> > > /sys/block/sd*/sd*. Is specific, and failure modes are kind of ugly (a
> > > timeout could be used, but if hotplug is really slow and is making
> > > progress you don't want to stop waiting).
> > > 
> > > 2) Waiting for all hotplug processes to finish.  This is more general
> > > purpose (not scsi or block specific, but doesn't handle children of
> > > kernel.hotplug that are not named the same (at least my script below has
> > > this problem).
> > 
> > You may not detach from the event process for short living programs,
> > that will leave the event in the queue. If you need to do really complicated
> > things, you may need to synchronize the events yourself at a higher level.
> 
> You mean you can't wait for a particular process to finish? 
> 
> Otherwise, I don't understand how the "event queue" relates to the
> problem. I see udev has a list of msg's. It forks udev_event_process, it
> will exec multiple run programs. When udev_event_process, reap_sigchilds()
> calls udev_done() and it removes the msg.

Yes, but if you try to resolve complex dependencies with udev event
processes itself, like let the event look at the queue, you have to be
very careful no to run into weird locking situations. It may work,
but I never tried this.
Today, only events for the parent devices or the underlying physical
devices are serialized by the udev daemon.

It may be better to hook asynchronously in the events for the actual
device events which have been created by the module load, instead of
just to continue with the setup script after loading the modules.
Depends on your setup, if that is feasible.

> > There is no "kernel.hotplug" that runs anything. Usually it's "udevsend",
> > which just passes the event to the udev daemon. On SUSE it's disabled
> > completely for a long time.
> 
> OK, I should have remembered that (even though I'm not running SUSE) ...

It is this way on Fedora since HAL depends on udev to cover the sysfs timing
issues.

> > The udev daemon listens to netlink and /sbin/hotplug does almost nothing
> > on all recent systems. It is only used for the "input" system which lacked
> > driver core support until 2.6.15.
> 
> > > Module loads should check this during boot and install, but I haven't
> > > found such checks (so far, I see some stuff in suse network code but that
> > > is for networks). Most cases will just work for a small number of disks
> > > and a sufficient interval between module load and mounting (for boot) or
> > > use (for an install) of the intended device.
> > 
> > What do you mean with: "Module loads should check this" in this case?
> 
> After you load a module, like a SCSI HBA driver or even sd_mod, you should
> really wait for the devices to show up before you try and use them.  For
> initrd, this means you should wait for the root /dev to show up (if it has
> a /sys/block/XX entry); for installs, this means wait for all /dev entries
> to show up.
> 
> People have asked about this for general hotplug (plugging in a device,
> loading a module, then immediately trying to access the /dev/XXX node),
> but I haven't seen reports of problem for this on normal boot (initrd root
> mounting) or for installs, maybe iSCSI makes hitting this more likely.

Yes, if you know what you are waiting for, like the root device, you
just may spin until it arrives, for all other cases, watching the event queue
should work pretty well.

> > > With current FC rawhide/devel, I am working on iscsi install, and had udev
> > > "error" logging enabled on a medium speed dual processor system. So the
> > > scripts are very slow, and the partition setup code is being run right
> > > after iscsi scan (really discovery/login and then scan) and it does not
> > > see all the devices (and I probably have some bugs still). 
> > 
> > Hmm, care to explain that better, I don't really understand...
> 
> Just that udev /dev creation is slow with udev_log="err", so after a
> login to an iSCSI target completes (iSCSI tools issue a scan of the target
> via user space), the /dev entries are not available for about 60 seconds.
> 
> So if immediately after the iSCSI login, if we look for or try to use /dev
> block install targets, we can have problems.
> 
> > > It takes over a second per scsi sd, and I have about 60 of them.
> > 
> > But these are not serialized, are they?
> 
> Right. I can see multiple processes running. I don't know why they are
> _so_ slow. With udev_log=info, they run fairly fast (i.e. I don't think
> there is a problem), but with udev_log=err, it's really slow, so I guess
> all that the logging is the bottleneck (maybe some fsync in syslog?).

Yes, prefix the logfile in syslog.conf with a "-", which is not the default
on Fedora, and this will go away.

> How can I change the udev_log level without rebooting?

You can change the udev daemons internal setting with:
  udevcontrol log_priority=<level/number>

And you may want to add:
  RUN+="socket:/org/kernel/udev/monitor"
as one of the last rules to your udev rule set, I think Fedora does not
have this by default. Then run:
  udevmonitor
and watch the exact event timing - from the kernel event, until the actual
udev event was handled. That is a lot easier than reading syslog.

Kay


-------------------------------------------------------
This SF.Net email is sponsored by the JBoss Inc.  Get Certified Today
Register for a JBoss Training Course.  Free Certification Exam
for All Training Attendees Through End of 2005. For more info visit:
http://ads.osdn.com/?ad_idv28&alloc_id\x16845&op=click
_______________________________________________
Linux-hotplug-devel mailing list  http://linux-hotplug.sourceforge.net
Linux-hotplug-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/linux-hotplug-devel

Re: waiting for an unknown set of udev /dev entries to complete

[Pozsar Balazs]

On Sun, Nov 20, 2005 at 07:03:57AM +0100, Kay Sievers wrote:
> On Fri, Nov 18, 2005 at 02:30:45PM -0800, Patrick Mansfield wrote:
> > Any ideas on the best way to wait for udev to finish handling a set of
> > hotplug events?
> 
> I have changes in my udev tree, to export the udevd event queue. It
> will probably be in 076 - if it works as expected. For every queued
> or running event, a symlink in /dev/.udev/queue/ is created and removed
> after the udev event process has exited. For all failing udev events,
> the queue files are moved to /dev/.udev/failed.
> 
> We want this for coldplug after bootup, to try to trigger failed events
> again, if requirements are fulfilled only at a later stage of the boot
> process.

What failed events do you think of? I suppose you really need this for 
some cases, and I can't imagine what it is.


Side note about coldplug: wouldn't it be nicer if udevd would:
 - trigger all events using the uevent files under /sys
 - wait for these events to arrive and to finish
 - _then_ fork and daemonize itself.

This would mostly eliminate the need to hack ugly scripts around udev to 
watch its queue, wait for devices on boot etc.


-- 
pozsy


-------------------------------------------------------
This SF.Net email is sponsored by the JBoss Inc.  Get Certified Today
Register for a JBoss Training Course.  Free Certification Exam
for All Training Attendees Through End of 2005. For more info visit:
http://ads.osdn.com/?ad_idv28&alloc_id\x16845&op=click
_______________________________________________
Linux-hotplug-devel mailing list  http://linux-hotplug.sourceforge.net
Linux-hotplug-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/linux-hotplug-devel

Re: waiting for an unknown set of udev /dev entries to complete

[Marco d'Itri]

[-- Attachment #1: Type: text/plain, Size: 444 bytes --]

On Nov 20, Kay Sievers <kay.sievers@vrfy.org> wrote:

> You should be able to look for /dev/.udev/queue/*block*, and see if
> there are events still running.
I still see a problem with this design: there is a period (which may be
easy to hit if the udevd is spawning the maximum allowed number of
children), between when events are sent to udevd and when it starts
processing them, when the on-disk queue is empty.

-- 
ciao,
Marco

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 189 bytes --]

Re: waiting for an unknown set of udev /dev entries to complete

[Kay Sievers]

On Sun, Nov 20, 2005 at 09:23:33PM +0100, Marco d'Itri wrote:
> On Nov 20, Kay Sievers <kay.sievers@vrfy.org> wrote:
> 
> > You should be able to look for /dev/.udev/queue/*block*, and see if
> > there are events still running.
> I still see a problem with this design: there is a period (which may be
> easy to hit if the udevd is spawning the maximum allowed number of
> children), between when events are sent to udevd and when it starts
> processing them, when the on-disk queue is empty.

It's the received event queue, which is exported, not the processes
currently running. What problem do you see related to the maximum number
of childs?

There is still a small theorethical window between the module load
and the first event for a device created by that module, but it's unlikely
to hit that and it should be easy to work around that, if necessary.

Kay


-------------------------------------------------------
This SF.Net email is sponsored by the JBoss Inc.  Get Certified Today
Register for a JBoss Training Course.  Free Certification Exam
for All Training Attendees Through End of 2005. For more info visit:
http://ads.osdn.com/?ad_idv28&alloc_id\x16845&op=click
_______________________________________________
Linux-hotplug-devel mailing list  http://linux-hotplug.sourceforge.net
Linux-hotplug-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/linux-hotplug-devel

Re: waiting for an unknown set of udev /dev entries to complete

[Kay Sievers]

On Sun, Nov 20, 2005 at 09:10:35PM +0100, Pozsar Balazs wrote:
> On Sun, Nov 20, 2005 at 07:03:57AM +0100, Kay Sievers wrote:
> > On Fri, Nov 18, 2005 at 02:30:45PM -0800, Patrick Mansfield wrote:
> > > Any ideas on the best way to wait for udev to finish handling a set of
> > > hotplug events?
> > 
> > I have changes in my udev tree, to export the udevd event queue. It
> > will probably be in 076 - if it works as expected. For every queued
> > or running event, a symlink in /dev/.udev/queue/ is created and removed
> > after the udev event process has exited. For all failing udev events,
> > the queue files are moved to /dev/.udev/failed.
> > 
> > We want this for coldplug after bootup, to try to trigger failed events
> > again, if requirements are fulfilled only at a later stage of the boot
> > process.
> 
> What failed events do you think of? I suppose you really need this for 
> some cases, and I can't imagine what it is.

For checkpointing the boot process. Simplest example is that programs
needed for some devices to set-up are not on the root filesystem. These
events will fail, cause RUN will fail with the coldplug calls. The failed
events can be tried again, after the localfs's are mounted.
We will see what weird things we will need to work around, to support
all the insane options people invented in the unix history.

> Side note about coldplug: wouldn't it be nicer if udevd would:
>  - trigger all events using the uevent files under /sys
>  - wait for these events to arrive and to finish
>  - _then_ fork and daemonize itself.

That would make no difference, besides hardcoding the event trigger
order into the daemon.

> This would mostly eliminate the need to hack ugly scripts around udev to 
> watch its queue, wait for devices on boot etc.

Well these scripts are not ugly, it's pretty simple and flexible. Be
sure, we have much uglier things to handle and we will need the
flexibility and control over the event generation.

Kay


-------------------------------------------------------
This SF.Net email is sponsored by the JBoss Inc.  Get Certified Today
Register for a JBoss Training Course.  Free Certification Exam
for All Training Attendees Through End of 2005. For more info visit:
http://ads.osdn.com/?ad_idv28&alloc_id\x16845&op=click
_______________________________________________
Linux-hotplug-devel mailing list  http://linux-hotplug.sourceforge.net
Linux-hotplug-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/linux-hotplug-devel

Re: waiting for an unknown set of udev /dev entries to complete

[Marco d'Itri]

[-- Attachment #1: Type: text/plain, Size: 915 bytes --]

On Nov 21, Kay Sievers <kay.sievers@vrfy.org> wrote:

> > I still see a problem with this design: there is a period (which may be
> > easy to hit if the udevd is spawning the maximum allowed number of
> > children), between when events are sent to udevd and when it starts
> > processing them, when the on-disk queue is empty.
> It's the received event queue, which is exported, not the processes
> currently running. What problem do you see related to the maximum number
> of childs?
Nevermind, this makes the race much shorter but it's still there.

> There is still a small theorethical window between the module load
> and the first event for a device created by that module, but it's unlikely
> to hit that and it should be easy to work around that, if necessary.
Yes, this is what I meant (in the context of boot-time events synthesis)
and I do not like taking this risk.

-- 
ciao,
Marco

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 189 bytes --]

Re: waiting for an unknown set of udev /dev entries to complete

[Kay Sievers]

On Mon, Nov 21, 2005 at 01:01:54AM +0100, Marco d'Itri wrote:
> On Nov 21, Kay Sievers <kay.sievers@vrfy.org> wrote:
> 
> > > I still see a problem with this design: there is a period (which may be
> > > easy to hit if the udevd is spawning the maximum allowed number of
> > > children), between when events are sent to udevd and when it starts
> > > processing them, when the on-disk queue is empty.
> > It's the received event queue, which is exported, not the processes
> > currently running. What problem do you see related to the maximum number
> > of childs?
> Nevermind, this makes the race much shorter but it's still there.
> 
> > There is still a small theorethical window between the module load
> > and the first event for a device created by that module, but it's unlikely
> > to hit that and it should be easy to work around that, if necessary.
> Yes, this is what I meant (in the context of boot-time events synthesis)
> and I do not like taking this risk.

Scott sent a nice patch to remove the /dev/.udev.queue directory if it's
empty. That makes it pretty easy to work around this:
  start udevd
  mkdir -p /dev/.udev/queue
  trigger uevent's in /sys
  while test -d /dev/.udev/queue; do sleep 0.1; done

The last event will just rmdir() the created queue directory.

Kay


-------------------------------------------------------
This SF.Net email is sponsored by the JBoss Inc.  Get Certified Today
Register for a JBoss Training Course.  Free Certification Exam
for All Training Attendees Through End of 2005. For more info visit:
http://ads.osdn.com/?ad_idv28&alloc_id\x16845&op=click
_______________________________________________
Linux-hotplug-devel mailing list  http://linux-hotplug.sourceforge.net
Linux-hotplug-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/linux-hotplug-devel

Re: waiting for an unknown set of udev /dev entries to complete

[Scott James Remnant]

[-- Attachment #1: Type: text/plain, Size: 1178 bytes --]

On Wed, 2005-11-23 at 00:13 +0100, Kay Sievers wrote:

> Scott sent a nice patch to remove the /dev/.udev.queue directory if it's
> empty. That makes it pretty easy to work around this:
>   start udevd
>   mkdir -p /dev/.udev/queue
>   trigger uevent's in /sys
>   while test -d /dev/.udev/queue; do sleep 0.1; done
> 
> The last event will just rmdir() the created queue directory.
> 
There's still a potential race here: if an event arrives after you've
created the queue directory and finishes just as you finish triggering
the uevents, but before the first triggered event gets queued, the queue
directory will be removed and the script won't wait.  

Looking through the kernel code it does look like it's almost
impossible, as the triggering of a uevent seems to not complete the
write() call until it's been posted down the netlink socket.

Almost isn't totally though; so it's there ... but it's such a remote
chance that I think it's fine for now, and the only real way to avoid
race conditions is just not to wait and arrange your entire boot
sequence to be event-driven rather than serial.

Scott
-- 
Scott James Remnant
scott@ubuntu.com

[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 189 bytes --]

Re: waiting for an unknown set of udev /dev entries to complete

[Pozsar Balazs]

On Wed, Nov 23, 2005 at 08:26:15AM +0000, Scott James Remnant wrote:
> On Wed, 2005-11-23 at 00:13 +0100, Kay Sievers wrote:
> 
> > Scott sent a nice patch to remove the /dev/.udev.queue directory if it's
> > empty. That makes it pretty easy to work around this:
> >   start udevd
> >   mkdir -p /dev/.udev/queue
> >   trigger uevent's in /sys
> >   while test -d /dev/.udev/queue; do sleep 0.1; done
> > 
> > The last event will just rmdir() the created queue directory.
> > 
> There's still a potential race here: if an event arrives after you've
> created the queue directory and finishes just as you finish triggering
> the uevents, but before the first triggered event gets queued, the queue
> directory will be removed and the script won't wait.  
> 
> Looking through the kernel code it does look like it's almost
> impossible, as the triggering of a uevent seems to not complete the
> write() call until it's been posted down the netlink socket.
> 
> Almost isn't totally though; so it's there ... but it's such a remote
> chance that I think it's fine for now, and the only real way to avoid
> race conditions is just not to wait and arrange your entire boot
> sequence to be event-driven rather than serial.

I have an idea for this: after triggering all the uevents, one more 
special event should be sent to the netlink socket, meaning that's the 
end of the event series were are interested in. When udev sees this 
event, and finished processing all the events _before_ this special 
event, then we are ready to go on.

I still think, the best solution would be:
 - udevd itself triggers all the uevents
 - when it finished processing all these events, only then daemonizes 
   itself.
So the udevd startup script could be nothing more than:
  udevd --daemon

Nice daemons always have this kind of behavior: they does not fork 
and return immediately, but return if the daemon successfully started 
and initalized itself. For a good example, see hald.


-- 
pozsy


-------------------------------------------------------
This SF.Net email is sponsored by the JBoss Inc.  Get Certified Today
Register for a JBoss Training Course.  Free Certification Exam
for All Training Attendees Through End of 2005. For more info visit:
http://ads.osdn.com/?ad_idv28&alloc_id\x16845&op=click
_______________________________________________
Linux-hotplug-devel mailing list  http://linux-hotplug.sourceforge.net
Linux-hotplug-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/linux-hotplug-devel

Re: waiting for an unknown set of udev /dev entries to complete

[Harald Hoyer]

[-- Attachment #1: Type: text/plain, Size: 591 bytes --]

Pozsar Balazs wrote:
> I still think, the best solution would be:
>  - udevd itself triggers all the uevents
>  - when it finished processing all these events, only then daemonizes 
>    itself.
> So the udevd startup script could be nothing more than:
>   udevd --daemon
> 
> Nice daemons always have this kind of behavior: they does not fork 
> and return immediately, but return if the daemon successfully started 
> and initalized itself. For a good example, see hald.
> 
> 

/signed

you may test:
ftp://people.redhat.com/harald/udev/udev-075-5.src.rpm
which has the attached patch :)


[-- Attachment #2: udev-075-daemon.patch --]
[-- Type: text/x-patch, Size: 9802 bytes --]

--- udev-075/udevd.c.daemon	2005-11-23 14:21:40.000000000 +0100
+++ udev-075/udevd.c	2005-11-23 15:41:41.000000000 +0100
@@ -42,14 +42,16 @@
 #include <linux/types.h>
 #include <linux/netlink.h>
 
-#include "list.h"
+#include "libsysfs/sysfs/libsysfs.h"
 #include "udev_libc_wrapper.h"
+#include "udev_sysfs.h"
 #include "udev.h"
 #include "udev_version.h"
-#include "udev_rules.h"
+#include "logging.h"
 #include "udev_utils.h"
+#include "udev_rules.h"
+#include "list.h"
 #include "udevd.h"
-#include "logging.h"
 
 struct udev_rules rules;
 static int udevd_sock;
@@ -71,6 +73,277 @@
 static LIST_HEAD(running_list);
 
 
+struct device {
+	struct list_head node;
+	char path[PATH_SIZE];
+	char subsys[NAME_SIZE];
+};
+
+/* sort files in lexical order */
+static int device_list_insert(const char *path, char *subsystem, struct list_head *device_list)
+{
+	struct device *loop_device;
+	struct device *new_device;
+	const char *devpath = &path[strlen(sysfs_path)];
+
+	dbg("insert: '%s'\n", devpath);
+
+	list_for_each_entry(loop_device, device_list, node) {
+		if (strcmp(loop_device->path, devpath) > 0) {
+			break;
+		}
+	}
+
+	new_device = malloc(sizeof(struct device));
+	if (new_device == NULL) {
+		dbg("error malloc");
+		return -ENOMEM;
+	}
+
+	strlcpy(new_device->path, devpath, sizeof(new_device->path));
+	strlcpy(new_device->subsys, subsystem, sizeof(new_device->subsys));
+	list_add_tail(&new_device->node, &loop_device->node);
+	dbg("add '%s' from subsys '%s'", new_device->path, new_device->subsys);
+	return 0;
+}
+
+/* list of devices that we should run last due to any one of a number of reasons */
+static char *last_list[] = {
+	"/block/dm",	/* on here because dm wants to have the block devices around before it */
+	NULL,
+};
+
+/* list of devices that we should run first due to any one of a number of reasons */
+static char *first_list[] = {
+	"/class/mem",
+	"/class/tty",
+	"/bus",
+	NULL,
+};
+
+static int add_device(const char *devpath, const char *subsystem)
+{
+  char filename[PATH_SIZE];
+  int fd;
+
+  snprintf(filename, sizeof(filename), "%s/%s/uevent", sysfs_path, devpath);
+  filename[sizeof(filename)-1] = '\0';
+
+  dbg("Trigger %s", filename);
+  fd = open(filename, O_WRONLY);
+  if (fd > 0) {
+    write(fd, "add\n", 4);
+    close(fd);
+  }
+  else return 1;
+  
+  return 0;
+}
+
+static void do_exec_list(struct list_head *device_list)
+{
+	struct device *loop_device;
+	struct device *tmp_device;
+	int i;
+
+	/* handle the "first" type devices first */
+	list_for_each_entry_safe(loop_device, tmp_device, device_list, node) {
+		for (i = 0; first_list[i] != NULL; i++) {
+			if (strncmp(loop_device->path, first_list[i], strlen(first_list[i])) == 0) {
+				add_device(loop_device->path, loop_device->subsys);
+				list_del(&loop_device->node);
+				free(loop_device);
+				break;
+			}
+		}
+	}
+
+	/* handle the devices we are allowed to, excluding the "last" type devices */
+	list_for_each_entry_safe(loop_device, tmp_device, device_list, node) {
+		int found = 0;
+		for (i = 0; last_list[i] != NULL; i++) {
+			if (strncmp(loop_device->path, last_list[i], strlen(last_list[i])) == 0) {
+				found = 1;
+				break;
+			}
+		}
+		if (found)
+			continue;
+
+		add_device(loop_device->path, loop_device->subsys);
+		list_del(&loop_device->node);
+		free(loop_device);
+	}
+
+	/* handle the rest of the devices left over, if any */
+	list_for_each_entry_safe(loop_device, tmp_device, device_list, node) {
+		add_device(loop_device->path, loop_device->subsys);
+		list_del(&loop_device->node);
+		free(loop_device);
+	}
+}
+
+static int has_devt(const char *path)
+{
+	char filename[PATH_SIZE];
+	struct stat statbuf;
+
+	snprintf(filename, sizeof(filename), "%s/uevent", path);
+	filename[sizeof(filename)-1] = '\0';
+
+	if (stat(filename, &statbuf) == 0)
+		return 1;
+
+	return 0;
+}
+
+static void udev_scan_block(struct list_head *device_list)
+{
+	char base[PATH_SIZE];
+	DIR *dir;
+	struct dirent *dent;
+
+	snprintf(base, sizeof(base), "%s/block", sysfs_path);
+	base[sizeof(base)-1] = '\0';
+
+	dir = opendir(base);
+	if (dir != NULL) {
+		for (dent = readdir(dir); dent != NULL; dent = readdir(dir)) {
+			char dirname[PATH_SIZE];
+			DIR *dir2;
+			struct dirent *dent2;
+
+			if (dent->d_name[0] == '.')
+				continue;
+
+			snprintf(dirname, sizeof(dirname), "%s/%s", base, dent->d_name);
+			dirname[sizeof(dirname)-1] = '\0';
+			if (has_devt(dirname))
+				device_list_insert(dirname, "block", device_list);
+			else
+				continue;
+
+			/* look for partitions */
+			dir2 = opendir(dirname);
+			if (dir2 != NULL) {
+				for (dent2 = readdir(dir2); dent2 != NULL; dent2 = readdir(dir2)) {
+					char dirname2[PATH_SIZE];
+
+					if (dent2->d_name[0] == '.')
+						continue;
+
+					snprintf(dirname2, sizeof(dirname2), "%s/%s", dirname, dent2->d_name);
+					dirname2[sizeof(dirname2)-1] = '\0';
+
+					if (has_devt(dirname2))
+						device_list_insert(dirname2, "block", device_list);
+				}
+				closedir(dir2);
+			}
+		}
+		closedir(dir);
+	}
+}
+
+static void udev_scan_bus(struct list_head *device_list)
+{
+	char base[PATH_SIZE];
+	DIR *dir;
+	struct dirent *dent;
+
+	snprintf(base, sizeof(base), "%s/bus", sysfs_path);
+	base[sizeof(base)-1] = '\0';
+
+	dir = opendir(base);
+	if (dir != NULL) {
+		for (dent = readdir(dir); dent != NULL; dent = readdir(dir)) {
+			char dirname[PATH_SIZE];
+			DIR *dir2;
+			struct dirent *dent2;
+
+			if (dent->d_name[0] == '.')
+				continue;
+
+			snprintf(dirname, sizeof(dirname), "%s/%s/devices", base, dent->d_name);
+			dirname[sizeof(dirname)-1] = '\0';
+
+			dir2 = opendir(dirname);
+			if (dir2 != NULL) {
+				for (dent2 = readdir(dir2); dent2 != NULL; dent2 = readdir(dir2)) {
+					char dirname2[PATH_SIZE];
+
+					if (dent2->d_name[0] == '.')
+						continue;
+
+					snprintf(dirname2, sizeof(dirname2), "%s/%s", dirname, dent2->d_name);
+					dirname2[sizeof(dirname2)-1] = '\0';
+
+					if (has_devt(dirname2))
+						device_list_insert(dirname2, dent->d_name, device_list);
+				}
+				closedir(dir2);
+			}
+		}
+		closedir(dir);
+	}
+}
+
+
+static void udev_scan_class(struct list_head *device_list)
+{
+	char base[PATH_SIZE];
+	DIR *dir;
+	struct dirent *dent;
+
+	snprintf(base, sizeof(base), "%s/class", sysfs_path);
+	base[sizeof(base)-1] = '\0';
+
+	dir = opendir(base);
+	if (dir != NULL) {
+		for (dent = readdir(dir); dent != NULL; dent = readdir(dir)) {
+			char dirname[PATH_SIZE];
+			DIR *dir2;
+			struct dirent *dent2;
+
+			if (dent->d_name[0] == '.')
+				continue;
+
+			snprintf(dirname, sizeof(dirname), "%s/%s", base, dent->d_name);
+			dirname[sizeof(dirname)-1] = '\0';
+
+			dir2 = opendir(dirname);
+			if (dir2 != NULL) {
+				for (dent2 = readdir(dir2); dent2 != NULL; dent2 = readdir(dir2)) {
+					char dirname2[PATH_SIZE];
+
+					if (dent2->d_name[0] == '.')
+						continue;
+
+					snprintf(dirname2, sizeof(dirname2), "%s/%s", dirname, dent2->d_name);
+					dirname2[sizeof(dirname2)-1] = '\0';
+
+					if (has_devt(dirname2))
+						device_list_insert(dirname2, dent->d_name, device_list);
+				}
+				closedir(dir2);
+			}
+		}
+		closedir(dir);
+	}
+}
+
+static int udevstart(void)
+{
+	LIST_HEAD(device_list);
+	udev_scan_bus(&device_list);
+	udev_scan_class(&device_list);
+	udev_scan_block(&device_list);
+	do_exec_list(&device_list);
+	return 0;
+}
+
+
+
 #ifdef USE_LOG
 void log_message(int priority, const char *format, ...)
 {
@@ -691,6 +964,41 @@
 	return 0;
 }
 
+
+static int do_daemonize(void)
+{
+  pid_t pid;
+  int rc = 0;
+
+  pid = fork();
+  switch (pid) {
+  case 0:
+    dbg("daemonized fork running");
+    break;
+  case -1:
+    err("fork of daemon failed: %s", strerror(errno));
+    rc = 4;
+    goto exit;
+  default:
+    dbg("child [%u] running, parent exits", pid);
+    exit(0);
+  }
+
+	/* set scheduling priority for the daemon */
+	setpriority(PRIO_PROCESS, 0, UDEVD_PRIORITY);
+
+	chdir("/");
+	umask(022);
+
+	/* become session leader */
+	sid = setsid();
+	dbg("our session is %d", sid);
+
+ exit:
+       return rc;
+
+}
+
 int main(int argc, char *argv[], char *envp[])
 {
 	int retval;
@@ -702,6 +1010,10 @@
 	int i;
 	int rc = 0;
 	int maxfd;
+	int daemonized = 0;
+
+	struct timeval timeout = { 0, 200000 };
+	struct timeval *ptimeout = &timeout;
 
 	/* redirect std fd's, if the kernel forks us, we don't have them at all */
 	fd = open("/dev/null", O_RDWR);
@@ -762,24 +1074,6 @@
 	/* parse the rules and keep it in memory */
 	udev_rules_init(&rules, 1);
 
-	if (daemonize) {
-		pid_t pid;
-
-		pid = fork();
-		switch (pid) {
-		case 0:
-			dbg("daemonized fork running");
-			break;
-		case -1:
-			err("fork of daemon failed: %s", strerror(errno));
-			rc = 4;
-			goto exit;
-		default:
-			dbg("child [%u] running, parent exits", pid);
-			goto exit;
-		}
-	}
-
 	/* set scheduling priority for the daemon */
 	setpriority(PRIO_PROCESS, 0, UDEVD_PRIORITY);
 
@@ -859,6 +1153,8 @@
 	maxfd = UDEV_MAX(maxfd, signal_pipe[READ_END]);
 	maxfd = UDEV_MAX(maxfd, inotify_fd);
 
+	udevstart();
+
 	while (!udev_exit) {
 		struct uevent_msg *msg;
 		int fdcount;
@@ -870,13 +1166,30 @@
 		if (inotify_fd > 0)
 			FD_SET(inotify_fd, &readfds);
 
-		fdcount = select(maxfd+1, &readfds, NULL, NULL, NULL);
+		fdcount = select(maxfd+1, &readfds, NULL, NULL, ptimeout);
+
 		if (fdcount < 0) {
 			if (errno != EINTR)
 				err("error in select: %s", strerror(errno));
 			continue;
 		}
 
+		if (ptimeout) {
+		  ptimeout->tv_sec = 0;
+		  ptimeout->tv_usec = 200000;
+		}
+
+		if (fdcount == 0) {
+		  if (list_empty(&running_list)) {
+		    if (daemonized == 0 && daemonize) {
+		      do_daemonize();
+		      daemonized++;
+		    }
+		    ptimeout = NULL;
+		  }
+		  continue;
+		}
+
 		/* get user socket message */
 		if (FD_ISSET(udevd_sock, &readfds)) {
 			msg = get_udevd_msg();

Re: waiting for an unknown set of udev /dev entries to complete

[Kay Sievers]

On Wed, Nov 23, 2005 at 11:54:22AM +0100, Pozsar Balazs wrote:
> On Wed, Nov 23, 2005 at 08:26:15AM +0000, Scott James Remnant wrote:
> > On Wed, 2005-11-23 at 00:13 +0100, Kay Sievers wrote:
> > 
> > > Scott sent a nice patch to remove the /dev/.udev.queue directory if it's
> > > empty. That makes it pretty easy to work around this:
> > >   start udevd
> > >   mkdir -p /dev/.udev/queue
> > >   trigger uevent's in /sys
> > >   while test -d /dev/.udev/queue; do sleep 0.1; done
> > > 
> > > The last event will just rmdir() the created queue directory.
> > > 
> > There's still a potential race here: if an event arrives after you've
> > created the queue directory and finishes just as you finish triggering
> > the uevents, but before the first triggered event gets queued, the queue
> > directory will be removed and the script won't wait.  
> > 
> > Looking through the kernel code it does look like it's almost
> > impossible, as the triggering of a uevent seems to not complete the
> > write() call until it's been posted down the netlink socket.
> > 
> > Almost isn't totally though; so it's there ... but it's such a remote
> > chance that I think it's fine for now, and the only real way to avoid
> > race conditions is just not to wait and arrange your entire boot
> > sequence to be event-driven rather than serial.
> 
> I have an idea for this: after triggering all the uevents, one more 
> special event should be sent to the netlink socket, meaning that's the 
> end of the event series were are interested in. When udev sees this 
> event, and finished processing all the events _before_ this special 
> event, then we are ready to go on.
> 
> I still think, the best solution would be:
>  - udevd itself triggers all the uevents
>  - when it finished processing all these events, only then daemonizes 
>    itself.
> So the udevd startup script could be nothing more than:
>   udevd --daemon
> 
> Nice daemons always have this kind of behavior: they does not fork 
> and return immediately, but return if the daemon successfully started 
> and initalized itself. For a good example, see hald.

That's a different issue. HAL is a stateful daemon, that offers data to
userspace applications. It also reads static data and does not deal with
hardware initialization, which has a lot of corner cases. That doesn't
count for udev.

Daemon initialization has nothing to to with coldplug. Be sure, you
don't want to hardcode device replay order, or event staging into the
daemon itself and mixup complete different tasks in bootup.

And again, it just doesn't make any sense, to give up the flexibility
to exclude events from being triggered, or wait only for specific events
at a certain stage. You also want to retry failed events with the same
logic at a later stage. And there are setups, you really want to work
around nasty things without patching a daemon.

Kay


-------------------------------------------------------
This SF.net email is sponsored by: Splunk Inc. Do you grep through log files
for problems?  Stop!  Download the new AJAX search engine that makes
searching your log files as easy as surfing the  web.  DOWNLOAD SPLUNK!
http://ads.osdn.com/?ad_idv37&alloc_id\x16865&op=click
_______________________________________________
Linux-hotplug-devel mailing list  http://linux-hotplug.sourceforge.net
Linux-hotplug-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/linux-hotplug-devel

Re: waiting for an unknown set of udev /dev entries to complete

[Pozsar Balazs]

On Wed, Nov 23, 2005 at 05:50:40PM +0100, Kay Sievers wrote:
> On Wed, Nov 23, 2005 at 11:54:22AM +0100, Pozsar Balazs wrote:
> > On Wed, Nov 23, 2005 at 08:26:15AM +0000, Scott James Remnant wrote:
> > > On Wed, 2005-11-23 at 00:13 +0100, Kay Sievers wrote:
> > > 
> > > > Scott sent a nice patch to remove the /dev/.udev.queue directory if it's
> > > > empty. That makes it pretty easy to work around this:
> > > >   start udevd
> > > >   mkdir -p /dev/.udev/queue
> > > >   trigger uevent's in /sys
> > > >   while test -d /dev/.udev/queue; do sleep 0.1; done
> > > > 
> > > > The last event will just rmdir() the created queue directory.
> > > > 
> > > There's still a potential race here: if an event arrives after you've
> > > created the queue directory and finishes just as you finish triggering
> > > the uevents, but before the first triggered event gets queued, the queue
> > > directory will be removed and the script won't wait.  
> > > 
> > > Looking through the kernel code it does look like it's almost
> > > impossible, as the triggering of a uevent seems to not complete the
> > > write() call until it's been posted down the netlink socket.
> > > 
> > > Almost isn't totally though; so it's there ... but it's such a remote
> > > chance that I think it's fine for now, and the only real way to avoid
> > > race conditions is just not to wait and arrange your entire boot
> > > sequence to be event-driven rather than serial.
> > 
> > I have an idea for this: after triggering all the uevents, one more 
> > special event should be sent to the netlink socket, meaning that's the 
> > end of the event series were are interested in. When udev sees this 
> > event, and finished processing all the events _before_ this special 
> > event, then we are ready to go on.
> > 
> > I still think, the best solution would be:
> >  - udevd itself triggers all the uevents
> >  - when it finished processing all these events, only then daemonizes 
> >    itself.
> > So the udevd startup script could be nothing more than:
> >   udevd --daemon
> > 
> > Nice daemons always have this kind of behavior: they does not fork 
> > and return immediately, but return if the daemon successfully started 
> > and initalized itself. For a good example, see hald.
> 
> That's a different issue. HAL is a stateful daemon, that offers data to
> userspace applications. It also reads static data and does not deal with
> hardware initialization, which has a lot of corner cases. That doesn't
> count for udev.
> 
> Daemon initialization has nothing to to with coldplug. Be sure, you
> don't want to hardcode device replay order, or event staging into the
> daemon itself and mixup complete different tasks in bootup.
> 
> And again, it just doesn't make any sense, to give up the flexibility
> to exclude events from being triggered, or wait only for specific events
> at a certain stage. You also want to retry failed events with the same
> logic at a later stage. And there are setups, you really want to work
> around nasty things without patching a daemon.

Well, my basic concern is not that you want flexibility via replaying 
event outside from udev. That I can understand, though my opinion is 
that the booting should not be special. If I am not mistaken, you are 
trying to order the events (generated by writing to the uevent files), 
which will be handled by udev:
 - in parallel way
 - in an unpredictable order
What am I missing?

My other concern is, that there should be a nice and clean way to
"wait until all the events are handled (which are replayed via uevents 
on boot)". And _no_, that is not polling a directory, of whose 
disappearence is _hopefully_ the sign. Send a signal, send a netlink, 
whatever.


-- 
pozsy


-------------------------------------------------------
This SF.net email is sponsored by: Splunk Inc. Do you grep through log files
for problems?  Stop!  Download the new AJAX search engine that makes
searching your log files as easy as surfing the  web.  DOWNLOAD SPLUNK!
http://ads.osdn.com/?ad_idv37&alloc_id\x16865&op=click
_______________________________________________
Linux-hotplug-devel mailing list  http://linux-hotplug.sourceforge.net
Linux-hotplug-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/linux-hotplug-devel

Re: waiting for an unknown set of udev /dev entries to complete

[Kay Sievers]

On Wed, Nov 23, 2005 at 06:25:20PM +0100, Pozsar Balazs wrote:
> On Wed, Nov 23, 2005 at 05:50:40PM +0100, Kay Sievers wrote:
> > On Wed, Nov 23, 2005 at 11:54:22AM +0100, Pozsar Balazs wrote:
> > > On Wed, Nov 23, 2005 at 08:26:15AM +0000, Scott James Remnant wrote:
> > > > On Wed, 2005-11-23 at 00:13 +0100, Kay Sievers wrote:
> > > > 
> > > > > Scott sent a nice patch to remove the /dev/.udev.queue directory if it's
> > > > > empty. That makes it pretty easy to work around this:
> > > > >   start udevd
> > > > >   mkdir -p /dev/.udev/queue
> > > > >   trigger uevent's in /sys
> > > > >   while test -d /dev/.udev/queue; do sleep 0.1; done
> > > > > 
> > > > > The last event will just rmdir() the created queue directory.
> > > > > 
> > > > There's still a potential race here: if an event arrives after you've
> > > > created the queue directory and finishes just as you finish triggering
> > > > the uevents, but before the first triggered event gets queued, the queue
> > > > directory will be removed and the script won't wait.  
> > > > 
> > > > Looking through the kernel code it does look like it's almost
> > > > impossible, as the triggering of a uevent seems to not complete the
> > > > write() call until it's been posted down the netlink socket.
> > > > 
> > > > Almost isn't totally though; so it's there ... but it's such a remote
> > > > chance that I think it's fine for now, and the only real way to avoid
> > > > race conditions is just not to wait and arrange your entire boot
> > > > sequence to be event-driven rather than serial.
> > > 
> > > I have an idea for this: after triggering all the uevents, one more 
> > > special event should be sent to the netlink socket, meaning that's the 
> > > end of the event series were are interested in. When udev sees this 
> > > event, and finished processing all the events _before_ this special 
> > > event, then we are ready to go on.
> > > 
> > > I still think, the best solution would be:
> > >  - udevd itself triggers all the uevents
> > >  - when it finished processing all these events, only then daemonizes 
> > >    itself.
> > > So the udevd startup script could be nothing more than:
> > >   udevd --daemon
> > > 
> > > Nice daemons always have this kind of behavior: they does not fork 
> > > and return immediately, but return if the daemon successfully started 
> > > and initalized itself. For a good example, see hald.
> > 
> > That's a different issue. HAL is a stateful daemon, that offers data to
> > userspace applications. It also reads static data and does not deal with
> > hardware initialization, which has a lot of corner cases. That doesn't
> > count for udev.
> > 
> > Daemon initialization has nothing to to with coldplug. Be sure, you
> > don't want to hardcode device replay order, or event staging into the
> > daemon itself and mixup complete different tasks in bootup.
> > 
> > And again, it just doesn't make any sense, to give up the flexibility
> > to exclude events from being triggered, or wait only for specific events
> > at a certain stage. You also want to retry failed events with the same
> > logic at a later stage. And there are setups, you really want to work
> > around nasty things without patching a daemon.
> 
> Well, my basic concern is not that you want flexibility via replaying 
> event outside from udev. That I can understand, though my opinion is 
> that the booting should not be special. If I am not mistaken, you are 
> trying to order the events (generated by writing to the uevent files), 
> which will be handled by udev:
>  - in parallel way

Right.

>  - in an unpredictable order

Wrong.

> What am I missing?

A clue.

> My other concern is, that there should be a nice and clean way to
> "wait until all the events are handled (which are replayed via uevents 
> on boot)". And _no_, that is not polling a directory, of whose 
> disappearence is _hopefully_ the sign. Send a signal, send a netlink, 
> whatever.

Sticking completely different things in a monolithic deamon is neighter
nice nor clean. We may end up with something pretty much different from
what we have today, but be sure, udevd will not scan sysfs and trigger
events.

Kay


-------------------------------------------------------
This SF.net email is sponsored by: Splunk Inc. Do you grep through log files
for problems?  Stop!  Download the new AJAX search engine that makes
searching your log files as easy as surfing the  web.  DOWNLOAD SPLUNK!
http://ads.osdn.com/?ad_idv37&alloc_id\x16865&op=click
_______________________________________________
Linux-hotplug-devel mailing list  http://linux-hotplug.sourceforge.net
Linux-hotplug-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/linux-hotplug-devel

Re: waiting for an unknown set of udev /dev entries to complete

[Pozsar Balazs]

On Wed, Nov 23, 2005 at 06:46:57PM +0100, Kay Sievers wrote:
> On Wed, Nov 23, 2005 at 06:25:20PM +0100, Pozsar Balazs wrote:
> > On Wed, Nov 23, 2005 at 05:50:40PM +0100, Kay Sievers wrote:
> > > On Wed, Nov 23, 2005 at 11:54:22AM +0100, Pozsar Balazs wrote:
> > > > On Wed, Nov 23, 2005 at 08:26:15AM +0000, Scott James Remnant wrote:
> > > > > On Wed, 2005-11-23 at 00:13 +0100, Kay Sievers wrote:
> > > > > 
> > > > > > Scott sent a nice patch to remove the /dev/.udev.queue directory if it's
> > > > > > empty. That makes it pretty easy to work around this:
> > > > > >   start udevd
> > > > > >   mkdir -p /dev/.udev/queue
> > > > > >   trigger uevent's in /sys
> > > > > >   while test -d /dev/.udev/queue; do sleep 0.1; done
> > > > > > 
> > > > > > The last event will just rmdir() the created queue directory.
> > > > > > 
> > > > > There's still a potential race here: if an event arrives after you've
> > > > > created the queue directory and finishes just as you finish triggering
> > > > > the uevents, but before the first triggered event gets queued, the queue
> > > > > directory will be removed and the script won't wait.  
> > > > > 
> > > > > Looking through the kernel code it does look like it's almost
> > > > > impossible, as the triggering of a uevent seems to not complete the
> > > > > write() call until it's been posted down the netlink socket.
> > > > > 
> > > > > Almost isn't totally though; so it's there ... but it's such a remote
> > > > > chance that I think it's fine for now, and the only real way to avoid
> > > > > race conditions is just not to wait and arrange your entire boot
> > > > > sequence to be event-driven rather than serial.
> > > > 
> > > > I have an idea for this: after triggering all the uevents, one more 
> > > > special event should be sent to the netlink socket, meaning that's the 
> > > > end of the event series were are interested in. When udev sees this 
> > > > event, and finished processing all the events _before_ this special 
> > > > event, then we are ready to go on.
> > > > 
> > > > I still think, the best solution would be:
> > > >  - udevd itself triggers all the uevents
> > > >  - when it finished processing all these events, only then daemonizes 
> > > >    itself.
> > > > So the udevd startup script could be nothing more than:
> > > >   udevd --daemon
> > > > 
> > > > Nice daemons always have this kind of behavior: they does not fork 
> > > > and return immediately, but return if the daemon successfully started 
> > > > and initalized itself. For a good example, see hald.
> > > 
> > > That's a different issue. HAL is a stateful daemon, that offers data to
> > > userspace applications. It also reads static data and does not deal with
> > > hardware initialization, which has a lot of corner cases. That doesn't
> > > count for udev.
> > > 
> > > Daemon initialization has nothing to to with coldplug. Be sure, you
> > > don't want to hardcode device replay order, or event staging into the
> > > daemon itself and mixup complete different tasks in bootup.
> > > 
> > > And again, it just doesn't make any sense, to give up the flexibility
> > > to exclude events from being triggered, or wait only for specific events
> > > at a certain stage. You also want to retry failed events with the same
> > > logic at a later stage. And there are setups, you really want to work
> > > around nasty things without patching a daemon.
> > 
> > Well, my basic concern is not that you want flexibility via replaying 
> > event outside from udev. That I can understand, though my opinion is 
> > that the booting should not be special. If I am not mistaken, you are 
> > trying to order the events (generated by writing to the uevent files), 
> > which will be handled by udev:
> >  - in parallel way
> 
> Right.
> 
> >  - in an unpredictable order
> 
> Wrong.
> 
> > What am I missing?
> 
> A clue.

Sorry, I still don't get it...

udevd receives the events, and starts to handle them in parallel.
From that on, there are no guarantees about execution order, it all 
depends on the linux kernel scheduler. Are there?


-- 
pozsy


-------------------------------------------------------
This SF.net email is sponsored by: Splunk Inc. Do you grep through log files
for problems?  Stop!  Download the new AJAX search engine that makes
searching your log files as easy as surfing the  web.  DOWNLOAD SPLUNK!
http://ads.osdn.com/?ad_idv37&alloc_id\x16865&op=click
_______________________________________________
Linux-hotplug-devel mailing list  http://linux-hotplug.sourceforge.net
Linux-hotplug-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/linux-hotplug-devel

Re: waiting for an unknown set of udev /dev entries to complete

[Kay Sievers]

On Wed, Nov 23, 2005 at 07:16:58PM +0100, Pozsar Balazs wrote:
> On Wed, Nov 23, 2005 at 06:46:57PM +0100, Kay Sievers wrote:
> > On Wed, Nov 23, 2005 at 06:25:20PM +0100, Pozsar Balazs wrote:
> > > On Wed, Nov 23, 2005 at 05:50:40PM +0100, Kay Sievers wrote:
> > > > On Wed, Nov 23, 2005 at 11:54:22AM +0100, Pozsar Balazs wrote:
> > > > > On Wed, Nov 23, 2005 at 08:26:15AM +0000, Scott James Remnant wrote:
> > > > > > On Wed, 2005-11-23 at 00:13 +0100, Kay Sievers wrote:
> > > > > > 
> > > > > > > Scott sent a nice patch to remove the /dev/.udev.queue directory if it's
> > > > > > > empty. That makes it pretty easy to work around this:
> > > > > > >   start udevd
> > > > > > >   mkdir -p /dev/.udev/queue
> > > > > > >   trigger uevent's in /sys
> > > > > > >   while test -d /dev/.udev/queue; do sleep 0.1; done
> > > > > > > 
> > > > > > > The last event will just rmdir() the created queue directory.
> > > > > > > 
> > > > > > There's still a potential race here: if an event arrives after you've
> > > > > > created the queue directory and finishes just as you finish triggering
> > > > > > the uevents, but before the first triggered event gets queued, the queue
> > > > > > directory will be removed and the script won't wait.  
> > > > > > 
> > > > > > Looking through the kernel code it does look like it's almost
> > > > > > impossible, as the triggering of a uevent seems to not complete the
> > > > > > write() call until it's been posted down the netlink socket.
> > > > > > 
> > > > > > Almost isn't totally though; so it's there ... but it's such a remote
> > > > > > chance that I think it's fine for now, and the only real way to avoid
> > > > > > race conditions is just not to wait and arrange your entire boot
> > > > > > sequence to be event-driven rather than serial.
> > > > > 
> > > > > I have an idea for this: after triggering all the uevents, one more 
> > > > > special event should be sent to the netlink socket, meaning that's the 
> > > > > end of the event series were are interested in. When udev sees this 
> > > > > event, and finished processing all the events _before_ this special 
> > > > > event, then we are ready to go on.
> > > > > 
> > > > > I still think, the best solution would be:
> > > > >  - udevd itself triggers all the uevents
> > > > >  - when it finished processing all these events, only then daemonizes 
> > > > >    itself.
> > > > > So the udevd startup script could be nothing more than:
> > > > >   udevd --daemon
> > > > > 
> > > > > Nice daemons always have this kind of behavior: they does not fork 
> > > > > and return immediately, but return if the daemon successfully started 
> > > > > and initalized itself. For a good example, see hald.
> > > > 
> > > > That's a different issue. HAL is a stateful daemon, that offers data to
> > > > userspace applications. It also reads static data and does not deal with
> > > > hardware initialization, which has a lot of corner cases. That doesn't
> > > > count for udev.
> > > > 
> > > > Daemon initialization has nothing to to with coldplug. Be sure, you
> > > > don't want to hardcode device replay order, or event staging into the
> > > > daemon itself and mixup complete different tasks in bootup.
> > > > 
> > > > And again, it just doesn't make any sense, to give up the flexibility
> > > > to exclude events from being triggered, or wait only for specific events
> > > > at a certain stage. You also want to retry failed events with the same
> > > > logic at a later stage. And there are setups, you really want to work
> > > > around nasty things without patching a daemon.
> > > 
> > > Well, my basic concern is not that you want flexibility via replaying 
> > > event outside from udev. That I can understand, though my opinion is 
> > > that the booting should not be special. If I am not mistaken, you are 
> > > trying to order the events (generated by writing to the uevent files), 
> > > which will be handled by udev:
> > >  - in parallel way
> > 
> > Right.
> > 
> > >  - in an unpredictable order
> > 
> > Wrong.
> > 
> > > What am I missing?
> > 
> > A clue.
> 
> Sorry, I still don't get it...
> 
> udevd receives the events, and starts to handle them in parallel.
> From that on, there are no guarantees about execution order, it all 
> depends on the linux kernel scheduler. Are there?

All events will be forked in order of arrival, not random. Events
for the same devpath will be serialized, one runs after the other.

Udevd throttles forking of childs and limits the running processes.
Later events will wait for execution, if there are already to many
in running state. If the event carries TIMEOUT, the event will run
immedediately, without queuing.

Events will not be forked, if a parent of the devpath is already
running, or the physical device event backing the device is still
running. Physical device chains in /sys/devices are all serialized
by it's devpath and run one after the other.

Another example, partition events will wait for the main block device
event to finish, so the partition can import the parents information
stored in the udev database. Therefore, the events for the main
devices will need to be triggered _before_ the partition devices and
udevd will take care of the right execution sequence.

For some events, special rules are needed, cause you want to setup all
block devices, before you setup a raid, and a lot of other crazy things
like this, especially for the not-so-common architectures and setups.

Believe me, you don't want to code that into a binary daemon and get
support calls to debug or work around this. I'm more than happy to
have all this in a few lines of shell script.

Kay


-------------------------------------------------------
This SF.net email is sponsored by: Splunk Inc. Do you grep through log files
for problems?  Stop!  Download the new AJAX search engine that makes
searching your log files as easy as surfing the  web.  DOWNLOAD SPLUNK!
http://ads.osdn.com/?ad_idv37&alloc_id\x16865&op=click
_______________________________________________
Linux-hotplug-devel mailing list  http://linux-hotplug.sourceforge.net
Linux-hotplug-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/linux-hotplug-devel

Re: waiting for an unknown set of udev /dev entries to complete

[Patrick Mansfield]

On Fri, Nov 18, 2005 at 02:30:45PM -0800, Patrick Mansfield wrote:

> With current FC rawhide/devel, I am working on iscsi install, and had udev
> "error" logging enabled on a medium speed dual processor system. So the
> scripts are very slow, and the partition setup code is being run right
> after iscsi scan (really discovery/login and then scan) and it does not
> see all the devices (and I probably have some bugs still). 

It ends up that it is the iSCSI scan is running very slowly, and that the
current open-iscsi returns if the scan takes more than 3 seconds (and also
leaves an unreaped iscsid process each time).

So, it was not that udev was slow here, it was that the iscsi login/scan
returned before the scan was even *complete*, so of course there were
still udev events in progress after the iscsadm login returned.

It took a while to notice since I had udev info (not error ...) logging
on, and missed the intermixed scsi discovery output. Plus, iscsid didn't
log any errors.

-- Patrick Mansfield


-------------------------------------------------------
This SF.net email is sponsored by: Splunk Inc. Do you grep through log files
for problems?  Stop!  Download the new AJAX search engine that makes
searching your log files as easy as surfing the  web.  DOWNLOAD SPLUNK!
http://ads.osdn.com/?ad_idv37&alloc_id\x16865&op=click
_______________________________________________
Linux-hotplug-devel mailing list  http://linux-hotplug.sourceforge.net
Linux-hotplug-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/linux-hotplug-devel

Re: waiting for an unknown set of udev /dev entries to complete

[Pozsar Balazs]

Hi Kay,

[sorry for the lag]

> All events will be forked in order of arrival, not random. Events
> for the same devpath will be serialized, one runs after the other.
> 
> Udevd throttles forking of childs and limits the running processes.
> Later events will wait for execution, if there are already to many
> in running state. If the event carries TIMEOUT, the event will run
> immedediately, without queuing.
> 
> Events will not be forked, if a parent of the devpath is already
> running, or the physical device event backing the device is still
> running. Physical device chains in /sys/devices are all serialized
> by it's devpath and run one after the other.
> 
> Another example, partition events will wait for the main block device
> event to finish, so the partition can import the parents information
> stored in the udev database. Therefore, the events for the main
> devices will need to be triggered _before_ the partition devices and
> udevd will take care of the right execution sequence.
> 
> For some events, special rules are needed, cause you want to setup all
> block devices, before you setup a raid, and a lot of other crazy things
> like this, especially for the not-so-common architectures and setups.


What I still do not understand is: why do you want to emit the class/mem 
and class/tty events first, and the block/md events last?


thanks,

-- 
pozsy


-------------------------------------------------------
This SF.net email is sponsored by: Splunk Inc. Do you grep through log files
for problems?  Stop!  Download the new AJAX search engine that makes
searching your log files as easy as surfing the  web.  DOWNLOAD SPLUNK!
http://ads.osdn.com/?ad_idv37&alloc_id\x16865&op=click
_______________________________________________
Linux-hotplug-devel mailing list  http://linux-hotplug.sourceforge.net
Linux-hotplug-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/linux-hotplug-devel

Re: waiting for an unknown set of udev /dev entries to complete

[Kay Sievers]

On Mon, Nov 28, 2005 at 11:09:55PM +0100, Pozsar Balazs wrote:
> > All events will be forked in order of arrival, not random. Events
> > for the same devpath will be serialized, one runs after the other.
> > 
> > Udevd throttles forking of childs and limits the running processes.
> > Later events will wait for execution, if there are already to many
> > in running state. If the event carries TIMEOUT, the event will run
> > immedediately, without queuing.
> > 
> > Events will not be forked, if a parent of the devpath is already
> > running, or the physical device event backing the device is still
> > running. Physical device chains in /sys/devices are all serialized
> > by it's devpath and run one after the other.
> > 
> > Another example, partition events will wait for the main block device
> > event to finish, so the partition can import the parents information
> > stored in the udev database. Therefore, the events for the main
> > devices will need to be triggered _before_ the partition devices and
> > udevd will take care of the right execution sequence.
> > 
> > For some events, special rules are needed, cause you want to setup all
> > block devices, before you setup a raid, and a lot of other crazy things
> > like this, especially for the not-so-common architectures and setups.
> 
> 
> What I still do not understand is: why do you want to emit the class/mem 
> and class/tty events first, and the block/md events last?

The md devices want all other block devices around before they are
set-up. The tty's are needed very early for all sort of tools, mem
like /dev/zero, /dev/mem, may be needed very eearly too.

The required tty and mem devices are usually part of the static setup,
copied to or created in /dev as the first action, so it may not be
needed. It's a left-over from udevstart, which started on a completely
empty /dev and wanted it around first, to be able to run tools for
other devices.

Kay


-------------------------------------------------------
This SF.net email is sponsored by: Splunk Inc. Do you grep through log files
for problems?  Stop!  Download the new AJAX search engine that makes
searching your log files as easy as surfing the  web.  DOWNLOAD SPLUNK!
http://ads.osdn.com/?ad_idv37&alloc_id\x16865&op=click
_______________________________________________
Linux-hotplug-devel mailing list  http://linux-hotplug.sourceforge.net
Linux-hotplug-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/linux-hotplug-devel