cold plugging

cold plugging

[Treeve Jelbert]

I am running kernel-2.6.15 and udev-80 in a build everything from source 
environment.

the kernel has lots of modules.

when the system is fully up, new hardware is detected and processed correctly 
by udev.

my problem concerns hardware which is present before udevd starts, eg usb, 
pcmcia, 1394, ....

udev does not seem to process any UEVENTS  for these things.  I tried 
udevstart, but this seems to have no effect.

Which program should be run to detect entries in /sys , trigger a UEVENT and 
hence the execution of the udev rules, in order to load modules and create 
device nodes?

I assume that somebody has a script or program that does this.

With previous kernels, I used udevsend as a hotplug driver and the relevant 
modules did get loaded.

-- 
Regards, Treeve


-------------------------------------------------------
This SF.net email is sponsored by: Splunk Inc. Do you grep through log files
for problems?  Stop!  Download the new AJAX search engine that makes
searching your log files as easy as surfing the  web.  DOWNLOAD SPLUNK!
http://ads.osdn.com/?ad_idv37&alloc_id\x16865&op=click
_______________________________________________
Linux-hotplug-devel mailing list  http://linux-hotplug.sourceforge.net
Linux-hotplug-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/linux-hotplug-devel

Re: cold plugging

[Kay Sievers]

On Fri, Jan 13, 2006 at 02:15:30PM +0100, Treeve Jelbert wrote:
> I am running kernel-2.6.15 and udev-80 in a build everything from source 
> environment.
> 
> the kernel has lots of modules.
> 
> when the system is fully up, new hardware is detected and processed correctly 
> by udev.
> 
> my problem concerns hardware which is present before udevd starts, eg usb, 
> pcmcia, 1394, ....
> 
> udev does not seem to process any UEVENTS  for these things.  I tried 
> udevstart, but this seems to have no effect.

Don't use udev, udevsend and udevstart anymore, if you don't really need
to. The 2.6.15 kernel and udevd is all you need.

> Which program should be run to detect entries in /sys , trigger a UEVENT and 
> hence the execution of the udev rules, in order to load modules and create 
> device nodes?
> 
> I assume that somebody has a script or program that does this.
> 
> With previous kernels, I used udevsend as a hotplug driver and the relevant 
> modules did get loaded.

No, this never worked with udevsend. At the time udevsend is executed and
the modules too, all early events are already lost unless you have recorded
them in initramfs.

You need to retrigger all events after you have started udevd, just
write "add" to all "uevent" files in /sys and the events will be sent to
the udev daemon, which will create the device nodes and run modprobe and
other tools to setup the devices.

I'm using this:
	list=$(echo /sys/bus/*/devices/*/uevent)
	list="$list $(echo /sys/class/*/*/uevent)"
	list="$list $(echo /sys/block/*/uevent /sys/block/*/*/uevent)"
	for i in $list; do
	    case "$i" in
		*/device/uevent|*\**)
		    continue
		    ;;
		*/class/mem/*|*/class/tty/*)
		    first="$first $i"
		    ;;
		*/block/md*)
		    last="$last $i"
		    ;;
		*/*)
		    default="$default $i"
		    ;;
	    esac
	done

	# trigger the sorted events
	for i in $first $default $last; do
	    echo "add" > "$i"
	done


To wait for the events to finish (if you need to), I do:
	loop00
	while test -d /dev/.udev/queue; do
	    sleep 0.1;
	    test "$loop" -gt 0 || break
	    loop=$(($loop - 1))
	done

Kay


-------------------------------------------------------
This SF.net email is sponsored by: Splunk Inc. Do you grep through log files
for problems?  Stop!  Download the new AJAX search engine that makes
searching your log files as easy as surfing the  web.  DOWNLOAD SPLUNK!
http://ads.osdn.com/?ad_idv37&alloc_id\x16865&op=click
_______________________________________________
Linux-hotplug-devel mailing list  http://linux-hotplug.sourceforge.net
Linux-hotplug-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/linux-hotplug-devel

Re: cold plugging

[Alexander E. Patrakov]

Kay Sievers wrote:

>I'm using this:
>	list=$(echo /sys/bus/*/devices/*/uevent)
>	list="$list $(echo /sys/class/*/*/uevent)"
>	list="$list $(echo /sys/block/*/uevent /sys/block/*/*/uevent)"
>	for i in $list; do
>	    case "$i" in
>		*/device/uevent|*\**)
>		    continue
>		    ;;
>		*/class/mem/*|*/class/tty/*)
>		    first="$first $i"
>		    ;;
>		*/block/md*)
>		    last="$last $i"
>		    ;;
>		*/*)
>		    default="$default $i"
>		    ;;
>	    esac
>	done
>
>	# trigger the sorted events
>	for i in $first $default $last; do
>	    echo "add" > "$i"
>	done
>  
>
Why sort the events? They arrive at udevd in random order anyway.

>To wait for the events to finish (if you need to), I do:
>	loop00
>	while test -d /dev/.udev/queue; do
>	    sleep 0.1;
>	    test "$loop" -gt 0 || break
>	    loop=$(($loop - 1))
>	done
>  
>
USB uevents easily get past this loop. I have a check in my script that 
sleeps 2 seconds after that and reports all new uevents in those 2 
seconds as possible bugs.

-- 
Alexander E. Patrakov


-------------------------------------------------------
This SF.net email is sponsored by: Splunk Inc. Do you grep through log files
for problems?  Stop!  Download the new AJAX search engine that makes
searching your log files as easy as surfing the  web.  DOWNLOAD SPLUNK!
http://ads.osdn.com/?ad_idv37&alloc_id\x16865&op=click
_______________________________________________
Linux-hotplug-devel mailing list  http://linux-hotplug.sourceforge.net
Linux-hotplug-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/linux-hotplug-devel

Re: cold plugging

[Darren Salt]

I demand that Alexander E. Patrakov may or may not have written...

> Kay Sievers wrote:
[snip]
>> To wait for the events to finish (if you need to), I do:
>> 	loop00
>> 	while test -d /dev/.udev/queue; do
>> 	    sleep 0.1;
>> 	    test "$loop" -gt 0 || break
>> 	    loop=$(($loop - 1))
>> 	done

> USB uevents easily get past this loop.

Also slower machines. I have a machine (PII 366) here which has randomly
failed to create /dev/hda* in time, causing problems for fsck; doubling the
timeout avoids the problem.

[snip]
-- 
| Darren Salt | nr. Ashington, | d youmustbejoking,demon,co,uk
| Debian,     | Northumberland | s zap,tartarus,org
| RISC OS     | Toon Army      | @
|   Kill all extremists!

G No room for line, 0:1


-------------------------------------------------------
This SF.net email is sponsored by: Splunk Inc. Do you grep through log files
for problems?  Stop!  Download the new AJAX search engine that makes
searching your log files as easy as surfing the  web.  DOWNLOAD SPLUNK!
http://ads.osdn.com/?ad_idv37&alloc_id\x16865&op=click
_______________________________________________
Linux-hotplug-devel mailing list  http://linux-hotplug.sourceforge.net
Linux-hotplug-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/linux-hotplug-devel

Re: cold plugging

[Greg KH]

On Fri, Jan 13, 2006 at 09:23:51PM +0500, Alexander E. Patrakov wrote:
> Kay Sievers wrote:
> 
> >I'm using this:
> >	list=$(echo /sys/bus/*/devices/*/uevent)
> >	list="$list $(echo /sys/class/*/*/uevent)"
> >	list="$list $(echo /sys/block/*/uevent /sys/block/*/*/uevent)"
> >	for i in $list; do
> >	    case "$i" in
> >		*/device/uevent|*\**)
> >		    continue
> >		    ;;
> >		*/class/mem/*|*/class/tty/*)
> >		    first="$first $i"
> >		    ;;
> >		*/block/md*)
> >		    last="$last $i"
> >		    ;;
> >		*/*)
> >		    default="$default $i"
> >		    ;;
> >	    esac
> >	done
> >
> >	# trigger the sorted events
> >	for i in $first $default $last; do
> >	    echo "add" > "$i"
> >	done
> > 
> >
> Why sort the events? They arrive at udevd in random order anyway.

Because many things need /dev/null and the tty device nodes in their
handlers and scripts, so they need to be created first.

Try it and see if you don't believe us :)

> >To wait for the events to finish (if you need to), I do:
> >	loop00
> >	while test -d /dev/.udev/queue; do
> >	    sleep 0.1;
> >	    test "$loop" -gt 0 || break
> >	    loop=$(($loop - 1))
> >	done
> > 
> >
> USB uevents easily get past this loop. I have a check in my script that 
> sleeps 2 seconds after that and reports all new uevents in those 2 
> seconds as possible bugs.

Why does USB take longer for you?  What is taking such a long time?

thanks,

greg k-h


-------------------------------------------------------
This SF.net email is sponsored by: Splunk Inc. Do you grep through log files
for problems?  Stop!  Download the new AJAX search engine that makes
searching your log files as easy as surfing the  web.  DOWNLOAD SPLUNK!
http://ads.osdn.com/?ad_idv37&alloc_id\x16865&op=click
_______________________________________________
Linux-hotplug-devel mailing list  http://linux-hotplug.sourceforge.net
Linux-hotplug-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/linux-hotplug-devel

Re: cold plugging

[Scott James Remnant]

[-- Attachment #1: Type: text/plain, Size: 827 bytes --]

On Fri, 2006-01-13 at 18:47 +0000, Darren Salt wrote:

> I demand that Alexander E. Patrakov may or may not have written...
> 
> > Kay Sievers wrote:
> [snip]
> >> To wait for the events to finish (if you need to), I do:
> >> 	loop=300
> >> 	while test -d /dev/.udev/queue; do
> >> 	    sleep 0.1;
> >> 	    test "$loop" -gt 0 || break
> >> 	    loop=$(($loop - 1))
> >> 	done
> 
> > USB uevents easily get past this loop.
> 
> Also slower machines. I have a machine (PII 366) here which has randomly
> failed to create /dev/hda* in time, causing problems for fsck; doubling the
> timeout avoids the problem.
> 
This is almost certainly due to the buggy IDE subsystem patch in
Debian's kernel.

Scott
-- 
Have you ever, ever felt like this?
Had strange things happen?  Are you going round the twist?

[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 189 bytes --]

Re: cold plugging

[Marco d'Itri]

[-- Attachment #1: Type: text/plain, Size: 211 bytes --]

On Jan 13, Scott James Remnant <scott@netsplit.com> wrote:

> This is almost certainly due to the buggy IDE subsystem patch in
> Debian's kernel.
I understand that it has been removed.

-- 
ciao,
Marco

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 189 bytes --]

Re: cold plugging

[Darren Salt]

[context restored]

I demand that Marco d'Itri may or may not have written...

> On Jan 13, Scott James Remnant <scott@netsplit.com> wrote:
>> On Fri, 2006-01-13 at 18:47 +0000, Darren Salt wrote:
>>> I demand that Alexander E. Patrakov may or may not have written...
[snip; udev initial population delay timeout]
>>>> USB uevents easily get past this loop.
>>> Also slower machines. I have a machine (PII 366) here which has randomly
>>> failed to create /dev/hda* in time, causing problems for fsck; doubling
>>> the timeout avoids the problem.

>> This is almost certainly due to the buggy IDE subsystem patch in Debian's
>> kernel.

> I understand that it has been removed.

It's a long time since I used a Debian kernel for anything other than
installation. (I do use make-kpkg, though.) I might be missing out on a patch
here and there, but I can live with that :-)

My kernels are somewhat monolithic, the intent being to be able to mount
filesystems from init=/bin/sh (given root=/dev/hda*) without needing to load
any modules.

FWIW, the machine in question is currently running 2.6.15+dyntick and udev
0.079-1.

-- 
| Darren Salt | nr. Ashington, | d youmustbejoking,demon,co,uk
| Debian,     | Northumberland | s zap,tartarus,org
| RISC OS     | Toon Army      | @
|   We've got Souness, we don't want him

You are taking advantage of the good nature of a friend. Be careful.


-------------------------------------------------------
This SF.net email is sponsored by: Splunk Inc. Do you grep through log files
for problems?  Stop!  Download the new AJAX search engine that makes
searching your log files as easy as surfing the  web.  DOWNLOAD SPLUNK!
http://ads.osdn.com/?ad_idv37&alloc_id\x16865&op=click
_______________________________________________
Linux-hotplug-devel mailing list  http://linux-hotplug.sourceforge.net
Linux-hotplug-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/linux-hotplug-devel

Re: cold plugging

[Pozsar Balazs]

On Fri, Jan 13, 2006 at 10:52:03AM -0800, Greg KH wrote:
> On Fri, Jan 13, 2006 at 09:23:51PM +0500, Alexander E. Patrakov wrote:
> > Kay Sievers wrote:
> > 
> > >I'm using this:
> > >	list=$(echo /sys/bus/*/devices/*/uevent)
> > >	list="$list $(echo /sys/class/*/*/uevent)"
> > >	list="$list $(echo /sys/block/*/uevent /sys/block/*/*/uevent)"
> > >	for i in $list; do
> > >	    case "$i" in
> > >		*/device/uevent|*\**)
> > >		    continue
> > >		    ;;
> > >		*/class/mem/*|*/class/tty/*)
> > >		    first="$first $i"
> > >		    ;;
> > >		*/block/md*)
> > >		    last="$last $i"
> > >		    ;;
> > >		*/*)
> > >		    default="$default $i"
> > >		    ;;
> > >	    esac
> > >	done
> > >
> > >	# trigger the sorted events
> > >	for i in $first $default $last; do
> > >	    echo "add" > "$i"
> > >	done
> > > 
> > >
> > Why sort the events? They arrive at udevd in random order anyway.
> 
> Because many things need /dev/null and the tty device nodes in their
> handlers and scripts, so they need to be created first.

But the above script does not guarantee that /dev/null will be created 
first. It only orders the events, but as events are processed in 
parallel, it may well happen that a later event gets processed earlier.


-- 
pozsy


-------------------------------------------------------
This SF.net email is sponsored by: Splunk Inc. Do you grep through log files
for problems?  Stop!  Download the new AJAX search engine that makes
searching your log files as easy as surfing the  web.  DOWNLOAD SPLUNK!
http://ads.osdn.com/?ad_idv37&alloc_id\x16865&op=click
_______________________________________________
Linux-hotplug-devel mailing list  http://linux-hotplug.sourceforge.net
Linux-hotplug-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/linux-hotplug-devel

Re: cold plugging

[Greg KH]

On Sat, Jan 14, 2006 at 12:29:45AM +0100, Pozsar Balazs wrote:
> On Fri, Jan 13, 2006 at 10:52:03AM -0800, Greg KH wrote:
> > On Fri, Jan 13, 2006 at 09:23:51PM +0500, Alexander E. Patrakov wrote:
> > > Kay Sievers wrote:
> > > 
> > > >I'm using this:
> > > >	list=$(echo /sys/bus/*/devices/*/uevent)
> > > >	list="$list $(echo /sys/class/*/*/uevent)"
> > > >	list="$list $(echo /sys/block/*/uevent /sys/block/*/*/uevent)"
> > > >	for i in $list; do
> > > >	    case "$i" in
> > > >		*/device/uevent|*\**)
> > > >		    continue
> > > >		    ;;
> > > >		*/class/mem/*|*/class/tty/*)
> > > >		    first="$first $i"
> > > >		    ;;
> > > >		*/block/md*)
> > > >		    last="$last $i"
> > > >		    ;;
> > > >		*/*)
> > > >		    default="$default $i"
> > > >		    ;;
> > > >	    esac
> > > >	done
> > > >
> > > >	# trigger the sorted events
> > > >	for i in $first $default $last; do
> > > >	    echo "add" > "$i"
> > > >	done
> > > > 
> > > >
> > > Why sort the events? They arrive at udevd in random order anyway.
> > 
> > Because many things need /dev/null and the tty device nodes in their
> > handlers and scripts, so they need to be created first.
> 
> But the above script does not guarantee that /dev/null will be created 
> first. It only orders the events, but as events are processed in 
> parallel, it may well happen that a later event gets processed earlier.

Have you seen this happen?

thanks,

greg k-h


-------------------------------------------------------
This SF.net email is sponsored by: Splunk Inc. Do you grep through log files
for problems?  Stop!  Download the new AJAX search engine that makes
searching your log files as easy as surfing the  web.  DOWNLOAD SPLUNK!
http://ads.osdn.com/?ad_idv37&alloc_id\x16865&op=click
_______________________________________________
Linux-hotplug-devel mailing list  http://linux-hotplug.sourceforge.net
Linux-hotplug-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/linux-hotplug-devel

Re: cold plugging

[Alexander E. Patrakov]

Greg KH wrote:

>Why does USB take longer for you?  What is taking such a long time?
>  
>
The /dev/bus/usb/... node for my Mustek Plug-N-Scan 1200 UB Plus scanner.

-- 
Alexander E. Patrakov



-------------------------------------------------------
This SF.net email is sponsored by: Splunk Inc. Do you grep through log files
for problems?  Stop!  Download the new AJAX search engine that makes
searching your log files as easy as surfing the  web.  DOWNLOAD SPLUNK!
http://ads.osdn.com/?ad_idv37&alloc_id\x16865&op=click
_______________________________________________
Linux-hotplug-devel mailing list  http://linux-hotplug.sourceforge.net
Linux-hotplug-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/linux-hotplug-devel

Re: cold plugging

[Alexander E. Patrakov]

Greg KH wrote:

>On Sat, Jan 14, 2006 at 12:29:45AM +0100, Pozsar Balazs wrote:
>  
>
>>But the above script does not guarantee that /dev/null will be created 
>>first. It only orders the events, but as events are processed in 
>>parallel, it may well happen that a later event gets processed earlier.
>>    
>>
>
>Have you seen this happen?
>  
>
Yes, /dev/null is never created first even if echoing to uevent files is 
ordered by Kay's script. There are always three or four messages on boot 
about /dev/null not being found. I solved this by creating 
/lib/udev/devices/null.

-- 
Alexander E. Patrakov


-------------------------------------------------------
This SF.net email is sponsored by: Splunk Inc. Do you grep through log files
for problems?  Stop!  Download the new AJAX search engine that makes
searching your log files as easy as surfing the  web.  DOWNLOAD SPLUNK!
http://ads.osdn.com/?ad_idv37&alloc_id\x16865&op=click
_______________________________________________
Linux-hotplug-devel mailing list  http://linux-hotplug.sourceforge.net
Linux-hotplug-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/linux-hotplug-devel

Re: cold plugging

[Pozsar Balazs]

On Fri, Jan 13, 2006 at 03:53:33PM -0800, Greg KH wrote:
> > > > Why sort the events? They arrive at udevd in random order anyway.
> > > 
> > > Because many things need /dev/null and the tty device nodes in their
> > > handlers and scripts, so they need to be created first.
> > 
> > But the above script does not guarantee that /dev/null will be created 
> > first. It only orders the events, but as events are processed in 
> > parallel, it may well happen that a later event gets processed earlier.
> 
> Have you seen this happen?

To be honest: no, I just wanted to make a point about theoretical 
incorrectness.
One solution could be to wait after each pass for udev to finish 
processing events just like after sending all the events in the above 
script.


-- 
pozsy


-------------------------------------------------------
This SF.net email is sponsored by: Splunk Inc. Do you grep through log files
for problems?  Stop!  Download the new AJAX search engine that makes
searching your log files as easy as surfing the  web.  DOWNLOAD SPLUNK!
http://ads.osdn.com/?ad_idv37&alloc_id\x16865&op=click
_______________________________________________
Linux-hotplug-devel mailing list  http://linux-hotplug.sourceforge.net
Linux-hotplug-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/linux-hotplug-devel

Re: cold plugging

[Kay Sievers]

On Sat, Jan 14, 2006 at 10:00:29AM +0500, Alexander E. Patrakov wrote:
> Greg KH wrote:
> 
> >Why does USB take longer for you?  What is taking such a long time?
> > 
> >
> The /dev/bus/usb/... node for my Mustek Plug-N-Scan 1200 UB Plus scanner.

It's the simplest way to wait for all events. Depending how you handle
dependencies/checkoints on bootup, it may be sudfficient if all localfs
devices have appeared and you can let the other events finish asynchronously.

Kay


-------------------------------------------------------
This SF.net email is sponsored by: Splunk Inc. Do you grep through log files
for problems?  Stop!  Download the new AJAX search engine that makes
searching your log files as easy as surfing the  web.  DOWNLOAD SPLUNK!
http://ads.osdn.com/?ad_idv37&alloc_id\x16865&op=click
_______________________________________________
Linux-hotplug-devel mailing list  http://linux-hotplug.sourceforge.net
Linux-hotplug-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/linux-hotplug-devel

Re: cold plugging

[Kay Sievers]

On Sat, Jan 14, 2006 at 10:57:46AM +0500, Alexander E. Patrakov wrote:
> Greg KH wrote:
> 
> >On Sat, Jan 14, 2006 at 12:29:45AM +0100, Pozsar Balazs wrote:
> > 
> >
> >>But the above script does not guarantee that /dev/null will be created 
> >>first. It only orders the events, but as events are processed in 
> >>parallel, it may well happen that a later event gets processed earlier.
> >
> >Have you seen this happen?
> > 
> Yes, /dev/null is never created first even if echoing to uevent files is 
> ordered by Kay's script. There are always three or four messages on boot 
> about /dev/null not being found. I solved this by creating 
> /lib/udev/devices/null.

Sure, /dev/null is usually created/copied by the first init script
itself, but that's not what Greg meant, you may want to run pretty
complex software for bringing up all kinds of block devices and similar,
so it's just safer to have the random, zero, mem, ... and the tty's
around, before the block devices are running. The order is mainly a
copy of what udevstart was doing in the past.

With the async events, you can wait at any point during coldplug event
triggering for the events to finish. We need to find out, what's the
best approch to do this, that's the reason none of the current alternatives
is in the udev tree itself today. I wanted to start simple and this
stupid shell script snippet, works fine for me. We'll see with what
we'll end with and if we can agree on some default, we all can use,
which would be nice.

Thanks,
Kay


-------------------------------------------------------
This SF.net email is sponsored by: Splunk Inc. Do you grep through log files
for problems?  Stop!  Download the new AJAX search engine that makes
searching your log files as easy as surfing the  web.  DOWNLOAD SPLUNK!
http://ads.osdn.com/?ad_idv37&alloc_id\x16865&op=click
_______________________________________________
Linux-hotplug-devel mailing list  http://linux-hotplug.sourceforge.net
Linux-hotplug-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/linux-hotplug-devel

Re: cold plugging

[Alexander E. Patrakov]

Kay Sievers wrote:

>On Sat, Jan 14, 2006 at 10:00:29AM +0500, Alexander E. Patrakov wrote:
>  
>
>>Greg KH wrote:
>>
>>    
>>
>>>Why does USB take longer for you?  What is taking such a long time?
>>>      
>>>
>>The /dev/bus/usb/... node for my Mustek Plug-N-Scan 1200 UB Plus scanner.
>>    
>>
>
>It's the simplest way to wait for all events. Depending how you handle
>dependencies/checkoints on bootup, it may be sudfficient if all localfs
>devices have appeared and you can let the other events finish asynchronously.
>  
>
Thanks, but you probably missed the point. Your loop does NOT wait for 
all events. I think that in my case the following happens:

1) The script creates an empty /dev/.udev/queue directory
2) The script causes the kernel to emit uevents
3) udevd gets those uevents, and reacts upon them. This reaction 
involves, among other things, making device nodes and calling 
/sbin/modprobe to load new drivers.
4) Those drivers are bound to devices by the kernel, this produces new 
uevents, go to (3).
5) When the internal queue of udevd becomes empty, it removes the 
/dev/.udev/queue directory and the loop exits.

The problem is that some modules take longer than 0.1s to detect their 
hardware, so there is a period of time longer than 0.1s with the empty 
udevd queue, after which new uevents arrive.

6) such late uevents reach udevd after the loop ends, and then it may be 
too late (but it is OK for me to ignore this for my scanner).

A "solution" is to rewrite the script so that it repeatedly checks that 
the queue stays empty for some period of time (i.e., retests the 
existence of /dev/.udev/queue several times with delays between them) 
instead of exiting immediately when the queue becomes empty for a moment.

-- 
Alexander E. Patrakov


-------------------------------------------------------
This SF.net email is sponsored by: Splunk Inc. Do you grep through log files
for problems?  Stop!  Download the new AJAX search engine that makes
searching your log files as easy as surfing the  web.  DOWNLOAD SPLUNK!
http://ads.osdn.com/?ad_idv37&alloc_id\x16865&op=click
_______________________________________________
Linux-hotplug-devel mailing list  http://linux-hotplug.sourceforge.net
Linux-hotplug-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/linux-hotplug-devel

Re: cold plugging

[Kay Sievers]

On Sat, Jan 14, 2006 at 06:10:48PM +0500, Alexander E. Patrakov wrote:
> Kay Sievers wrote:
> >On Sat, Jan 14, 2006 at 10:00:29AM +0500, Alexander E. Patrakov wrote:
> >>Greg KH wrote:
> >>
> >>>Why does USB take longer for you?  What is taking such a long time?
> >>>
> >>The /dev/bus/usb/... node for my Mustek Plug-N-Scan 1200 UB Plus scanner.
> >
> >It's the simplest way to wait for all events. Depending how you handle
> >dependencies/checkoints on bootup, it may be sudfficient if all localfs
> >devices have appeared and you can let the other events finish 
> >asynchronously.
> > 
> >
> Thanks, but you probably missed the point. Your loop does NOT wait for 
> all events. I think that in my case the following happens:
> 
> 1) The script creates an empty /dev/.udev/queue directory
> 2) The script causes the kernel to emit uevents
> 3) udevd gets those uevents, and reacts upon them. This reaction 
> involves, among other things, making device nodes and calling 
> /sbin/modprobe to load new drivers.
> 4) Those drivers are bound to devices by the kernel, this produces new 
> uevents, go to (3).
> 5) When the internal queue of udevd becomes empty, it removes the 
> /dev/.udev/queue directory and the loop exits.
> 
> The problem is that some modules take longer than 0.1s to detect their 
> hardware, so there is a period of time longer than 0.1s with the empty 
> udevd queue, after which new uevents arrive.

Which modprobe for which module is it, that returns before events for
the devices are generated?

Kay


-------------------------------------------------------
This SF.net email is sponsored by: Splunk Inc. Do you grep through log files
for problems?  Stop!  Download the new AJAX search engine that makes
searching your log files as easy as surfing the  web.  DOWNLOAD SPLUNK!
http://ads.osdn.com/?ad_idv37&alloc_id\x16865&op=click
_______________________________________________
Linux-hotplug-devel mailing list  http://linux-hotplug.sourceforge.net
Linux-hotplug-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/linux-hotplug-devel

Re: cold plugging

[Alexander E. Patrakov]

(reposting from a different address, since SpamCop blacklisted the SMTP 
server and the message sent from patrakov@ums.usu.ru didn't reach the 
list. Sorry if you get this twice.)

Kay Sievers wrote:

>On Sat, Jan 14, 2006 at 06:10:48PM +0500, Alexander E. Patrakov wrote:
>  
>
>>Kay Sievers wrote:
>>    
>>
>>>On Sat, Jan 14, 2006 at 10:00:29AM +0500, Alexander E. Patrakov wrote:
>>>      
>>>
>>>>Greg KH wrote:
>>>>
>>>>        
>>>>
>>>>>Why does USB take longer for you?  What is taking such a long time?
>>>>>
>>>>>          
>>>>>
>>>>The /dev/bus/usb/... node for my Mustek Plug-N-Scan 1200 UB Plus scanner.
>>>>        
>>>>
>>>It's the simplest way to wait for all events. Depending how you handle
>>>dependencies/checkoints on bootup, it may be sudfficient if all localfs
>>>devices have appeared and you can let the other events finish 
>>>asynchronously.
>>>
>>>
>>>      
>>>
>>Thanks, but you probably missed the point. Your loop does NOT wait for 
>>all events. I think that in my case the following happens:
>>
>>1) The script creates an empty /dev/.udev/queue directory
>>2) The script causes the kernel to emit uevents
>>3) udevd gets those uevents, and reacts upon them. This reaction 
>>involves, among other things, making device nodes and calling 
>>/sbin/modprobe to load new drivers.
>>4) Those drivers are bound to devices by the kernel, this produces new 
>>uevents, go to (3).
>>5) When the internal queue of udevd becomes empty, it removes the 
>>/dev/.udev/queue directory and the loop exits.
>>
>>The problem is that some modules take longer than 0.1s to detect their 
>>hardware, so there is a period of time longer than 0.1s with the empty 
>>udevd queue, after which new uevents arrive.
>>    
>>
>
>Which modprobe for which module is it, that returns before events for
>the devices are generated?
>
>  
>
Sorry, after playing with udev versions and changing rules, I lost my
ability to reproduce this with my scanner alone. But this does happen
when I boot with both my scanner and my MP3 player plugged in. Leaked
events (i.e.: those that happen within 2 seconds after the end of the
loop) are pasted at the end of this mail, so I assume that it is related
with usb_storage for this report. Note that if I boot with only one of
those devices, nothing leaks.

UDEV_LOG=0
ACTION­d
DEVPATH=/devices/pci0000:00/0000:00:07.2/usb1/1-2
SUBSYSTEM=usb
SEQNUM\x1518
PHYSDEVBUS=usb
PHYSDEVDRIVER=usb
- -------------------------------------
UDEV_LOG=0
ACTION­d
DEVPATH=/devices/pci0000:00/0000:00:07.2/usb1/1-2/1-2:1.0
SUBSYSTEM=usb
SEQNUM\x1519
PHYSDEVBUS=usb
DEVICE=/proc/bus/usb/001/003
PRODUCTA9/a002/100
TYPE=0/0/0
INTERFACE=8/6/80
MODALIAS=usb:v0419pA002d0100dc00dsc00dp00ic08isc06ip50
UDEVD_EVENT=1
- -------------------------------------
UDEV_LOG=0
ACTION­d
DEVPATH=/class/usb_device/usbdev1.3
SUBSYSTEM=usb_device
SEQNUM\x1520
PHYSDEVPATH=/devices/pci0000:00/0000:00:07.2/usb1/1-2
PHYSDEVBUS=usb
PHYSDEVDRIVER=usb
MAJOR\x189
MINOR=2
UDEVD_EVENT=1
DEVNAME=/dev/bus/usb/001/003
- -------------------------------------
UDEV_LOG=0
ACTION­d
DEVPATH=/class/scsi_host/host0
SUBSYSTEM=scsi_host
SEQNUM\x1526
PHYSDEVPATH=/devices/pci0000:00/0000:00:07.2/usb1/1-2/1-2:1.0/host0
UDEVD_EVENT=1

-- 
Alexander E. Patrakov


-------------------------------------------------------
This SF.net email is sponsored by: Splunk Inc. Do you grep through log files
for problems?  Stop!  Download the new AJAX search engine that makes
searching your log files as easy as surfing the  web.  DOWNLOAD SPLUNK!
http://ads.osdn.com/?ad_idv37&alloc_id\x16865&op=click
_______________________________________________
Linux-hotplug-devel mailing list  http://linux-hotplug.sourceforge.net
Linux-hotplug-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/linux-hotplug-devel

Re: cold plugging

[Dmitry Torokhov]

On Saturday 14 January 2006 08:27, Kay Sievers wrote:
> 
> Which modprobe for which module is it, that returns before events for
> the devices are generated?
> 

Everything that has to do with serio - i8042, atkbd, psmouse, etc.
Device creation and driver binding/unbinding is offloaded to kseriod
thread so it is quite likely that module init function returns early.

-- 
Dmitry


-------------------------------------------------------
This SF.net email is sponsored by: Splunk Inc. Do you grep through log files
for problems?  Stop!  Download the new AJAX search engine that makes
searching your log files as easy as surfing the  web.  DOWNLOAD SPLUNK!
http://ads.osdn.com/?ad_idv37&alloc_id\x16865&op=click
_______________________________________________
Linux-hotplug-devel mailing list  http://linux-hotplug.sourceforge.net
Linux-hotplug-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/linux-hotplug-devel

Re: cold plugging

[Aras Vaichas]

Alexander E. Patrakov wrote:
> Thanks, but you probably missed the point. Your loop does NOT wait for 
> all events. I think that in my case the following happens:
> 
> 1) The script creates an empty /dev/.udev/queue directory
> 2) The script causes the kernel to emit uevents
> 3) udevd gets those uevents, and reacts upon them. This reaction 
> involves, among other things, making device nodes and calling 
> /sbin/modprobe to load new drivers.
> 4) Those drivers are bound to devices by the kernel, this produces new 
> uevents, go to (3).
> 5) When the internal queue of udevd becomes empty, it removes the 
> /dev/.udev/queue directory and the loop exits.
> 
> The problem is that some modules take longer than 0.1s to detect their 
> hardware, so there is a period of time longer than 0.1s with the empty 
> udevd queue, after which new uevents arrive.
>
> 6) such late uevents reach udevd after the loop ends, and then it may be 
> too late (but it is OK for me to ignore this for my scanner).

I had a similar problem with a USB Flash drive that had a delay in the driver 
for it to "warm up". I just had a larger delay (and less cycles) in my queue 
polling script and it works OK. The worst case scenario is that this loop 
delays the boot for 1 second, which I can live with.

mkdir -p /dev/.udev/queue
udevsynthesize
udevd_timeout\x10
while [ -d /dev/.udev/queue/ ]; do
         /usr/bin/sleep 1
         udevd_timeout=$(($udevd_timeout - 1))
         if [ $udevd_timeout -eq 0 ]; then
                 echo "timeout"
                 break
         fi
done
echo "coldplug complete"

regards,

Aras Vaichas


-------------------------------------------------------
This SF.net email is sponsored by: Splunk Inc. Do you grep through log files
for problems?  Stop!  Download the new AJAX search engine that makes
searching your log files as easy as surfing the  web.  DOWNLOAD SPLUNK!
http://ads.osdn.com/?ad_idv37&alloc_id\x16865&op=click
_______________________________________________
Linux-hotplug-devel mailing list  http://linux-hotplug.sourceforge.net
Linux-hotplug-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/linux-hotplug-devel

Re: cold plugging

[Kay Sievers]

On Mon, Jan 16, 2006 at 09:44:38AM +1100, Aras Vaichas wrote:
> Alexander E. Patrakov wrote:
> >Thanks, but you probably missed the point. Your loop does NOT wait for 
> >all events. I think that in my case the following happens:
> >
> >1) The script creates an empty /dev/.udev/queue directory
> >2) The script causes the kernel to emit uevents
> >3) udevd gets those uevents, and reacts upon them. This reaction 
> >involves, among other things, making device nodes and calling 
> >/sbin/modprobe to load new drivers.
> >4) Those drivers are bound to devices by the kernel, this produces new 
> >uevents, go to (3).
> >5) When the internal queue of udevd becomes empty, it removes the 
> >/dev/.udev/queue directory and the loop exits.
> >
> >The problem is that some modules take longer than 0.1s to detect their 
> >hardware, so there is a period of time longer than 0.1s with the empty 
> >udevd queue, after which new uevents arrive.
> >
> >6) such late uevents reach udevd after the loop ends, and then it may be 
> >too late (but it is OK for me to ignore this for my scanner).
> 
> I had a similar problem with a USB Flash drive that had a delay in the 
> driver for it to "warm up". I just had a larger delay (and less cycles) in 
> my queue polling script and it works OK.

Sure, this can happen with usb_storage. Secondary events, that arrive
after a module has scheduled device creation and return immediately,
have never been handled and there is no sane way to do it for udev. 

I'm sure we don't want to introduce another round of sleep 1 to the
device management, like the nightmare we had in the past. The right
way to solve that - if you really depend on these devices at all - is
to have certain checkpoint during boot that have the dependency on
these devices recorded.

Ususal setups don't depend on scanners and such stuff to be fully
initialized before you can continue with booting, you can perfectly
do this asynchronously. If you depend on a usb-storage volume, you
better put that in fstab and let your "localfs" boot script wait until
all reuqirements are fulfilled, instead of guessing how long it could
take that "all" events have finished.

Thanks,
Kay


-------------------------------------------------------
This SF.net email is sponsored by: Splunk Inc. Do you grep through log files
for problems?  Stop!  Download the new AJAX search engine that makes
searching your log files as easy as surfing the  web.  DOWNLOAD SPLUNK!
http://ads.osdn.com/?ad_idv37&alloc_id\x16865&op=click
_______________________________________________
Linux-hotplug-devel mailing list  http://linux-hotplug.sourceforge.net
Linux-hotplug-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/linux-hotplug-devel

Re: cold plugging

[Kay Sievers]

On Sat, Jan 14, 2006 at 11:13:40PM -0500, Dmitry Torokhov wrote:
> On Saturday 14 January 2006 08:27, Kay Sievers wrote:
> > 
> > Which modprobe for which module is it, that returns before events for
> > the devices are generated?
> 
> Everything that has to do with serio - i8042, atkbd, psmouse, etc.
> Device creation and driver binding/unbinding is offloaded to kseriod
> thread so it is quite likely that module init function returns early.

Ah, good to know that "input" does that too. usb-storage does that
for long, it has a default internal timeout of 5 seconds to let the
device "settle down". The modprobe returns before the devices are
created.

Thanks,
Kay


-------------------------------------------------------
This SF.net email is sponsored by: Splunk Inc. Do you grep through log files
for problems?  Stop!  Download the new AJAX search engine that makes
searching your log files as easy as surfing the  web.  DOWNLOAD SPLUNK!
http://ads.osdn.com/?ad_idv37&alloc_id\x16865&op=click
_______________________________________________
Linux-hotplug-devel mailing list  http://linux-hotplug.sourceforge.net
Linux-hotplug-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/linux-hotplug-devel

Re: cold plugging

[Olivier Blin]

Kay Sievers <kay.sievers@vrfy.org> writes:

>> >6) such late uevents reach udevd after the loop ends, and then it may be 
>> >too late (but it is OK for me to ignore this for my scanner).
>> 
>> I had a similar problem with a USB Flash drive that had a delay in the 
>> driver for it to "warm up". I just had a larger delay (and less cycles) in 
>> my queue polling script and it works OK.
>
> Sure, this can happen with usb_storage. Secondary events, that arrive
> after a module has scheduled device creation and return immediately,
> have never been handled and there is no sane way to do it for udev. 
>
> I'm sure we don't want to introduce another round of sleep 1 to the
> device management, like the nightmare we had in the past. The right
> way to solve that - if you really depend on these devices at all - is
> to have certain checkpoint during boot that have the dependency on
> these devices recorded.

We've added some workarounds in Mandriva's rc.sysinit to wait for
usb/firewire devices if needed:

# wait for usb and firewire storage scanning processes to finish
[ -n "$needusbstorage" ] && while ps -eocomm | grep -q usb-stor-scan; do sleep 1; done
[ -n "$needfirewirestorage" ] && while ps -eocomm | grep -q kfwrescan; do sleep 1; done

-- 
Olivier Blin - Mandriva



-------------------------------------------------------
This SF.net email is sponsored by: Splunk Inc. Do you grep through log files
for problems?  Stop!  Download the new AJAX search engine that makes
searching your log files as easy as surfing the  web.  DOWNLOAD SPLUNK!
http://ads.osdn.com/?ad_idv37&alloc_id\x16865&op=click
_______________________________________________
Linux-hotplug-devel mailing list  http://linux-hotplug.sourceforge.net
Linux-hotplug-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/linux-hotplug-devel