udev problem with udevsettle

udev problem with udevsettle

[Farkas Levente]

hi,
i've got a problem which i report in:
https://bugzilla.redhat.com/show_bug.cgi?idD9955
https://bugzilla.redhat.com/show_bug.cgi?idE1241
it's seems there is a deadlock between udev and cryptsetup. it happend
since cryptsetup 1.0.6 (which is the latest version) since this patch 
was added to it
http://article.gmane.org/gmane.linux.kernel.device-mapper.dm-crypt/2249/match=udevsettle
so now cryptsetup call udevsettle and as i call "cryptsetup luksOpen" 
from udev it seems there are some kind of deadlock.
here is my thread on dm-crypt list:
http://thread.gmane.org/gmane.linux.kernel.device-mapper.dm-crypt/2747
but it seems nobody able to solve the problem and nobody really 
understand the problem. imho it'd have to work as it's worked earlier.
that's why i wrote to here may be you can solve it or can tell us how 
can we solve this problem even in cryptsetup or in udev or in udevsettle 
(anyway there is not any kind of docs about udevsettle).
thank you for you help in advance.

-- 
   Levente                               "Si vis pacem para bellum!"

Re: udev problem with udevsettle

[Bryan Kadzban]

[-- Attachment #1: Type: text/plain, Size: 2049 bytes --]

On Thu, Jul 03, 2008 at 03:11:38PM +0200, Farkas Levente wrote:
> it's seems there is a deadlock between udev and cryptsetup.

Not exactly, see below.

> so now cryptsetup call udevsettle and as i call "cryptsetup luksOpen" 
> from udev it seems there are some kind of deadlock.

You shouldn't call cryptsetup from the udev RUN key.  From the udev(7)
manpage:

> RUN
>   <...> This can only be used for very short running tasks. Running an
>   event process for a long period of time may block all further events
>   for this or a dependent device. Long running tasks need to be
>   immediately detached from the event process itself.

When your RUN rule is executed, udev is in the middle of processing the
uevent for the USB storage device.  So while your script is running,
udev has not settled (the test that udevsettle uses to decide this will
never succeed, because the RUN rule that called udevsettle is not
finished).  To get the rule to finish processing, the program that RUN
ran needs to exit.  *Then* udevsettle will finish.

Yes, it's a deadlock, but the only way around it is to either (a) skip
calling udevsettle, or (b) don't call cryptsetup from a RUN key.  It
looks like the reason for calling udevsettle might be sound; if so, then
you'll need to either use something other than RUN, or detach.

It probably makes the most sense to just make your script run most of
its logic in the background, instead of the foreground.  I'm not sure
exactly what logic udevd uses, but I think it will work to write a
wrapper script that just runs your current script in the background:

#!/bin/bash
/path/to/current/script "$@" &

and then exits.

(Also, it's no longer "udevsettle" anymore.  It's "udevadm settle" now:
somebody should probably ask the cryptsetup people to change their
program to work with both names.  The udevsettle symlink is still
installed for now, but it wouldn't surprise me if it eventually goes
away.  It's in the "scheduled for removal" section of the udev TODO
file.)


[-- Attachment #2: Type: application/pgp-signature, Size: 189 bytes --]

Re: udev problem with udevsettle

[Marco d'Itri]

[-- Attachment #1: Type: text/plain, Size: 260 bytes --]

On Jul 03, Bryan Kadzban <bryan@kadzban.is-a-geek.net> wrote:

> #!/bin/bash
> /path/to/current/script "$@" &

This is quite inefficient. Much better:

#!/bin/sh -e

do_everything {
  echo do stuff
}

do_everything &
exit 0

-- 
ciao,
Marco

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 197 bytes --]

Re: udev problem with udevsettle

[Farkas Levente]

Bryan Kadzban wrote:
> You shouldn't call cryptsetup from the udev RUN key.  From the udev(7)
> manpage:
> 
>> RUN
>>   <...> This can only be used for very short running tasks. Running an
>>   event process for a long period of time may block all further events
>>   for this or a dependent device. Long running tasks need to be
>>   immediately detached from the event process itself.
> 
> When your RUN rule is executed, udev is in the middle of processing the
> uevent for the USB storage device.  So while your script is running,
> udev has not settled (the test that udevsettle uses to decide this will
> never succeed, because the RUN rule that called udevsettle is not
> finished).  To get the rule to finish processing, the program that RUN
> ran needs to exit.  *Then* udevsettle will finish.

yes i see it, but anyway what does the "very short running tasks" means? 
  my scripts is about a second (under normal circumstances) which is 
imho very short from a shell script:-)

> It probably makes the most sense to just make your script run most of
> its logic in the background, instead of the foreground.  I'm not sure
> exactly what logic udevd uses, but I think it will work to write a
> wrapper script that just runs your current script in the background:

yes i already solve it, but imho it's not a good habit to call 
udevsettle from cryptsetup.

-- 
   Levente                               "Si vis pacem para bellum!"

Re: udev problem with udevsettle

[Bryan Kadzban]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: RIPEMD160

Marco d'Itri wrote:
> On Jul 03, Bryan Kadzban <bryan@kadzban.is-a-geek.net> wrote:
> 
>> #!/bin/bash
>> /path/to/current/script "$@" &
> 
> This is quite inefficient. Much better:
> 
> #!/bin/sh -e
> 
> do_everything {
>   echo do stuff
> }
> 
> do_everything &
> exit 0

Ah, yes, that'd work fine as well, and would save at least one PID.  I
didn't even think to use a shell function (obviously).

Thanks!
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.7 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iD8DBQFIbUV9S5vET1Wea5wRAxJuAKC4cXU2rc2YoPvtIbgYTvpDLk6mogCgghTy
tqiH10ly5m6bPRCiqk+gCYA=UE5H
-----END PGP SIGNATURE-----

Re: udev problem with udevsettle

[Bryan Kadzban]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: RIPEMD160

Farkas Levente wrote:
> Bryan Kadzban wrote:
>> So while your script is running, udev has not settled (the test
>> that udevsettle uses to decide this will never succeed, because the
>> RUN rule that called udevsettle is not finished).  To get the rule
>> to finish processing, the program that RUN ran needs to exit.
> 
> yes i see it, but anyway what does the "very short running tasks"
> means?

It means that if your program takes a "long time" to run, it will delay
udevd's processing of this event any anything else that depends on it.
That's not a good thing to have happen.

As far as what the definition of "long time" is, that's not exactly
relevant to the issue here: the issue here is that udevsettle *will
never finish* as long as your RUN-key script hasn't finished.  That
means that it's automatically a "long time".

> my scripts is about a second (under normal circumstances) which is
> imho very short from a shell script:-)

For a shell script, maybe.  (But e.g. create_floppy_devices exits in a
*much* shorter time.)  But for a uevent, not really: most of the system
uevents are finished within a few seconds of running "udevadm trigger";
that's *way* less than a second per event.

But whatever: the issue isn't (directly) how long the script takes to
run normally.  The issue is that when the script runs from the RUN key
and doesn't fork into the background, the uevent has not finished (and
won't finish until the script exits), so any calls to udevsettle will
deadlock.  (And eventually time out.)

> yes i already solve it, but imho it's not a good habit to call 
> udevsettle from cryptsetup.

Very well -- and you may be correct -- but I believe the cryptsetup
maintainers are the ones that added that, not the udev maintainers.  ;-)

However, I am not so sure that it's a bad idea to call udevsettle from
where it's being called.  It's being called after cryptsetup creates a
new device-mapper device, to wait until udev has actually created the
correct device node for the new device-mapper device.  If you remove the
udevsettle call, you'll reintroduce the race that led to its addition in
the first place.

The problem is, udevsettle *can't* finish while a uevent is halfway
processed.  And running the script in the foreground means the uevent
will be halfway processed until the script finishes.  You need to just
run the script in the background.

(This should be fine, because the script is simply using RUN as a
notification.  Nothing about the script -- that I can see -- requires it
to run in the foreground.)
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.7 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iD8DBQFIbUlPS5vET1Wea5wRA06FAJ9xJl87hjoLrxLJ1upgV1mZh6bg3gCgtkgX
eEDnT++k9WjDH4FBuTHmHUQªAI
-----END PGP SIGNATURE-----