Re: Report: Threaded udevd

[Alan Jenkins <alan-jenkins@tuffmail.co.uk>]

Scott James Remnant wrote:
> On Wed, 2008-10-22 at 15:10 +0100, Alan Jenkins wrote:
>
>   
>> = Justification for udev-exec =
>>
>> External programs are run from a "udev-exec" subprocess.  This was
>> measured as reducing the fork() overhead.  To reduce overheads further,
>> it allowed me to use of vfork() without temporarily suspending all the
>> threads.  The commands to run are passed over a unix datagram socket;
>> pipes for stdout / stderr pipes are passed as ancillary data.  This
>> udev-exec accounts for over half the code.  The message passing
>> shouldn't impose too much overhead.  I think udev-exec should be used
>> even if threading has been disabled, otherwise there would be too many
>> ugly #ifdefs.
>>
>>     
> This sounds odd to me; udev-exec still needs to fork() for each child,
> and there's a cost of passing the command over the socket -- so how is
> this cheaper than just fork()ing inside udevd?
>
>   

My guess was that it avoids cloning the mappings for the thread stacks. 
I can test this again and compare the two profiles.  There was
definitely a significant difference, but I should have another look and
find out why.

I also measured an improvement from using vfork().  I don't know how
that horror interacts with threading, and I don't want to find out :).

>> udev-exec also avoids running external programs with extra file
>> descriptors - the problem "close on exec" was designed to solve.  I
>> don't think I could insist on a kernel recent enough to provide "close
>> on exec" support.  Plus I don't think "close on exec" helps with pipes
>> used to communicate with child processes.  Without this, the Ubuntu
>> "watershed" command seemed to complain about extra file descriptors.
>>
>>     
> I see no problem with depending on the very latest kernels; using a
> latest udev version goes hand-in-hand with using a latest kernel.
>
>   

Hmm.  I guess libudev ought to use it either way.  (libudev can test if
the flag is supported; I don't think it needs the new system calls).

> The pipe2() syscall allows you to specify flags for the returned file
> descriptors:
>
> 	pipe2 (&fds, O_CLOEXEC)
>   

Ah!  I see how it's supposed to be used now.

pipe2 (&fds, O_CLOEXEC)
if (fork() = 0) {
   dup2(&fds[WRITE_END], STDOUT_FILENO);
   /* insert call to fcntl to remove O_CLOEXEC */
   exec(program_name);
...

I was missing the call to fcntl().  It's doesn't really work without it :).

>> Lastly, udev-exec helped with implementing timeouts - event threads just
>> call select() with a timeout, waiting to read the exitcode of the
>> command from a pipe.  None of the wait() family of commands include a
>> timeout.  alarm() wouldn't work because it's per-process.  Without
>> udev-exec, I would have to create posix timers with callbacks to signal
>> the correct thread - and disable SA_RESTART before waiting.
>>
>>     
> Ironically, I have a long-standing proposal to fix this ;)  I've just
> got to get around to persuading a kernel developer to do it, or do it
> myself.
>
> The idea was you'd do:
>
>   fd = waitfd (P_PID, pid, 0);
>
>   FD_SET (fd, &readfds);
>
>   select (fd + 1, &readfds, NULL, NULL, &timeout);
>
>   read (fd, &siginfo, sizeof (struct siginfo_t));
>
> ie. a version of waitid() that returns the data through a socket - then
> all daemon main loop events could be done with select() [we already have
> signalfd(), timerfd_*(), etc.)
>   

I saw a similar complaint by Bernstein when I looked up the self-pipe
trick.  "Of course, the Right Thing would be to have fork() return a
file descriptor".  FWIW, wayback says the page was put up in 2000.

Thanks for the comments
Alan