New software: Hotplug2, aimed at early userspace

New software: Hotplug2, aimed at early userspace

[iSteve]

Greetings,
I'd like to present my new application, Hotplug2.

Hotplug2 is aimed at early Linux user space, that is, initramfs or initrd,
possibly also at embedded devices (such as WRT-like routers) or very weak
machines, such as 386/486 (some still use them as workstations).

Hotplug2 connects to the uevent netlink socket and read events. The
application generally operates in two modes, dumb and smart.

In dumb mode, when event arrives, we generally only check if it has MODALIAS
set; if yes, we spawn modprobe and carry on.

In smart mode, simple form of rules are applied. Those rules allow matching of
the variables obtained by the uevent socket and allows execution of
applications, with those variables set as their environmental variables.
Further documentation of rules syntax is available at the homepage of the
project.

Hotplug2 supports cold plugging simply by calling the 'udevtrigger' binary
internally. As udevtrigger is fairly independent on the rest of udev, it can be
easily embedded along with Hotplug2. Given that Hotplug2 is in it's infancy,
this can be considered a minor detail and replaced some time later.

When compiled to support smart mode, Hotplug2 v0.2 has only 14 156 bytes on my
machine (x86, dynamically linked with glibc, gcc 4.1.2) with no optimisation
flags for GCC and 10 768 bytes with -Os.

The advantages over udev are, as I believe:
 * Faster (way faster!) processing of events, because far less actions are
performed. 
 * The size is (obviously) also way smaller, which is a generic advantage;)

The disadvantages would be:
 * Far less actions are performed and far less actions _can_ be performed. For
anything more complex, including creating relevant symlinks, one must have
another application or script handling that. 
 
Therefore, as said earlier, Hotplug2 definitely is not aimed as replacement
for udev on desktop, with the exception of initramfs / initrd.

URL of the project is: http://isteve.depressant.org/hotplug2/

I'd welcome any patches, comments or thoughts whatsoever.
-- 
 -- iSteve

-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid\x120709&bid&3057&dat\x121642
_______________________________________________
Linux-hotplug-devel mailing list  http://linux-hotplug.sourceforge.net
Linux-hotplug-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/linux-hotplug-devel

Re: New software: Hotplug2, aimed at early userspace

[iSteve]

In addition, I have 0.3 released with more rules configurations available.

I've also attempted to compare udev and hotplug2 performance for trivial tasks
(load all modules, create all devices without any specific changes).

It appears that hotplug2 outperforms udev significantly in trivial tasks.

The full benchmark:
http://isteve.depressant.org/hotplug2/benchmark.txt

Thanks in advance for any thoughts and comments.
-- 
 -- iSteve

-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid\x120709&bid&3057&dat\x121642
_______________________________________________
Linux-hotplug-devel mailing list  http://linux-hotplug.sourceforge.net
Linux-hotplug-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/linux-hotplug-devel

Re: New software: Hotplug2, aimed at early userspace

[Aras Vaichas]

iSteve wrote:
> In addition, I have 0.3 released with more rules configurations available.
> 
> I've also attempted to compare udev and hotplug2 performance for trivial tasks
> (load all modules, create all devices without any specific changes).
> 
> It appears that hotplug2 outperforms udev significantly in trivial tasks.
> 
> The full benchmark:
> http://isteve.depressant.org/hotplug2/benchmark.txt
> 
> Thanks in advance for any thoughts and comments.

What is the minimum version of libc required to compile this application? Your
application doesn't compile on my system.

$ ./build.sh
hotplug2.c: In function ‘init_netlink_socket’:
hotplug2.c:165: error: ‘SO_RCVBUFFORCE’ undeclared (first use in this function)
hotplug2.c:165: error: (Each undeclared identifier is reported only once
hotplug2.c:165: error: for each function it appears in.)

A Google search reveals that SO_RCVBUFFORCE is apparently a fairly recent
addition to the kernel. I am using libc-2.3.6 for x86 and libc-2.3.5 for ARM
and neither can compile this application.

You could also try adding this:

#ifndef SO_RCVBUFFORCE
#define SO_RCVBUFFORCE	(33)
#endif

regards,

Aras Vaichas

-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid\x120709&bid&3057&dat\x121642
_______________________________________________
Linux-hotplug-devel mailing list  http://linux-hotplug.sourceforge.net
Linux-hotplug-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/linux-hotplug-devel

Re: New software: Hotplug2, aimed at early userspace

[iSteve]

On Mon, 04 Sep 2006 11:04:48 +1000
Aras Vaichas <arasv@magellan-technology.com> wrote:
> What is the minimum version of libc required to compile this application? Your
> application doesn't compile on my system.

I've built it on this:
$ dpkg-query -W libc6 linux-kernel-headers
libc6   2.3.6.ds1-4
linux-kernel-headers    2.6.17.10-1

> $ ./build.sh
> hotplug2.c: In function ‘init_netlink_socket’:
> hotplug2.c:165: error: ‘SO_RCVBUFFORCE’ undeclared (first use in this
> function) hotplug2.c:165: error: (Each undeclared identifier is reported only
> once hotplug2.c:165: error: for each function it appears in.)
> 
> A Google search reveals that SO_RCVBUFFORCE is apparently a fairly recent
> addition to the kernel. I am using libc-2.3.6 for x86 and libc-2.3.5 for ARM
> and neither can compile this application.
> 
> You could also try adding this:
> 
> #ifndef SO_RCVBUFFORCE
> #define SO_RCVBUFFORCE	(33)
> #endif

Thanks, added, released as 0.4 along with fixing of several bugs in the rules
documentation. Looks like udev has been solving the same issue half a year
ago...

> regards,
> 
> Aras Vaichas

-- 
 -- iSteve

-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid\x120709&bid&3057&dat\x121642
_______________________________________________
Linux-hotplug-devel mailing list  http://linux-hotplug.sourceforge.net
Linux-hotplug-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/linux-hotplug-devel

Re: New software: Hotplug2, aimed at early userspace

[iSteve]

On Mon, 04 Sep 2006 17:09:47 +1000
Aras Vaichas <arasv@magellan-technology.com> wrote:
> I tested 0.4 and it compiles OK with only 1 warning:
> 
> hotplug2.c: In function `main':
> hotplug2.c:244: warning: 'rules' might be used uninitialized in this function
> 
> I can fix by:
> 
> --- hotplug2.c.orig     2006-09-04 17:06:24.000000000 +1000
> +++ hotplug2.c  2006-09-04 17:06:43.000000000 +1000
> @@ -241,7 +241,7 @@
>         int max_child_c = 20;
>         int dumb = 0;
> 
> -       struct rules_t *rules;
> +       struct rules_t *rules = NULL;
>         struct stat statbuf;
>         void *filemap;
>         int rule_fd;
> 

Thanks, applied for not yet released 0.5.

> I'm currently running udev-080 which doesn't have udevtrigger, so I installed
> the udev-098 version of udevtrigger into my system and your hotplug2 seems to
> work OK. My system is a small embedded system so it isn't that much of a speed
> increase because I don't have many devices, but I can at least tell you that
> I've tested it. :)

Great! I have to emphasize that if you're testing, it's important that you kill
the udevd for the time of testing -- otherwise both will try to handle the
given uevents and therefore both will suffer significant slowdown. During the
benchmark, I've attempted to run hotplug2 while having udevd running, hotplug2
scored a little over 10s (ie. approximately 50% slowdown compared to test
without udevd running).

> regards,
> 
> Aras Vaichas
> 
-- 
 -- iSteve

-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid\x120709&bid&3057&dat\x121642
_______________________________________________
Linux-hotplug-devel mailing list  http://linux-hotplug.sourceforge.net
Linux-hotplug-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/linux-hotplug-devel

Re: New software: Hotplug2, aimed at early userspace

[Greg KH]

On Sat, Sep 02, 2006 at 04:30:17PM +0200, iSteve wrote:
> 
> The advantages over udev are, as I believe:
>  * Faster (way faster!) processing of events, because far less actions are
> performed. 
>  * The size is (obviously) also way smaller, which is a generic advantage;)

If you want a small and faster versionof udev, that does less things,
the very first few versions of udev will do just that :)

> The disadvantages would be:
>  * Far less actions are performed and far less actions _can_ be performed. For
> anything more complex, including creating relevant symlinks, one must have
> another application or script handling that. 
>  
> Therefore, as said earlier, Hotplug2 definitely is not aimed as replacement
> for udev on desktop, with the exception of initramfs / initrd.

But people already use udev today with initramfs / initrd.  And then
there's busybox's implementation of a udev-like program, which is about
2 years behind udev's evolution.

I'm all for people working on what they want to work on, but perhaps you
might want to help working with us on making udev fit your requirements?

thanks,

greg k-h

-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid\x120709&bid&3057&dat\x121642
_______________________________________________
Linux-hotplug-devel mailing list  http://linux-hotplug.sourceforge.net
Linux-hotplug-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/linux-hotplug-devel

Re: New software: Hotplug2, aimed at early userspace

[iSteve]

On Tue, 5 Sep 2006 18:27:43 -0700
Greg KH <greg@kroah.com> wrote:
> If you want a small and faster versionof udev, that does less things,
> the very first few versions of udev will do just that :)

I thought the first versions were still using /proc/sys/kernel/hotplug
interface?

> But people already use udev today with initramfs / initrd.  And then
> there's busybox's implementation of a udev-like program, which is about
> 2 years behind udev's evolution.

Yes, I'm aware that people use udev in initramfs/initrd... the point is, I
disagree with it... I think fullblown udev for initramfs/initrd is just a huge
overkill -- not that most people actually care...

> I'm all for people working on what they want to work on, but perhaps you
> might want to help working with us on making udev fit your requirements?

Hmm, I guess I could; I thought it'd be faster (and it'd involve less hassle
while discussing it with people) to write a fully working application, at least
as a proof of concept...

Thanks for your feedback
-- 
 -- iSteve

-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid\x120709&bid&3057&dat\x121642
_______________________________________________
Linux-hotplug-devel mailing list  http://linux-hotplug.sourceforge.net
Linux-hotplug-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/linux-hotplug-devel

Re: New software: Hotplug2, aimed at early userspace

[Kay Sievers]

On Fri, 2006-09-08 at 16:00 +0200, iSteve wrote:
> On Tue, 5 Sep 2006 18:27:43 -0700
> Greg KH <greg@kroah.com> wrote:
> > If you want a small and faster versionof udev, that does less things,
> > the very first few versions of udev will do just that :)
> 
> I thought the first versions were still using /proc/sys/kernel/hotplug
> interface?
> 
> > But people already use udev today with initramfs / initrd.  And then
> > there's busybox's implementation of a udev-like program, which is about
> > 2 years behind udev's evolution.
> 
> Yes, I'm aware that people use udev in initramfs/initrd... the point is, I
> disagree with it... I think fullblown udev for initramfs/initrd is just a huge
> overkill -- not that most people actually care...

Well, if you don't provide complete persistent device-naming, especially
in initramfs - all that can never really solve the problems you have
today. A kernel device name is nothing more than a cookie these days.
Only embedded-like systems may work without that need, but they usually
don't need a hotplug-style setup in initramfs anyway. So the field, that
is left for such a stripped-down project seems very very small to me. :)

> > I'm all for people working on what they want to work on, but perhaps you
> > might want to help working with us on making udev fit your requirements?
> 
> Hmm, I guess I could; I thought it'd be faster (and it'd involve less hassle
> while discussing it with people) to write a fully working application, at least
> as a proof of concept...

Sure, it's perfectly fine to proof an idea that way. Even when I think
your time and effort would be better invested in something that really
replaces an "overkill", or something that offers a solution to a problem
that is not already solved.

Also please don't compare a clone() based udevd in qemu, that does not
give any useful results because fork() is very slow in this simple
emulation environment.

And as stated above, a solution without symlinks, persistent names, and
the ability to plug-in external programs to name your device is not less
"overkill", it is just not usable in the usual setup, also not for a
real world initramfs.

Good luck,
Kay


-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid\x120709&bid&3057&dat\x121642
_______________________________________________
Linux-hotplug-devel mailing list  http://linux-hotplug.sourceforge.net
Linux-hotplug-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/linux-hotplug-devel

Re: New software: Hotplug2, aimed at early userspace

[Greg KH]

On Fri, Sep 08, 2006 at 04:00:42PM +0200, iSteve wrote:
> On Tue, 5 Sep 2006 18:27:43 -0700
> Greg KH <greg@kroah.com> wrote:
> > If you want a small and faster versionof udev, that does less things,
> > the very first few versions of udev will do just that :)
> 
> I thought the first versions were still using /proc/sys/kernel/hotplug
> interface?

Yes they are, but it would be pretty trivial to change that if you so
desired.

> > But people already use udev today with initramfs / initrd.  And then
> > there's busybox's implementation of a udev-like program, which is about
> > 2 years behind udev's evolution.
> 
> Yes, I'm aware that people use udev in initramfs/initrd... the point is, I
> disagree with it... I think fullblown udev for initramfs/initrd is just a huge
> overkill -- not that most people actually care...

Why do you think this?  You need to do /dev stuff there, and you want
persistent names at that point in time too.

Are you running out of space somewhere that makes udev too big for your
initramfs?

> > I'm all for people working on what they want to work on, but perhaps you
> > might want to help working with us on making udev fit your requirements?
> 
> Hmm, I guess I could; I thought it'd be faster (and it'd involve less hassle
> while discussing it with people) to write a fully working application, at least
> as a proof of concept...

Yes, proof of concepts are great.  Maybe I'm missing something, but how
does this instance show how things should be changed in udev?

thanks,

greg k-h

-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid\x120709&bid&3057&dat\x121642
_______________________________________________
Linux-hotplug-devel mailing list  http://linux-hotplug.sourceforge.net
Linux-hotplug-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/linux-hotplug-devel