kbuild: install to a single directory

kbuild: install to a single directory

[Jan Engelhardt]

Hi,



This proposed change will collect all kernel modules in the
single directory, e.g. /lib/modules/2.6.29-rc2/kernel/, without
any further directory structure. About 475 inodes (with
almost-allmodconfig) less are used, which should result
in faster directory traversal (less seeks).

---8<---
parent 53c67f8b25a2e6b8b69ac235372cdda036070cfc (v2.6.29-rc2-22-g53c67f8)
commit 5e61b62737a2184742c4d63c9cdd6c5450ec4034
Author: Jan Engelhardt <jengelh@medozas.de>
Date:   Sat Jan 17 20:58:58 2009 +0100

kbuild: install to a single directory

Most of the directory hierarchy in /lib/modules/$(uname -r)/kernel/
seems redundant to me, so just collapse it all and stuff all files in
that single directory. No module name can be used twice anyway, so no
collision will occur.

Signed-off-by: Jan Engelhardt <jengelh@medozas.de>
---
 scripts/Makefile.modinst |    2 +-
 1 files changed, 1 insertions(+), 1 deletions(-)

diff --git a/scripts/Makefile.modinst b/scripts/Makefile.modinst
index 4c7c163..5b34e82 100644
--- a/scripts/Makefile.modinst
+++ b/scripts/Makefile.modinst
@@ -25,7 +25,7 @@ quiet_cmd_modules_install = INSTALL $@
 INSTALL_MOD_DIR ?= extra
 ext-mod-dir = $(INSTALL_MOD_DIR)$(subst $(patsubst %/,%,$(KBUILD_EXTMOD)),,$(@D))
 
-modinst_dir = $(if $(KBUILD_EXTMOD),$(ext-mod-dir),kernel/$(@D))
+modinst_dir = $(if $(KBUILD_EXTMOD),$(ext-mod-dir),kernel/)
 
 $(modules):
 	$(call cmd,modules_install,$(MODLIB)/$(modinst_dir))
-- 
# Created with git-export-patch

Re: kbuild: install to a single directory

[Michael Tokarev]

Jan Engelhardt wrote:
> Hi,
> 
> This proposed change will collect all kernel modules in the
> single directory, e.g. /lib/modules/2.6.29-rc2/kernel/, without
> any further directory structure. About 475 inodes (with
> almost-allmodconfig) less are used, which should result
> in faster directory traversal (less seeks).

While I agree that current deep-n-wide directory structure is
not very useful.. It still helps in many situations.  For
example, I sometimes need to remember which module name this
particular scsi or network or [s]ata controller needs, and
looking at /lib/modules/$foo/kernel/driver/scsi is usually
sufficient since there isn't many of them in there.  Or,
see (old, compat) -t option to modprobe(8).

In the other words, if to change anything at all, something
in-between is needed instead.  No current deep-n-wide often
single-driver-in-directory structure, and not everything in
one place either.

Maybe add a make variable to tell kbuild to use specified
directory to install modules in this dir to, or to specify
that "this module should go to the parent dir" (eliminates
many driver-in-a-dir cases).

/mjt

Re: kbuild: install to a single directory

[Sam Ravnborg]

On Sun, Jan 18, 2009 at 10:17:16PM +0100, Jan Engelhardt wrote:
> Hi,
> 
> 
> 
> This proposed change will collect all kernel modules in the
> single directory, e.g. /lib/modules/2.6.29-rc2/kernel/, without
> any further directory structure. About 475 inodes (with
> almost-allmodconfig) less are used, which should result
> in faster directory traversal (less seeks).

Hi Jan.

Can you elaborate a little more why this is needed / a good thing.

Also:
-> Is mod-utils compatible with this change
-> Is userland in general compatible with this change

	Sam

Re: kbuild: install to a single directory

[Alan Cox]

On Sun, 18 Jan 2009 22:17:16 +0100 (CET)
Jan Engelhardt <jengelh@medozas.de> wrote:

> Hi,
> 
> 
> 
> This proposed change will collect all kernel modules in the
> single directory, e.g. /lib/modules/2.6.29-rc2/kernel/, without
> any further directory structure. About 475 inodes (with
> almost-allmodconfig) less are used, which should result
> in faster directory traversal (less seeks).

What do your benchmark numbers look like ?

Re: kbuild: install to a single directory

[Jan Engelhardt]

On Monday 2009-01-19 02:24, Sam Ravnborg wrote:
>On Sun, Jan 18, 2009 at 10:17:16PM +0100, Jan Engelhardt wrote:
>> 
>> This proposed change will collect all kernel modules in the
>> single directory, e.g. /lib/modules/2.6.29-rc2/kernel/, without
>> any further directory structure. About 475 inodes (with
>> almost-allmodconfig) less are used, which should result
>> in faster directory traversal (less seeks).
>
>Can you elaborate a little more why this is needed / a good thing.

|[Alan Cox wants the numbers, too]

# File sizes (LZMA'ed RPM payload, but archive contains .ko.gz, so
# it is so-so~~)
-rw-r--r-- 1 31090287 Jan 19 10:51 kernel-2.6.29~rc2.i586.rpm
-rw-r--r-- 1 31031150 Jan 17 23:39 kernel-SingleDir-2.6.29~rc2.i586.rpm

# fs barriers are turned off (got an UPS)
# /tmp is XFS, on a 5400rpm disk on a 2003ish Athlon4 system, 768MB RAM

# installing rpm
md /tmp/test1/var/lib/rpm/;
for ((i = 0; i < 5; ++i)); do
	rm -Rf /tmp/test1/var/lib/rpm/*;
	rpm --root /tmp/test1 --initdb;
	sync; echo 3 >/proc/sys/vm/drop_caches;
	time ( rpm --root /tmp/test1 -ihv kernel-2.6.29~rc2.i586.rpm \
		--nodeps --noscripts; sync; );
done

# kernel-2.6.29~rc2.i586.rpm
real	31.778	33.205	32.076	32.451	33.576
user	6.227	6.274	6.159	6.216	6.290
sys	1.837	1.806	1.888	1.890	1.843

# kernel-SingleDir-2.6.29~rc2.i586.rpm
real	18.666	19.257	19.013	19.257	19.280
user	6.222	6.225	6.232	6.245	6.270
sys	1.900	2.178	2.181	2.183	2.263

# running gzip -d for all .ko.gz
# running depmod
for ((i = 0; i < 5; ++i)); do
	sync; echo 3 >/proc/sys/vm/drop_caches;
	depmod -b /tmp/test1 -r 2.6.29-rc2;
done;

# depmod on kernel
real	32.256	32.242	32.275	32.392	32.529
user	2.004	2.029	2.034	2.051	1.963
sys	0.808	0.842	0.824	0.826	0.847

# depmod on kernel-SingleDir
real	21.062	19.293	19.326	19.411	19.361
user	1.691	1.693	1.660	1.724	1.760
sys	0.761	0.766	0.784	0.767	0.750


>Also:
>-> Is mod-utils compatible with this change

Yes, module-init-tools does not care about the directory layout --
how could it? If it depended such, m-i-t would have to be
adjusted on about every single new subsystem/directory that
is added.

>-> Is userland in general compatible with this change

More so than the .gz stuff, I'd say.