[PATCH] Symlinks for building external modules

[PATCH] Symlinks for building external modules

[Andreas Gruenbacher]

Hi Sam,

modules not in the kernel source tree need to locate both the source tree and 
the object tree (O=). Currently, the /lib/modules/$(uname -r)/build symlink 
is the only reference we have; it historically points to the source tree from 
2.4 times. The following patch changes this as follows (this is what we have 
in the current SUSE tree now):

	/lib/modules/$(uname -r)/source ==> source tree
	/lib/modules/$(uname -r)/build ==> object tree

Both links are required for building external modules with:

	make -C /lib/modules/$(uname -r)/source \
		O=/lib/modules/$(uname -r)/build \
		M=$(pwd)


Index: linux-2.6.7-rc2/Makefile
===================================================================
--- linux-2.6.7-rc2.orig/Makefile
+++ linux-2.6.7-rc2/Makefile
@@ -732,9 +732,10 @@ _modinst_:
 		sleep 1; \
 	fi
 	@rm -rf $(MODLIB)/kernel
-	@rm -f $(MODLIB)/build
+	@rm -f $(MODLIB)/{source,build}
 	@mkdir -p $(MODLIB)/kernel
-	@ln -s $(TOPDIR) $(MODLIB)/build
+	@ln -s $(srctree) $(MODLIB)/source
+	@ln -s $(objtree) $(MODLIB)/build
 	$(Q)$(MAKE) -rR -f $(srctree)/scripts/Makefile.modinst
 
 # If System.map exists, run depmod.  This deliberately does not have a


Thanks,
-- 
Andreas Gruenbacher <agruen@suse.de>
SUSE Labs, SUSE LINUX AG

Re: [PATCH] Symlinks for building external modules

[Måns Rullgård]

Andreas Gruenbacher <agruen@suse.de> writes:

> Hi Sam,
>
> modules not in the kernel source tree need to locate both the source
> tree and the object tree (O=). Currently, the /lib/modules/$(uname
> -r)/build symlink is the only reference we have; it historically
> points to the source tree from 2.4 times. The following patch
> changes this as follows (this is what we have in the current SUSE
> tree now):
>
> 	/lib/modules/$(uname -r)/source ==> source tree
> 	/lib/modules/$(uname -r)/build ==> object tree

This will break the building of all external modules until they are
updated, and break updated modules building against older kernels
unless they check the kernel version in the makefiles..  I suggest
leaving the 'build' link as is, and using a difference name for the
build directory, perhaps 'object'.  This might look confusing, so we
could have a 'source' link as well and remove the 'build' link when
most external modules have been updated.

-- 
Måns Rullgård
mru@kth.se

Re: [PATCH] Symlinks for building external modules

[Sam Ravnborg]

On Thu, Jun 03, 2004 at 07:09:42PM +0200, Måns Rullgård wrote:
> Andreas Gruenbacher <agruen@suse.de> writes:
> 
> > Hi Sam,
> >
> > modules not in the kernel source tree need to locate both the source
> > tree and the object tree (O=). Currently, the /lib/modules/$(uname
> > -r)/build symlink is the only reference we have; it historically
> > points to the source tree from 2.4 times. The following patch
> > changes this as follows (this is what we have in the current SUSE
> > tree now):
> >
> > 	/lib/modules/$(uname -r)/source ==> source tree
> > 	/lib/modules/$(uname -r)/build ==> object tree
> 
> This will break the building of all external modules until they are
> updated, and break updated modules building against older kernels
> unless they check the kernel version in the makefiles..  I suggest
> leaving the 'build' link as is, and using a difference name for the
> build directory, perhaps 'object'.  This might look confusing, so we
> could have a 'source' link as well and remove the 'build' link when
> most external modules have been updated.

The existing external modules are anyway broken when using separate 
directories for source and output directories. So noting lost here.
In the case where the kernel is build in the traditional way
the build and source tree will point to the same place.

So I do not see this patch breaking existing setups, but I see
external modules not being prepared for separate build and source
directories.

Patch looks good to me, and I will forward to Andrew soon.

	Sam

Re: [PATCH] Symlinks for building external modules

[Måns Rullgård]

Sam Ravnborg <sam@ravnborg.org> writes:

> On Thu, Jun 03, 2004 at 07:09:42PM +0200, Måns Rullgård wrote:
>> Andreas Gruenbacher <agruen@suse.de> writes:
>> 
>> > Hi Sam,
>> >
>> > modules not in the kernel source tree need to locate both the source
>> > tree and the object tree (O=). Currently, the /lib/modules/$(uname
>> > -r)/build symlink is the only reference we have; it historically
>> > points to the source tree from 2.4 times. The following patch
>> > changes this as follows (this is what we have in the current SUSE
>> > tree now):
>> >
>> > 	/lib/modules/$(uname -r)/source ==> source tree
>> > 	/lib/modules/$(uname -r)/build ==> object tree
>> 
>> This will break the building of all external modules until they are
>> updated, and break updated modules building against older kernels
>> unless they check the kernel version in the makefiles..  I suggest
>> leaving the 'build' link as is, and using a difference name for the
>> build directory, perhaps 'object'.  This might look confusing, so we
>> could have a 'source' link as well and remove the 'build' link when
>> most external modules have been updated.
>
> The existing external modules are anyway broken when using separate 
> directories for source and output directories.

They work fine if you pass O=/some/path on the make command line.

> So noting lost here.  In the case where the kernel is build in the
> traditional way the build and source tree will point to the same
> place.

True.

> So I do not see this patch breaking existing setups, but I see
> external modules not being prepared for separate build and source
> directories.
>
> Patch looks good to me, and I will forward to Andrew soon.

I'll just have to go fix my kernel modules then.

-- 
Måns Rullgård
mru@kth.se

Re: [PATCH] Symlinks for building external modules

[Jari Ruusu]

Sam Ravnborg wrote:
> On Thu, Jun 03, 2004 at 07:09:42PM +0200, Måns Rullgård wrote:
> > Andreas Gruenbacher <agruen@suse.de> writes:
> > > modules not in the kernel source tree need to locate both the source
> > > tree and the object tree (O=). Currently, the /lib/modules/$(uname
> > > -r)/build symlink is the only reference we have; it historically
> > > points to the source tree from 2.4 times. The following patch
> > > changes this as follows (this is what we have in the current SUSE
> > > tree now):
> > >
> > >     /lib/modules/$(uname -r)/source ==> source tree
> > >     /lib/modules/$(uname -r)/build ==> object tree
> >
> > This will break the building of all external modules until they are
> > updated, and break updated modules building against older kernels
> > unless they check the kernel version in the makefiles..  I suggest
> > leaving the 'build' link as is, and using a difference name for the
> > build directory, perhaps 'object'.  This might look confusing, so we
> > could have a 'source' link as well and remove the 'build' link when
> > most external modules have been updated.
> 
> The existing external modules are anyway broken when using separate
> directories for source and output directories. So noting lost here.

Wrong! Existing external modules build fine when specified an object
directory.

> In the case where the kernel is build in the traditional way
> the build and source tree will point to the same place.
> 
> So I do not see this patch breaking existing setups, but I see
> external modules not being prepared for separate build and source
> directories.

How long have you recommended building external modules like this:

 make -C /lib/modules/`uname -r`/build modules SUBDIRS=`pwd`
  or
 make -C /lib/modules/`uname -r`/build modules M=`pwd`

Now they all have to be changed to:

 make -C /lib/modules/`uname -r`/source modules SUBDIRS=`pwd`
  or
 make -C /lib/modules/`uname -r`/source modules M=`pwd`

And the worst part is that modules that have to support old and new
kernel versions have to first detect which method to use, and
select method that applies to that paricular kernel version.

> Patch looks good to me, and I will forward to Andrew soon.

That patch only breaks stuff. Please use the 'object' symlink
suggested by Måns Rullgård. That does not break anything.

-- 
Jari Ruusu  1024R/3A220F51 5B 4B F9 BB D3 3F 52 E9  DB 1D EB E3 24 0E A9 DD

Re: [PATCH] Symlinks for building external modules

[Sam Ravnborg]

On Fri, Jun 04, 2004 at 08:15:02PM +0300, Jari Ruusu wrote:
> 
> How long have you recommended building external modules like this:
> 
>  make -C /lib/modules/`uname -r`/build modules SUBDIRS=`pwd`
>   or
>  make -C /lib/modules/`uname -r`/build modules M=`pwd`
> 
> Now they all have to be changed to:
> 
>  make -C /lib/modules/`uname -r`/source modules SUBDIRS=`pwd`
>   or
>  make -C /lib/modules/`uname -r`/source modules M=`pwd`

That would not work either.
You need to tell kbuild both where to find the source and the output files.
So you have to specify both -C ... and O=...
KERNEL=/lib/modules/`uname -r`
make -C $KERNEL/source O=$KERNEL/build M=`pwd`

There is no way to do the proposed chang and be backward compatible
when the kernel is build using seperate output directories.

Think of the following situation
/usr/src/linux-2.6.6-xx/  <= kernel src

/lib/modules/linux-2.6.6-xx-smp/build/ <= output files (or a symlink to them)
/lib/modules/linux-2.6.6-xx-smp/source/ <= symlink to kernel src

/lib/modules/linux-2.6.6-xx-up/build/ <= output files (or symlink)
/lib/modules/linux-2.6.6-xx-up/source/ <= symlink to kernel src

/lib/modules/linux-2.6.6-xx-4g/build/ <= output files (or symlink)
/lib/modules/linux-2.6.6-xx-4g/source/ <= symlink to kernel src

Notice they all share the _same_ kernel src.
We just have three different .config files.

If there is a way to be backward compatible with
this feature please demonstrate it.

Plese note that the patch Andreas made did not break existing setups
if a seperate output directory was not used. The only effect
would be an additional symlink to the same dir. (build and source would
be links to the same dir).


Andreas - please expalin why you want build to be a symlink, and not
the directory used when actually building the kernel.
 
	Sam

Re: [PATCH] Symlinks for building external modules

[Måns Rullgård]

Sam Ravnborg <sam@ravnborg.org> writes:

> On Fri, Jun 04, 2004 at 08:15:02PM +0300, Jari Ruusu wrote:
>> 
>> How long have you recommended building external modules like this:
>> 
>>  make -C /lib/modules/`uname -r`/build modules SUBDIRS=`pwd`
>>   or
>>  make -C /lib/modules/`uname -r`/build modules M=`pwd`
>> 
>> Now they all have to be changed to:
>> 
>>  make -C /lib/modules/`uname -r`/source modules SUBDIRS=`pwd`
>>   or
>>  make -C /lib/modules/`uname -r`/source modules M=`pwd`
>
> That would not work either.
> You need to tell kbuild both where to find the source and the output files.
> So you have to specify both -C ... and O=...
> KERNEL=/lib/modules/`uname -r`
> make -C $KERNEL/source O=$KERNEL/build M=`pwd`
>
> There is no way to do the proposed chang and be backward compatible
> when the kernel is build using seperate output directories.

There are many external module packages that assume
/lib/modules/`uname -r`/build to contain the kernel sources.  These
can be built with

  make O=/some/path

If the content of /lib/modules/`uname -r`/build changes this will no
longer work.

> Think of the following situation
> /usr/src/linux-2.6.6-xx/  <= kernel src
>
> /lib/modules/linux-2.6.6-xx-smp/build/ <= output files (or a symlink to them)
> /lib/modules/linux-2.6.6-xx-smp/source/ <= symlink to kernel src
>
> /lib/modules/linux-2.6.6-xx-up/build/ <= output files (or symlink)
> /lib/modules/linux-2.6.6-xx-up/source/ <= symlink to kernel src
>
> /lib/modules/linux-2.6.6-xx-4g/build/ <= output files (or symlink)
> /lib/modules/linux-2.6.6-xx-4g/source/ <= symlink to kernel src
>
> Notice they all share the _same_ kernel src.
> We just have three different .config files.
>
> If there is a way to be backward compatible with
> this feature please demonstrate it.
>
> Plese note that the patch Andreas made did not break existing setups
> if a seperate output directory was not used. The only effect
> would be an additional symlink to the same dir. (build and source would
> be links to the same dir).
>
> Andreas - please expalin why you want build to be a symlink, and not
> the directory used when actually building the kernel.

I can't speak for Andreas, but I prefer to keep my root filesystem as
clean as possible.  Often it's mounted read-only.

-- 
Måns Rullgård
mru@kth.se

Re: [PATCH] Symlinks for building external modules

[Jari Ruusu]

Sam Ravnborg wrote:
> On Fri, Jun 04, 2004 at 08:15:02PM +0300, Jari Ruusu wrote:
> > How long have you recommended building external modules like this:
> >
> >  make -C /lib/modules/`uname -r`/build modules SUBDIRS=`pwd`
> >   or
> >  make -C /lib/modules/`uname -r`/build modules M=`pwd`
> >
> > Now they all have to be changed to:
> >
> >  make -C /lib/modules/`uname -r`/source modules SUBDIRS=`pwd`
> >   or
> >  make -C /lib/modules/`uname -r`/source modules M=`pwd`
> 
> That would not work either.
> You need to tell kbuild both where to find the source and the output files.
> So you have to specify both -C ... and O=...
> KERNEL=/lib/modules/`uname -r`
> make -C $KERNEL/source O=$KERNEL/build M=`pwd`
> 
> There is no way to do the proposed chang and be backward compatible
> when the kernel is build using seperate output directories.

What I and Måns Rullgård are saying is this: A symlink to object directory
is missing. The fix is to add the damn symlink. However, breaking symlink to
source directory at the same time is unacceptable.

In other words:
build  ->  Symlink to kernel sources and build machinery, Makefiles and
           scripts. No change here.
object ->  Symlink to separate object tree. This is new.

Existing functionality when using separate source and object directories is
that only the 'build' symlink exists, and that points to kernel build
machinery. And that 'build' symlink must stay that way.

> Plese note that the patch Andreas made did not break existing setups
> if a seperate output directory was not used. The only effect
> would be an additional symlink to the same dir. (build and source would
> be links to the same dir).

All code that uses the 'build' symlink needs to be changed if Andreas' patch
is merged to mainline. That is quite breakage.

Sam, you are wrong here. I hope you have enough balls to admit that instead
of forcing everyone to deal with the proposed breakage.

-- 
Jari Ruusu  1024R/3A220F51 5B 4B F9 BB D3 3F 52 E9  DB 1D EB E3 24 0E A9 DD

Re: [PATCH] Symlinks for building external modules

[Andreas Gruenbacher]

On Friday 04 June 2004 21:45, Måns Rullgård wrote:
> Sam Ravnborg <sam@ravnborg.org> writes:
> > Andreas - please expalin why you want build to be a symlink, and not
> > the directory used when actually building the kernel.
>
> I can't speak for Andreas, but I prefer to keep my root filesystem as
> clean as possible.  Often it's mounted read-only.

Either way would work, but I prefer to have the actual files below /usr/src.
The /lib directory is meant for files required during boot, whereas /usr may 
be mounted later. If we want to allow both possibilities, we probably want to 
add the following check:

Index: linux-2.6.7-rc2/Makefile
===================================================================
--- linux-2.6.7-rc2.orig/Makefile
+++ linux-2.6.7-rc2/Makefile
@@ -735,7 +735,8 @@ _modinst_:
        @rm -f $(MODLIB)/{source,build}
        @mkdir -p $(MODLIB)/kernel
        @ln -s $(srctree) $(MODLIB)/source
-       @ln -s $(objtree) $(MODLIB)/build
+       @[ $(objtree) -ef  $(MODLIB)/build ] \
+               || ln -s $(objtree) $(MODLIB)/build
        $(Q)$(MAKE) -rR -f $(srctree)/scripts/Makefile.modinst


Next, we may want to add a Makefile to the output directory so that external 
modules will build the same way they did before, without specifying the 
source tree separately:

Index: linux-2.6.7-rc2/Makefile
===================================================================
--- linux-2.6.7-rc2.orig/Makefile
+++ linux-2.6.7-rc2/Makefile
@@ -125,11 +125,19 @@ ifeq ($(skip-makefile),)
 # but instead _all depend on modules
 .PHONY: all
 ifeq ($(KBUILD_EXTMOD),)
-_all: all
+_all: $(objtree)/Makefile all
 else
 _all: modules
 endif

+.PHONY: $(objtree)/Makefile
+$(objtree)/Makefile:
+       $(Q)if [ ! $(srctree) -ef $(objtree) ]; then \
+               ( echo "modules modules_install clean:" ; \
+                 echo -e "\t\$$(MAKE) -C $(srctree) \$$@ O=\$$(CURDIR)" \
+               ) > $(objtree)/Makefile ; \
+       fi
+
 # Make sure we're not wasting cpu-cycles doing locale handling, yet do make
 # sure error messages appear in the user-desired language
 ifdef LC_ALL


Cheers,
-- 
Andreas Gruenbacher <agruen@suse.de>
SUSE Labs, SUSE LINUX AG